Search Results: "esr"

4 December 2014

Chris Lamb: Don't ask your questions in private

(If I've linked you to this page, it is my feeble attempt to provide a more convincing justification.)


I often receive instant messages or emails requesting help or guidance at work or on one of my various programming projects. When asked why they asked privately, the responses vary; mostly along the lines of it simply being an accident, not knowing where else to ask, as well as not wishing to "disturb" others with their bespoke question. Some will be more candid and simply admit that they were afraid of looking unknowledgable in front of others. It is always tempting to simply reply with the answer, especially as helping another human is inherently rewarding unless one is a psychopath. However, one can actually do more good overall by insisting the the question is re-asked in a more public forum.
This is for many reasons. Most obviously, public questions are simply far more efficient as soon as more than one person asks that question the response can be found in a search engine or linked to in the future. These time savings soon add up, meaning that simply more stuff can be done in any given day. After all, most questions are not as unique as people think. Secondly, a private communication cannot be corrected or elaborated on if someone else notices it is incorrect or incomplete. Even this rather banal point is more subtle that it first appears the lack of possible corrections deprives both the person asking and the person responding of the true and correct answer. Lastly, conversations that happen in private are depriving others of the answer as well. Perhaps someone was curious but hadn't got around to asking? Maybe the answer or even the question! contains a clue to solving some other issue. None of this can happen if this is occurs behind closed doors. (There are lots of subtler reasons too in a large organisation or team, simply knowing what other people are curious about can be curiously valuable information.)
Note that this is not as you might immediately suspect simply a way of ensuring that one gets the public recognition or "kudos" from being seen helping others. I wouldn't deny that technical communities work on a gift economy basis to some degree, but to attribute all acts of assistance as "selfish" and value-extracting would be to take the argument too far in the other direction. Saying that, the lure and appeal of public recognition should not be understated and can certainly provide an incentive to elaborate and provide a generally superior response.

More philosophically, there's also something fundamentally "honest" about airing issues in an appropriately public and transparent manner. I feel it promotes a culture of egoless conversations, of being able to admit one's mistakes and ultimately a healthy personal mindset. So please, take care not only in the way you phrase and frame your question, but also consider wider context in which you are asking it. And don't take it too personally if I ask you to re-ask elsewhere...

27 October 2014

Petter Reinholdtsen: First Jessie based Debian Edu released (alpha0)

I am happy to report that I on behalf of the Debian Edu team just sent out this announcement:
The Debian Edu Team is pleased to announce the release of Debian Edu
Jessie 8.0+edu0~alpha0
Debian Edu is a complete operating system for schools. Through its
various installation profiles you can install servers, workstations
and laptops which will work together on the school network. With
Debian Edu, the teachers themselves or their technical support can
roll out a complete multi-user multi-machine study environment within
hours or a few days. Debian Edu comes with hundreds of applications
pre-installed, but you can always add more packages from Debian.
For those who want to give Debian Edu Jessie a try, download and
installation instructions are available, including detailed
instructions in the manual[1] explaining the first steps, such as
setting up a network or adding users. Please note that the password
for the user your prompted for during installation must have a length
of at least 5 characters!
 [1] <URL: https://wiki.debian.org/DebianEdu/Documentation/Jessie >
Would you like to give your school's computer a longer life? Are you
tired of sneaker administration, running from computer to computer
reinstalling the operating system? Would you like to administrate all
the computers in your school using only a couple of hours every week?
Check out Debian Edu Jessie!
Skolelinux is used by at least two hundred schools all over the world,
mostly in Germany and Norway.
About Debian Edu and Skolelinux
===============================
Debian Edu, also known as Skolelinux[2], is a Linux distribution based
on Debian providing an out-of-the box environment of a completely
configured school network. Immediately after installation a school
server running all services needed for a school network is set up just
waiting for users and machines being added via GOsa , a comfortable
Web-UI. A netbooting environment is prepared using PXE, so after
initial installation of the main server from CD or USB stick all other
machines can be installed via the network.  The provided school server
provides LDAP database and Kerberos authentication service,
centralized home directories, DHCP server, web proxy and many other
services.  The desktop contains more than 60 educational software
packages[3] and more are available from the Debian archive, and
schools can choose between KDE, Gnome, LXDE, Xfce and MATE desktop
environment.
 [2] <URL: http://www.skolelinux.org/ >
 [3] <URL: http://people.skolelinux.org/pere/blog/Educational_applications_included_in_Debian_Edu___Skolelinux__the_screenshot_collection____.html >
Full release notes and manual
=============================
Below the download URLs there is a list of some of the new features
and bugfixes of Debian Edu 8.0+edu0~alpha0 Codename Jessie. The full
list is part of the manual. (See the feature list in the manual[4] for
the English version.) For some languages manual translations are
available, see the manual translation overview[5].
 [4] <URL: https://wiki.debian.org/DebianEdu/Documentation/Jessie/Features >
 [5] <URL: http://maintainer.skolelinux.org/debian-edu-doc/ >
Where to get it
---------------
To download the multiarch netinstall CD release (624 MiB) you can use
 * ftp://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~alpha0-CD.iso
 * http://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~alpha0-CD.iso
 * rsync -avzP ftp.skolelinux.org::skolelinux-cd/debian-edu-8.0+edu0~alpha0-CD.iso .
The SHA1SUM of this image is: 361188818e036ce67280a572f757de82ebfeb095
New features for Debian Edu 8.0+edu0~alpha0 Codename Jessie released 2014-10-27
===============================================================================
Installation changes
--------------------
 * PXE installation now installs firmware automatically for the hardware present.
Software updates
----------------
Everything which is new in Debian Jessie 8.0, eg:
 * Linux kernel 3.16.x
 * Desktop environments KDE "Plasma" 4.11.12, GNOME 3.14, Xfce 4.10,
   LXDE 0.5.6 and MATE 1.8 (KDE "Plasma" is installed by default; to
   choose one of the others see manual.)
 * the browsers Iceweasel 31 ESR and Chromium 38 
 * !LibreOffice 4.3.3
 * GOsa 2.7.4
 * LTSP 5.5.4
 * CUPS print system 1.7.5
 * new boot framework: systemd
 * Educational toolbox GCompris 14.07 
 * Music creator Rosegarden 14.02
 * Image editor Gimp 2.8.14
 * Virtual stargazer Stellarium 0.13.0
 * golearn 0.9
 * tuxpaint 0.9.22
 * New version of debian-installer from Debian Jessie.
 * Debian Jessie includes about 42000 packages available for
   installation.
 * More information about Debian Jessie 8.0 is provided in the release
   notes[6] and the installation manual[7].
 [6] <URL: http://www.debian.org/releases/jessie/releasenotes >
 [7] <URL: http://www.debian.org/releases/jessie/installmanual >
Fixed bugs
----------
 * Inserting incorrect DNS information in Gosa will no longer break
   DNS completely, but instead stop DNS updates until the incorrect
   information is corrected (Debian bug #710362)
 * and many others.
Documentation and translation updates
------------------------------------- 
 * The Debian Edu Jessie Manual is fully translated to German, French,
   Italian, Danish and Dutch. Partly translated versions exist for
   Norwegian Bokmal and Spanish.
Other changes
-------------
 * Due to new Squid settings, powering off or rebooting the main
   server takes more time.
 * To manage printers localhost:631 has to be used, currently www:631
   doesn't work.
Regressions / known problems
----------------------------
 * Installing LTSP chroot fails with a bug related to eatmydata about
   exim4-config failing to run its postinst (see Debian bug #765694
   and Debian bug #762103).
 * Munin collection is not properly configured on clients (Debian bug
   #764594).  The fix is available in a newer version of munin-node.
 * PXE setup for Main Server and Thin Client Server setup does not
   work when installing on a machine without direct Internet access.
   Will be fixed when Debian bug #766960 is fixed in Jessie.
See the status page[8] for the complete list.
 [8] <URL: https://wiki.debian.org/DebianEdu/Status/Jessie >
How to report bugs
------------------
<URL: http://wiki.debian.org/DebianEdu/HowTo/ReportBugs >
About Debian
============
The Debian Project was founded in 1993 by Ian Murdock to be a truly
free community project. Since then the project has grown to be one of
the largest and most influential open source projects. Thousands of
volunteers from all over the world work together to create and
maintain Debian software. Available in 70 languages, and supporting a
huge range of computer types, Debian calls itself the universal
operating system.
Contact Information
For further information, please visit the Debian web pages[9] or send
mail to press@debian.org.
 [9] <URL: http://www.debian.org/ >

14 October 2014

Julian Andres Klode: Key transition

I started transitioning from 1024D to 4096R. The new key is available at: https://people.debian.org/~jak/pubkey.gpg and the keys.gnupg.net key server. A very short transition statement is available at: https://people.debian.org/~jak/transition-statement.txt and included below (the http version might get extended over time if needed). The key consists of one master key and 3 sub keys (signing, encryption, authentication). The sub keys are stored on an OpenPGP v2 Smartcard. That s really cool, isn t it? Somehow it seems that GnuPG 1.4.18 also works with 4096R keys on this smartcard (I accidentally used it instead of gpg2 and it worked fine), although only GPG 2.0.13 and newer is supposed to work.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA512
Because 1024D keys are not deemed secure enough anymore, I switched to
a 4096R one.
The old key will continue to be valid for some time, but i prefer all
future correspondence to come to the new one.  I would also like this
new key to be re-integrated into the web of trust.  This message is
signed by both keys to certify the transition.
the old key was:
pub   1024D/00823EC2 2007-04-12
      Key fingerprint = D9D9 754A 4BBA 2E7D 0A0A  C024 AC2A 5FFE 0082 3EC2
And the new key is:
pub   4096R/6B031B00 2014-10-14 [expires: 2017-10-13]
      Key fingerprint = AEE1 C8AA AAF0 B768 4019  C546 021B 361B 6B03 1B00
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAlQ9j+oACgkQrCpf/gCCPsKskgCgiRn7DoP5RASkaZZjpop9P8aG
zhgAnjHeE8BXvTSkr7hccNb2tZsnqlTaiQIcBAEBCgAGBQJUPY/qAAoJENc8OeVl
gLOGZiMP/1MHubKmA8aGDj8Ow5Uo4lkzp+A89vJqgbm9bjVrfjDHZQIdebYfWrjr
RQzXdbIHnILYnUfYaOHUzMxpBHya3rFu6xbfKesR+jzQf8gxFXoBY7OQVL4Ycyss
4Y++g9m4Lqm+IDyIhhDNY6mtFU9e3CkljI52p/CIqM7eUyBfyRJDRfeh6c40Pfx2
AlNyFe+9JzYG1i3YG96Z8bKiVK5GpvyKWiggo08r3oqGvWyROYY9E4nLM9OJu8EL
GuSNDCRJOhfnegWqKq+BRZUXA2wbTG0f8AxAuetdo6MKmVmHGcHxpIGFHqxO1QhV
VM7VpMj+bxcevJ50BO5kylRrptlUugTaJ6il/o5sfgy1FdXGlgWCsIwmja2Z/fQr
ycnqrtMVVYfln9IwDODItHx3hSwRoHnUxLWq8yY8gyx+//geZ0BROonXVy1YEo9a
PDplOF1HKlaFAHv+Zq8wDWT8Lt1H2EecRFN+hov3+lU74ylnogZLS+bA7tqrjig0
bZfCo7i9Z7ag4GvLWY5PvN4fbws/5Yz9L8I4CnrqCUtzJg4vyA44Kpo8iuQsIrhz
CKDnsoehxS95YjiJcbL0Y63Ed4mkSaibUKfoYObv/k61XmBCNkmNAAuRwzV7d5q2
/w3bSTB0O7FHcCxFDnn+tiLwgiTEQDYAP9nN97uibSUCbf98wl3/
=VRZJ
-----END PGP SIGNATURE-----

Filed under: Uncategorized

Julian Andres Klode: Key transition

I started transitioning from 1024D to 4096R. The new key is available at: https://people.debian.org/~jak/pubkey.gpg and the keys.gnupg.net key server. A very short transition statement is available at: https://people.debian.org/~jak/transition-statement.txt and included below (the http version might get extended over time if needed). The key consists of one master key and 3 sub keys (signing, encryption, authentication). The sub keys are stored on an OpenPGP v2 Smartcard. That s really cool, isn t it? Somehow it seems that GnuPG 1.4.18 also works with 4096R keys on this smartcard (I accidentally used it instead of gpg2 and it worked fine), although only GPG 2.0.13 and newer is supposed to work.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA512
Because 1024D keys are not deemed secure enough anymore, I switched to
a 4096R one.
The old key will continue to be valid for some time, but i prefer all
future correspondence to come to the new one.  I would also like this
new key to be re-integrated into the web of trust.  This message is
signed by both keys to certify the transition.
the old key was:
pub   1024D/00823EC2 2007-04-12
      Key fingerprint = D9D9 754A 4BBA 2E7D 0A0A  C024 AC2A 5FFE 0082 3EC2
And the new key is:
pub   4096R/6B031B00 2014-10-14 [expires: 2017-10-13]
      Key fingerprint = AEE1 C8AA AAF0 B768 4019  C546 021B 361B 6B03 1B00
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAlQ9j+oACgkQrCpf/gCCPsKskgCgiRn7DoP5RASkaZZjpop9P8aG
zhgAnjHeE8BXvTSkr7hccNb2tZsnqlTaiQIcBAEBCgAGBQJUPY/qAAoJENc8OeVl
gLOGZiMP/1MHubKmA8aGDj8Ow5Uo4lkzp+A89vJqgbm9bjVrfjDHZQIdebYfWrjr
RQzXdbIHnILYnUfYaOHUzMxpBHya3rFu6xbfKesR+jzQf8gxFXoBY7OQVL4Ycyss
4Y++g9m4Lqm+IDyIhhDNY6mtFU9e3CkljI52p/CIqM7eUyBfyRJDRfeh6c40Pfx2
AlNyFe+9JzYG1i3YG96Z8bKiVK5GpvyKWiggo08r3oqGvWyROYY9E4nLM9OJu8EL
GuSNDCRJOhfnegWqKq+BRZUXA2wbTG0f8AxAuetdo6MKmVmHGcHxpIGFHqxO1QhV
VM7VpMj+bxcevJ50BO5kylRrptlUugTaJ6il/o5sfgy1FdXGlgWCsIwmja2Z/fQr
ycnqrtMVVYfln9IwDODItHx3hSwRoHnUxLWq8yY8gyx+//geZ0BROonXVy1YEo9a
PDplOF1HKlaFAHv+Zq8wDWT8Lt1H2EecRFN+hov3+lU74ylnogZLS+bA7tqrjig0
bZfCo7i9Z7ag4GvLWY5PvN4fbws/5Yz9L8I4CnrqCUtzJg4vyA44Kpo8iuQsIrhz
CKDnsoehxS95YjiJcbL0Y63Ed4mkSaibUKfoYObv/k61XmBCNkmNAAuRwzV7d5q2
/w3bSTB0O7FHcCxFDnn+tiLwgiTEQDYAP9nN97uibSUCbf98wl3/
=VRZJ
-----END PGP SIGNATURE-----

Filed under: Uncategorized

1 March 2014

Michael Prokop: Jenkins on-demand slave selection through labels

Problem description: One of my customers had a problem with their Selenium tests in the Jenkins continuous integration system. While Perl s Test::WebDriver still worked just fine the Selenium tests using Ruby s selenium-webdriver suddenly reported failures. The problem was caused by Debian wheezy s upgrade of the Iceweasel web browser. Debian originally shipped Iceweasel version 17.0.10esr-1~deb7u1 in wheezy, but during a security-update version 24.3.0esr-1~deb7u1 was brought in through the wheezy-security channel. Because the selenium tests are used in an automated fashion in a quite large and long-running build pipeline we immediately rolled back to Iceweasel version 17.0.10esr-1~deb7u1 so everything can continue as expected. Of course we wanted to get the new Iceweasel version up and running, but we didn t want to break the existing workflow while working on it. This is where on-demand slave selection through labels comes in. Basics: As soon as you re using Jenkins slaves you can instruct Jenkins to run a specific project on a particular (slave) node. By attaching labels to your slaves you can also use a label instead of a specific node name, providing more flexibility and scalability (to e.g. avoid problems if a specific node is down or you want to scale to more systems). Then Jenkins decides which of the nodes providing the according label should be considered for job execution. In the following screenshot a job uses the selenium label to restrict its execution to the slaves providing selenium and currently there are two nodes available providing this label: TIP 1: Visiting $JENKINS_SERVER/label/$label/ provides a list of slaves that provide that given $label (as well as list of projects that use $label in their configuration), like:

TIP 2: Execute the following script on $JENKINS_SERVER/script to get a list of available labels of your Jenkins system:
import hudson.model.*
labels = Hudson.instance.getLabels()
labels.each  label -> println label.name  
Solution: In the according customer setup we re using the swarm plugin (with automated Debian deployment through Grml s netscript boot option, grml-debootstrap + Puppet) to automatically connect our Jenkins slaves to Jenkins master without any manual intervention. The swarm plugin allows you to define the labels through the -labels command line option. By using the NodeLabel Parameter plugin we can configure additional parameters in Jenkins jobs: node and label . The label parameter allows us to execute the jobs on the nodes providing the requested label: This is what we can use to gradually upgrade from the old Iceweasel version to the new one by keeping a given set of slaves at the old Iceweasel version while we re upgrading other nodes to the new Iceweasel version (same for the selenium-server version which we want to also control). We can include the version number of the Iceweasel and selenium-server packages inside the labels we announce through the swarm slaves, with something like:
if [ -r /etc/init.d/selenium-server ] ; then
  FLAGS="selenium"
  ICEWEASEL_VERSION="$(dpkg-query --show --showformat='$ Version ' iceweasel)"
  if [ -n "$ICEWEASEL_VERSION" ] ; then
    ICEWEASEL_FLAG="iceweasel-$ ICEWEASEL_VERSION%%.* "
    EXTRA_FLAGS="$EXTRA_FLAGS $ICEWEASEL_FLAG"
  fi
  SELENIUM_VERSION="$(dpkg-query --show --showformat='$ Version ' selenium-server)"
  if [ -n "$SELENIUM_VERSION" ] ; then
    SELENIUM_FLAG="selenium-$ SELENIUM_VERSION%-* "
    EXTRA_FLAGS="$EXTRA_FLAGS $SELENIUM_FLAG"
  fi
fi
Then by using -labels $FLAGS EXTRA_FLAGS in the swarm invocation script we end up with labels like selenium iceweasel-24 selenium-2.40.0 for the slaves providing the Iceweasel v24 and selenium v2.40.0 Debian packages and selenium iceweasel-17 selenium-2.40.0 for the slaves providing Iceweasel v17 and selenium v2.40.0. This is perfect for our needs, because instead of using the selenium label (which is still there) we can configure the selenium jobs that should continue to work as usual to default to the slaves with the iceweasel-17 label now. The development related jobs though can use label iceweasel-24 and fail as often as needed without interrupting the build pipeline used for production. To illustrate this here we have slave selenium-client2 providing Iceweasel v17 with selenium-server v2.40. When triggering the production selenium job it will get executed on selenium-client2, because that s the slave providing the requested labels: Whereas the development selenium job can point to the slaves providing Iceweasel v24, so it will be executed on slave selenium-client1 here: This setup allowed us to work on the selenium Ruby tests while not conflicting with any production build pipeline. By the time I m writing about this setup we ve already finished the migration to support Iceweasel v24 and the infrastructure is ready for further Iceweasel and selenium-server upgrades.

Ben Armstrong: Invisible CSS animations on Iceweasel consuming CPU

Thanks to bernat on #debian @ irc.debian.org for helping me track down this bug and devise a workaround. When working on my wife s netbook, I noticed that when idling on Facebook in iceweasel 24.3.0esr-1, the process was taking far too much CPU. I then retested on a wheezy system with the release iceweasel from mozilla.d.n, which at that time was 26, and I later upgraded to 27. Same problem there, too, on both versions. In fact, it seems the slowdown was amplified by the fact that I was running iceweasel in vnc4server, not the worlds most efficient X implementation. Even with all these versions tested, I have yet to file a Debian bug, as I will need some time on a system where the slowdown is noticeable and I m using a current Debian version. But I wanted to post now to give props to bernat for his help. If you think you have this issue, go read his article linked above, which contains the workaround.

28 February 2014

Michael Prokop: State of the art Debian/wheezy deployments with GRUB and LVM/SW-RAID/Crypto

Moving from Lilo to GRUB, using LVM as default, etc throughout the last years it was time to evaluate how well LVM works without a separate boot partition, possibly also on top of Software RAID. Big disks are asking for partitioning with GPT, just UEFI isn t my default yet, so I m still defaulting to Legacy BIOS for Debian/wheezy (I expect this to change for Debian/jessie and according hardware approaching at my customers). So what we have and want in this demonstration setup: System used for installation:
root@grml ~ # grml-version
grml64-full 2013.09 Release Codename Hefeknuddler [2013-09-27]
Partition setup:
root@grml ~ # parted /dev/sda
GNU Parted 2.3
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel gpt
(parted) mkpart primary 2048s 4095s
(parted) set 1 bios_grub on
(parted) name 1 "BIOS Boot Partition"
(parted) mkpart primary 4096s 100%
(parted) set 2 raid on
(parted) name 2 "SW-RAID / Linux"
(parted) quit
Information: You may need to update /etc/fstab.
Clone partition layout from sda to all the other disks:
root@grml ~ # for f in  b,c,d  ; sgdisk -R=/dev/sd$f /dev/sda
The operation has completed successfully.
The operation has completed successfully.
The operation has completed successfully.
Make sure each disk has its unique UUID:
root@grml ~ # for f in  b,c,d  ; sgdisk -G /dev/sd$f
The operation has completed successfully.
The operation has completed successfully.
The operation has completed successfully.
SW-RAID setup:
root@grml ~ # mdadm --create /dev/md0 --verbose --level=raid5 --raid-devices=4 /dev/sd a,b,c,d 2
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 512K
mdadm: size set to 1465004544K
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@grml ~ #
SW-RAID speedup (system dependent, YMMV):
root@grml ~ # cat /sys/block/md0/md/stripe_cache_size
256
root@grml ~ # echo 16384 > /sys/block/md0/md/stripe_cache_size # 16MB
root@grml ~ # blockdev --getra /dev/md0
6144
root@grml ~ # blockdev --setra 65536 /dev/md0 # 32 MB
root@grml ~ # sysctl dev.raid.speed_limit_max
dev.raid.speed_limit_max = 200000
root@grml ~ # sysctl -w dev.raid.speed_limit_max=9999999999
dev.raid.speed_limit_max = 9999999999
root@grml ~ # sysctl dev.raid.speed_limit_min
dev.raid.speed_limit_min = 1000
root@grml ~ # sysctl -w dev.raid.speed_limit_min=100000
dev.raid.speed_limit_min = 100000
LVM setup:
root@grml ~ # pvcreate /dev/md0
  Physical volume "/dev/md0" successfully created
root@grml ~ # vgcreate homesrv /dev/md0
  Volume group "homesrv" successfully created
root@grml ~ # lvcreate -n rootfs -L4G homesrv
  Logical volume "rootfs" created
root@grml ~ # lvcreate -n bootfs -L1G homesrv
  Logical volume "bootfs" created
Check partition setup + alignment:
root@grml ~ # parted -s /dev/sda print
Model: ATA WDC WD15EADS-00P (scsi)
Disk /dev/sda: 1500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name                 Flags
 1      1049kB  2097kB  1049kB               BIOS Boot Partition  bios_grub
 2      2097kB  1500GB  1500GB               SW-RAID / Linux      raid
root@grml ~ # parted -s /dev/sda unit s print
Model: ATA WDC WD15EADS-00P (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start  End          Size         File system  Name                 Flags
 1      2048s  4095s        2048s                     BIOS Boot Partition  bios_grub
 2      4096s  2930276351s  2930272256s               SW-RAID / Linux      raid
root@grml ~ # gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.5
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 2930277168 sectors, 1.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 212E463A-A4E3-428B-B7E5-8D5785141564
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 2930277134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2797 sectors (1.4 MiB)
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            4095   1024.0 KiB  EF02  BIOS Boot Partition
   2            4096      2930276351   1.4 TiB     FD00  SW-RAID / Linux
root@grml ~ # mdadm -E /dev/sda2
/dev/sda3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 7ed3b741:0774d529:d5a71c1f:cf942f0a
           Name : grml:0  (local to host grml)
  Creation Time : Fri Jan 31 15:26:12 2014
     Raid Level : raid5
   Raid Devices : 4
 Avail Dev Size : 2928060416 (1396.21 GiB 1499.17 GB)
     Array Size : 4392089088 (4188.62 GiB 4497.50 GB)
  Used Dev Size : 2928059392 (1396.21 GiB 1499.17 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : c1e4213b:81822fd1:260df456:2c9926fb
    Update Time : Mon Feb  3 09:41:48 2014
       Checksum : b8af8f6 - correct
         Events : 72
         Layout : left-symmetric
     chunk size : 512k
   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)
root@grml ~ # pvs -o +pe_start
  PV         VG      Fmt  Attr PSize PFree 1st PE
  /dev/md0   homesrv lvm2 a--  4.09t 4.09t   1.50m
root@grml ~ # pvs --units s -o +pe_start
  PV         VG      Fmt  Attr PSize       PFree       1st PE
  /dev/md0   homesrv lvm2 a--  8784175104S 8773689344S   3072S
root@grml ~ # pvs -o +pe_start
  PV         VG      Fmt  Attr PSize PFree 1st PE
  /dev/md0   homesrv lvm2 a--  4.09t 4.09t   1.50m
root@grml ~ # pvs --units s -o +pe_start
  PV         VG      Fmt  Attr PSize       PFree       1st PE
  /dev/md0   homesrv lvm2 a--  8784175104S 8773689344S   3072S
root@grml ~ # vgs -o +pe_start
  VG      #PV #LV #SN Attr   VSize VFree 1st PE
  homesrv   1   2   0 wz--n- 4.09t 4.09t   1.50m
root@grml ~ # vgs --units s -o +pe_start
  VG      #PV #LV #SN Attr   VSize       VFree       1st PE
  homesrv   1   2   0 wz--n- 8784175104S 8773689344S   3072S
Cryptsetup:
root@grml ~ # echo cryptsetup >> /etc/debootstrap/packages
root@grml ~ # cryptsetup luksFormat -c aes-xts-plain64 -s 256 /dev/mapper/homesrv-rootfs
WARNING!
========
This will overwrite data on /dev/mapper/homesrv-rootfs irrevocably.
Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
root@grml ~ # cryptsetup luksOpen /dev/mapper/homesrv-rootfs cryptorootfs
Enter passphrase for /dev/mapper/homesrv-rootfs:
Filesystems:
root@grml ~ # mkfs.ext4 /dev/mapper/cryptorootfs
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=384 blocks
262144 inodes, 1048192 blocks
52409 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1073741824
32 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
root@grml ~ # mkfs.ext4 /dev/mapper/homesrv-bootfs
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=384 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
Install Debian/wheezy:
root@grml ~ # mount /dev/mapper/cryptorootfs /media
root@grml ~ # mkdir /media/boot
root@grml ~ # mount /dev/mapper/homesrv-bootfs /media/boot
root@grml ~ # grml-debootstrap --target /media --password YOUR_PASSWORD --hostname YOUR_HOSTNAME
 * grml-debootstrap [0.57] - Please recheck configuration before execution:
   Target:          /media
   Install grub:    no
   Using release:   wheezy
   Using hostname:  YOUR_HOSTNAME
   Using mirror:    http://http.debian.net/debian
   Using arch:      amd64
   Important! Continuing will delete all data from /media!
 * Is this ok for you? [y/N] y
[...]
Enable grml-rescueboot (to have easy access to rescue ISO via GRUB):
root@grml ~ # mkdir /media/boot/grml
root@grml ~ # wget -O /media/boot/grml/grml64-full_$(date +%Y.%m.%d).iso http://daily.grml.org/grml64-full_testing/latest/grml64-full_testing_latest.iso
root@grml ~ # grml-chroot /media apt-get -y install grml-rescueboot
[NOTE: We're installing a daily ISO for grml-rescueboot here because the 2013.09 Grml release doesn't work for this LVM/SW-RAID setup while newer ISOs are working fine already. The upcoming Grml stable release is supposed to work just fine, so you will be able to choose http://download.grml.org/grml64-full_2014.XX.iso by then. :)] Install GRUB on all disks and adjust crypttab, fstab + initramfs:
root@grml ~ # grml-chroot /media /bin/bash
(grml)root@grml:/# for f in  a,b,c,d  ; do grub-install /dev/sd$f ; done
(grml)root@grml:/# update-grub
(grml)root@grml:/# echo "cryptorootfs /dev/mapper/homesrv-rootfs none luks" > /etc/crypttab
(grml)root@grml:/# echo "/dev/mapper/cryptorootfs / auto defaults,errors=remount-ro 0   1" > /etc/fstab
(grml)root@grml:/# echo "/dev/mapper/homesrv-bootfs /boot auto defaults 0 0" >> /etc/fstab
(grml)root@grml:/# update-initramfs -k all -u
(grml)root@grml:/# exit
Clean unmounted/removal for reboot:
root@grml ~ # umount /media/boot
root@grml ~ # umount /media/
root@grml ~ # cryptsetup luksClose cryptorootfs
root@grml ~ # dmsetup remove homesrv-bootfs
root@grml ~ # dmsetup remove homesrv-rootfs
NOTE: On a previous hardware installation I had to install GRUB 2.00-22 from Debian/unstable to get GRUB working.
Some metadata from different mdadm and LVM experiments seems to have been left and confused GRUB 1.99-27+deb7u2 from Debian/wheezy (I wasn t able to reproduce this issue in my VM demo/test setup).
Just in cause you might experience the following error message, try GRUB >=2.00-22:
  # grub-install --recheck /dev/sda
  error: unknown LVM metadata header.
  error: unknown LVM metadata header.
  /usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/mapper/cryptorootfs.  Check your device.map.
  Auto-detection of a filesystem of /dev/mapper/cryptorootfs failed.
  Try with --recheck.
  If the problem persists please report this together with the output of "/usr/sbin/grub-probe --device-map="/boot/grub/device.map"
  --target=fs -v /boot/grub" to <bug-grub@gnu.org>

30 November 2013

Guido G nther: CrystalHD progress

Following up on my port of the crystalhd plugin to the gstreamer 1.0 api I realized that the CrystalHD repo is pretty dormant. After reading slomo's nice article about GStreamer and hardware integration and a short off list mail exchange I decided to split the GStreamer part out of the CrystalHD repo and to try to get the plugin into gst-plugins-bad. Since the kernel part is already in linux kernel's staging area there would not be much left in the repo except for the libcrystalhd library itself and the firmware blobs. So I split them out as well and started to clean them up a bit by moving it to autoconf/automake, dropping the need for a C++ compiler and adding symbol versioning among other things. So up to know video is still smooth with:
gst-launch-1.0 filesrc location=sample.mp4 ! decodebin ! xvimagesink
after jhbuilding up to gst-plugins-bad. There are #ifdefs for macosx and windows but I doubt they're functional but in case anybody is building libcrystalhd on these these platforms it'd be great to know if it still works. Should these efforts lead to the crystalhd plugin being merged into GStreamer getting the kernel driver out of staging would be a great next step. This blog is flattr enabled.

24 September 2013

Simon Josefsson: BLURB: Software repository metadata convention

As a maintainer of several software packages I often find myself copying text snippets from the README file into different places (savannah, github, freecode, emails, etc). Recently I had a need to generate a list of software packages that included every project s name, brief summary, license and URL. I could have generated that list manually by copying the text from every project s README and COPYING files. However then I would have to maintain my list manually to keep it in sync with all the projects. This easily leads to stale information, so I thought a better approach would be to put the information I needed into each projects source code version control system. The advantage is that the manual work to extract the information may be automated by a script, since the data is in a usable format. I ll explain here how my solution works. To be able to find the information using only the URL to the repository, I needed a filename convention. The filename I chose was BLURB; for the etymology, see Wikipedia s page about Blurb. The data format in the file is similar to normal email headers. An example illustrate the principles well. Below is the BLURB file for one of my projects.
Author: Yubico
Basename: libyubikey
Homepage: http://opensource.yubico.com/yubico-c/
License: BSD-2-Clause
Name: Yubico C low-level Library
Project: yubico-c
Summary: C library for manipulating Yubico YubiKey One-Time Passwords (OTPs)
The format is simple: UTF-8 text with each line starting with a header followed by a colon ( : ), some whitespace, and some content. If a line starts with whitespace, it is a continuation of the previous line s content (trim leading whitespace). The following table describes the fields that I use. I may update this blog post in the future with new fields or improved explanations (for reference, current date is 2013-09-24).
Header Meaning
Project Short identifier for the project (e.g., gcc , emacs )
Name Official name of the project in English
Summary Brief one-line summary of the project s purpose in English
Author Origin of the project
Homepage URL to the project s website
License License keyword, preferrably using one of the SPDX license identifiers
Basename The tarball basename, if different from the project name
Finally some reflection of the solution. After quick design, I thought that I couldn t be the first one with this problem, and I tried to find other similar efforts. I haven t been able to find any standardization effort that have the following properties: The related efforts that I found were SPDX which at first look seemed to offer what I wanted. However on closer examination it failed to deliver on all the requirements above, and appeared to have somewhat different goals. However I found the SPDX license list useful and refer to it. Another effort is Eric S. Raymond s freecode-submit and shipper but its primary focus is to encode information about each release. The design of the BLURB file is clearly influenced by these tools. Another influence has been Debian s specification for machine-readable copyright information. The Free Software Foundation s list of software projects seemed like another candidate, but it doesn t suggest any way to store the information in the upstream project itself. flattr this!

26 April 2013

Petter Reinholdtsen: First alpha release of Debian Edu / Skolelinux based on Debian Wheezy

The Debian Edu / Skolelinux project is still going strong and made its first Wheezy based release today. This is the release announcement: New features for Debian Edu ~7.0.0 alpha0 released 2013-04-26 This is the release notes for for Debian Edu / Skolelinux ~7.0.0 edu alpha0, based on Debian with codename "Wheezy". About Debian Edu and Skolelinux Debian Edu, also known as Skolelinux, is a Linux distribution based on Debian providing an out-of-the box environment of a completely configured school network. Immediatly after installation a school server running all services needed for a school network is set up just waiting for users and machines being added via GOsa , a comfortable Web-UI. A netbooting environment is prepared using PXE, so after initial installation of the main server from CD, DVD or USB stick all other machines can be installed via the network. This is the first test release based on Wheezy (which currently is not released yet). Basically this is an updated and slightly improved version compared to the Squeeze release. Software updates Documentation LDAP related changes Other changes Regressions No updated artwork Where to get it To download the multiarch netinstall CD release you can use The MD5SUM of this image is: c5e773ddafdaa4f48c409c682f598b6c The SHA1SUM of this image is: 25934fabb9b7d20235499a0a51f08ce6c54215f2 How to report bugs http://wiki.debian.org/DebianEdu/HowTo/ReportBugs

27 December 2012

Jamie McClelland: Live video streaming using vp8enc and webm on Debian

The vp8 video codec and webm container format (which are alternates for theora/ogg and h264/flv) are getting lots of attention, especially as more web browsers are supporting html5 - which includes a new video tag that plays video directly in a web page, rather than requiring a plugin. With Google fully behind these new formats (and converting YouTube videos) and even skype using vp8, it seems as though it could supplant h264/flash as the default video codec and container. After years under the boot of the proprietary flash video, this change could usher in a lot of exciting developments. While the shift in formats is also significant for on-demand video, I'm focusing on live streaming in this blog. There are a lot of tools needed to successfully stream a live video. Enough of them support vp8 and webm to create a live video stream, however, not all of the right versions of these tools have landed in Debian. Without any patches or changes, if you try to send a live video stream to icecast using vp8 encoding and webm, you'll get something like this:
0 jamie@animal:~$ gst-launch v4l2src ! video/x-raw,width=320,height=240 ! vp8enc ! webmmux ! shout2send ip=icecast.server port=8000 password=secret mount=/test.webm
WARNING: erroneous pipeline: could not link webmmux0 to shout2send0
1 jamie@animal:~$
I've documented the steps for patching/upgrading gstreamer and icecast2 so you can get a live video stream using vp8 encoding and the webmc container that originates on a Debian wheezy machine using gstreamer and streams to a Debian squeeze server running icecast2. On the workstation, you'll need libshout3 version 2.3.0 or higher, which is at the moment available in experimental and installable without bringing in any dependencies. In addition, you will need to bring in gstreamer1.0 from sid and you will need to rebuild gstreamer1.0-plugins-good after libshout-dev has been installed from experimental. So, begin by adding experimental and sid to your sources list by creating the files /etc/apt/sources.list.d/experimental.list and /etc/apt/sources/lists.d/sid.list with the respective contents:
deb http://mirror.cc.columbia.edu/debian experimental main
and
deb http://mirror.cc.columbia.edu/debian sid main 
deb-src http://mirror.cc.columbia.edu/debian sid main
In order to avoid upgrading your entire machine to sid (experimental packages won't upgrade by default), you'll also need to add the file /etc/apt/preferences.d/sid:
Package: *
Pin: release n=sid
Pin-Priority: 200
Then update and install:
apt-get update
apt-get install -t experimental libshout3
apt-get install gir1.2-clutter-gst-1.0 gir1.2-gst-plugins-base-1.0 gir1.2-gstreamer-1.0 gstreamer0.10-ffmpeg  gstreamer1.0-libav:amd64 gstreamer1.0-plugins-bad:amd64 gstreamer1.0-plugins-base:amd64 gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-good:amd64 gstreamer1.0-pulseaudio:amd64 gstreamer1.0-tools gstreamer1.0-x:amd64 libclutter-gst-1.0-0:amd64 libgstreamer-plugins-bad1.0-0:amd64 libgstreamer-plugins-base1.0-0:amd64 libgstreamer-plugins-base1.0 libgstreamer1.0-0:amd64 libgstreamer1.0
Fortunately, the version of gstreamer1.0-plugins-good in sid includes a fix for a bug patched on November 31, 2012 that allows the use of the webm container. However, for it to work properly gstreamer1.0-plugins-good has to be built against libshout version 2.3.0 or higher (which is in experimental, not sid). Therefore, you have install libshout-dev from experimental and then rebuild gstreamer1.0-plugins-good.
sudo apt-get build-dep gstreamer1.0-plugins-good
sudo apt-get -t experimental install libshout-dev
apt-get source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.0.4
dch -v 1.0.4-1+webmmux
fakeroot debian/rules binary
sudo dpkg -i ../gstreamer1.0-plugins-good_1.0.4-1+webmmux_amd64.deb
Now if you try the command, things should work on the client side. When streaming to an icecast2 server running squeeze, I don't get any errors, however, when opening the streaming page via the icecast URL, there is a play button, which can be clicked, but no picture is shown. That's because the version of Icecast2 in squeeze doesn't support either vp8 or webm. On the server side, you will need icecast2 version 2.4beta or higher. This version hasn't landed anywhere in Debian as of this writing. However, you can build it from source by following these steps (the patches I'm removing have all been applied to the new upstream version).
apt-get source icecast2
wget http://downloads.xiph.org/releases/icecast/icecast-2.4-beta.tar.gz
tar -xzvf icecast-2.4-beta.tar.gz
cp -r icecast2-2.3.2/debian icecast-2.3.99.0/
cd icecast-2.3.99.0/
rm debian/patches/ 1001_autotools_avoid_debian-subdir.patch,1002_add_missing_xspf_file,1003_fix_memory_leak.patch,series 
dch -v 2.3.99.0-1
fakeroot debian/rules binary
When you install it, be sure to replace the icecast2.xml configuration file and update it with your server-specific customizations. Hopefully this version of icecast2 will make it to experimental soon. And now... you can successfully stream via:
gst-launch-1.0 v4l2src ! video/x-raw,width=320,height=240 ! vp8enc ! webmmux !  shout2send ip=icecast.server port=8000 password=secret mount=/test.webm
If you want full audio and video, here's a more complete pipeline: gst-launch-1.0 v4l2src ! queue ! video/x-raw,width=320,height=240 ! tee name=tscreen ! queue ! autovideosink tscreen. ! queue ! videorate ! video/x-raw,framerate=25/2 ! queue ! vp8enc ! queue ! webmmux name=mux pulsesrc ! queue ! audioconvert ! vorbisenc ! queue ! queue ! mux. mux. ! queue ! shout2send ip=hobo port=8000 mount=jamie.webm password=secret Unfortunately, since I'm not sure how to deal with the python2.7 vs python3 dependecies, my elegant mobi python script that creates this pipeline for you using python-gtk as been reduced to a crude bash script. Another item on the todo list is opus audio encoding. gstreamer doesn't seem able to use opus in the webm container (although it seems to work with ogg). In the course of this research I found directions for getting gstreamer-1.0 installed on debian independent of packages installed on the system. If you follow the directions above, these steps should not be necessary. I'm including in case you need the latest sources for another reason. Extra steps I needed were:

12 December 2012

Vincent Sanders: What man's mind can create, man's character can control.

I have a project that required me to programmatically control power to several devices. I have done this before using a Velleman vm8090 board which is relatively easy to control. However they are relatively expensive.

I turned to ebay and found a similar module at a substantially reduced cost. Upon receipt however I discovered that instead of being a simple serial USB interface it presented USB HID and the Debian system I was running it on has loaded the hiddev driver for me but it did not implement any of the standard HID Usage Pages leaving me with no way to control the device.
I did the obligatory
sudo lsusb -d 12bf:ff03 -vvv

Bus 003 Device 019: ID 12bf:ff03  
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x12bf
idProduct 0xff03
bcdDevice 1.00
iManufacturer 1 Matrix Multimedia Ltd.
iProduct 2 Flowcode USB HID
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 41
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 50mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 3 Human Interface Device
bInterfaceSubClass 0 No Subclass
bInterfaceProtocol 0 None
iInterface 0
HID Device Descriptor:
bLength 9
bDescriptorType 33
bcdHID 1.10
bCountryCode 0 Not supported
bNumDescriptors 1
bDescriptorType 34 Report
wDescriptorLength 54
Report Descriptors:
** UNAVAILABLE **
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 5
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x01 EP 1 OUT
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 5
Device Status: 0x0980
(Bus Powered)
This simply showed me what I already knew and surprised me that lsusb did not dump HID report descriptor items. Some searching revealed that teh device had to be unbound so lussb could access the descriptor.

Thus a simple
echo -n 3-1.1.4:1.0   sudo dd of=/sys/bus/usb/drivers/usbhid/unbind
resulted in lussb dumping the descriptor items:

Item(Global): Usage Page, data= [ 0xa0 0xff ] 65440
(null)
Item(Local ): Usage, data= [ 0x01 ] 1
(null)
Item(Main ): Collection, data= [ 0x01 ] 1
Application
Item(Local ): Usage, data= [ 0x02 ] 2
(null)
Item(Main ): Collection, data= [ 0x00 ] 0
Physical
Item(Global): Usage Page, data= [ 0xa1 0xff ] 65441
(null)
Item(Local ): Usage, data= [ 0x03 ] 3
(null)
Item(Local ): Usage, data= [ 0x04 ] 4
(null)
Item(Global): Logical Minimum, data= [ 0x00 ] 0
Item(Global): Logical Maximum, data= [ 0xff 0x00 ] 255
Item(Global): Physical Minimum, data= [ 0x00 ] 0
Item(Global): Physical Maximum, data= [ 0xff ] 255
Item(Global): Report Size, data= [ 0x08 ] 8
Item(Global): Report Count, data= [ 0x08 ] 8
Item(Main ): Input, data= [ 0x02 ] 2
Data Variable Absolute No_Wrap Linear
Preferred_State No_Null_Position Non_Volatile Bitfield
Item(Local ): Usage, data= [ 0x05 ] 5
(null)
Item(Local ): Usage, data= [ 0x06 ] 6
(null)
Item(Global): Logical Minimum, data= [ 0x00 ] 0
Item(Global): Logical Maximum, data= [ 0xff 0x00 ] 255
Item(Global): Physical Minimum, data= [ 0x00 ] 0
Item(Global): Physical Maximum, data= [ 0xff ] 255
Item(Global): Report Size, data= [ 0x08 ] 8
Item(Global): Report Count, data= [ 0x08 ] 8
Item(Main ): Output, data= [ 0x02 ] 2
Data Variable Absolute No_Wrap Linear
Preferred_State No_Null_Position Non_Volatile Bitfield
Item(Main ): End Collection, data=none
Item(Main ): End Collection, data=none

By consulting the device class definitions document I determined the device was using the "Vendor defined" Usage page (0xff00 to 0xffff) so I would definitely have to write a program to control the device.

Linux provides a really easy interface to deal with HID devices called hiddev (gosh, such adventurous naming) which I already had to unbind to get my descriptors decoded so I am fairly sure it works ;-)

The kernel documentation and header for hiddev provide the absolute basic mechanics of the interface but no example code or guidance. The obligatory web search turned up very little and even that had to be retrieved from the internet archive. So It seems I would be forced to work it through myself.

It seems the hiddev interface is orientated around HID devices generating reports which the program is expected to read. Numerous ioctl() are provided so the program can obtain the descriptor information necessary to control and process the received reports.

However in this case we need to be able to send reports to the device, all the descriptor information revealed was that there were eight (Item Report Count = 8) values with eight bits each (Item Report Size = 8) with logical and physical values representing the whole range of the octets.

Fortunately the seller provided a website with some control programs and even source. After some time rummaging through the Visual Basic program I finally found (in FrmMain.vb:2989) that the eight bytes were largely unused and the first was simply a bitmask of the eight relays coil status, set for energised clear for off. With bit 0 controlling relay labelled 1 through to bit 7 for relay 8.

To send a report to a HID device the hiddev interface uses the HIDIOCSREPORT ioctl where the report data is first set using HIDIOCSUSAGE .

The HIDIOCSUSAGE ioctl is passed a hiddev_usage_ref structure which must be initialised with information about the report descriptor identifier (constructed from the Usage Page and Usage as set by the items in the descriptor), the index of the item (named usage) we wish to set in the report (in this case the first which is 0) and the value we actually want to set.

After a great deal of debugging the final program is very short indeed but does the job, my main problem now is that if I switch too many (more than one) relays at once the whole device resets. The scope says the supply rails are behaving very badly when this happens, looks like I need to add a load of capacitance to the power well to stabilise it during the switching events.

Oh and add in the fact Relay 1 LED doest work unless you push on it and I do wonder about the wisdom of the economy in this case. Though yet again Linux makes the software side easy.

21 November 2012

Dirk Eddelbuettel: Rcpp attributes: A simple example 'making pi'

We introduced Rcpp 0.10.0 with a number of very nice new features a few days ago, and the activity on the rcpp-devel mailing list has been pretty responsive which is awesome. But because few things beat a nice example, this post tries to build some more excitement. We will illustrate how Rcpp attributes makes it really easy to add C++ code to R session, and that that code is as easy to grasp as R code. Our motivating example is everybody's favourite introduction to Monte Carlo simulation: estimating . A common method uses the fact the unit circle has a surface area equal to . We draw two uniform random numbers x and y, each between zero and one. We then check for the distance of the corresponding point (x,y) relative to the origin. If less than one (or equal), it is in the circle (or on it); if more than one it is outside. As the first quadrant is a quarter of a square of area one, the area of the whole circle is -- so our first quadrant approximates over four. The following figure, kindly borrowed from Wikipedia with full attribution and credit, illustrates this: Example of simulating pi Now, a vectorized version (drawing N such pairs at once) of this approach is provided by the following R function.
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
And in C++ we can write almost exactly the same function thanks the Rcpp sugar vectorisation available via Rcpp:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double piSugar(const int N)  
    RNGScope scope;		// ensure RNG gets set/reset
    NumericVector x = runif(N);
    NumericVector y = runif(N);
    NumericVector d = sqrt(x*x + y*y);
    return 4.0 * sum(d < 1.0) / N;
 
Sure, there are small differences: C++ is statically typed, R is not. We need one include file for declaration, and we need one instantiation of the RNGScope object to ensure random number draws remain coordinated between the calling R process and the C++ function calling into its (compiled C-code based) random number generators. That way we even get the exact same draws for the same seed. But the basic approach is identical: draw a vector x and vector y, compute the distance to the origin and then obtain the proportion within the unit circle -- which we scale by four. Same idea, same vectorised implementation in C++. But the real key here is the one short line with the [[Rcpp::export]] attribute. This is all it takes (along with sourceCpp() from Rcpp 0.10.0) to get the C++ code into R. The full example (which assumes the C++ file is saved as piSugar.cpp in the same directory) is now:
#!/usr/bin/r
library(Rcpp)
library(rbenchmark)
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
sourceCpp("piSugar.cpp")
N <- 1e6
set.seed(42)
resR <- piR(N)
set.seed(42)
resCpp <- piSugar(N)
## important: check results are identical with RNG seeded
stopifnot(identical(resR, resCpp))
res <- benchmark(piR(N), piSugar(N), order="relative")
print(res[,1:4])
and it does a few things: set up the R function, source the C++ function (and presto: we have a callable C++ function just like that), compute two simulations given the same seed and ensure they are in fact identical -- and proceed to compare the timing in a benchmarking exercise. That last aspect is not even that important -- we end up being almost-but-not-quite twice as fast on my machine for different values of N. The real takeaway here is the ease with which we can get a C++ function into R --- and the new process completely takes care of passing parameters in, results out, and does the compilation, linking and loading. More details about Rcpp attributes are in the new vignette. Now enjoy the .

Update:One somewhat bad typo fixed.

Update:Corrected one background tag.

14 November 2012

Tanguy Ortolo: Upcoming signing parties in France

A seal I am organizing two signing parties in France:

Lyon, November 17th This will be an informal key signing party: just come with your ID documents and paper strips with your public key fingerprint. The signing party will occur at 2012-11-17 12:15+01:00 in the Maison Pour Tous in Lyon, France.

Paris, November 25th This will be a formally organized key signing party. If you want to participate, you should follow the indications given in the event wiki page. Basically:
  1. send me your public key;
  2. print some copies of your key fingerprint for people that did not register;
  3. print the attendees list I shall send you before the event;
  4. come to the signing party with all that, ID documents and a pen.
The signing party will occurr at 2012-11-25 11:45+01:00 in the pita school in Le Kremlin-Bic tre, France.

Tips To print strips with your public key fingerprint, you can use the utility gpg-key2ps from the package signing-party. If you are a user of the CAcert.org SSL certification authority, you can come to these signing parties with some copies of the accreditation form: it will be an opportunity to get or give CAcert.org accreditation.

28 September 2012

Don Armstrong: Switching to KGB from CIA

CIA.vc has unfortunately disappeared, and is unlikely to return any time soon. I personally have decided to switch to KGB, but other alternatives such as FBI and irker exist. To switch, you first need to find or set up a kgb bot. If this is a Debian associated FOSS project, feel free to contact me or join #kgb-devel on irc.oftc.net and ask for someone to allow your project to talk to their bot. Once you've found a bot, we need to set up the client. [I'll talk about bot set up at the end.]

kgb-client configuration Install the kgb-client and kgb-client-git packages. Currently, kgb only supports subversion and git, but I suspect this will be rectified shortly. For git repositories, add a post-receive hook like the following:
#!/bin/sh
tee hooks/reflog   kgb-client --conf /path/to/kgbclient.conf --repository git --git-reflog -
For subversion repositories, add a post-commit hook like the following:
#!/bin/sh
kgb-client --conf /path/to/kgbclient.conf --repository svn "$1" "$2"
Then update the configuration file /path/to/kgbclient.conf:
---
repo-id: my-repository
servers:
 - uri: http://servername:9999/
   password: verysecret
# optional link to a website where the commits are;
# needs newish kgb-client and server
web-link: http://example.com/?p=my-repository;a=commitdiff;h=$ commit 
Then, send the bot owner the password, repo-id, channel, and network you'd like the changes to be reported to.

Configuring kgb-bot The bots just listen to soap requests and if the password matches, sends the commit to the appropriate IRC channel. To set one up, install kgb-bot. Then, enable the bot (set BOT_ENBALED=1 in /etc/default/kgb-bot), and configure the bot's configuration file /etc/kgb-bot/kgb.conf:
---
soap:
  server_addr: 0.0.0.0
  server_port: 9999
  service_name: KGB
queue_limit: 150
log_file: "/var/log/kgb-bot.log"
repositories:
  # just a name to identify it
  my-repository:
    # needs to be the same on the client
    password: verysecret
networks:
  oftc:
    nick: KGB-you
    ircname: KGB bot
    username: kgb
    password: ~
    nickserv_password: yournickservpassword
    server: irc.oftc.net
    port: 6667
  freenode:
    nick: KGB-you
    ircname: KGB bot
    username: kgb
    password: ~
    nickserv_password: yournickservpassword
    server: irc.freenode.net
    port: 6667
channels:
  - name: '#your-channel'
    network: oftc
    repos:
     - your-repo
  - name: '#commits'
    network: freenode
    repos:
     - your-repo
Then start the bot (/etc/init.d/kgb-bot start), and watch as it joins channels and reports your changes! You'll probably actually want to register whatever nick you are using on the networks, etc... but you can figure that out yourself!

28 July 2012

John Goerzen: How to get started programming?

I have been asked for advice from several people recently on how to get started programming, or how to further develop a nascent interest in coding or software engineering. The people asking the questions range in age from about 10 years old to older than me. These are people that, for various reasons, are not very easily able to take computer science courses right now. One would think that, since I ve been doing this for somewhere around a quarter century (oh I do feel old now), that I d be ready to offer up some great advice. And offer some suggestions I have. But I m not convinced they re good ones. I have two main tensions. The first is that I, like many in the communities I tend to hang out in such as Debian s, have a personality that leads me to take a deep dive into details of anything that holds my interest. Whether it s Linux, Haskell, or amateur radio, I want to do more than skim the surface if I m having fun with it. Many people are not like that. They may have a lot of fun programming in Visual Basic, not really caring that other languages are out there. Or some people are not like this yet. I feel unqualified to provide good advice to people that are different from me in that way. To put it a different way: most people don t want to wait 4 years to be useful, and want to start out right away and get better over time (and I was the same way too.) The second is related. I learned programming at a time when, other than BASIC, interpreted languages were not really available to me. (Yes, they were available, but not to me.) I cut my teeth on BASIC, Pascal, and C. Although I rarely use C anymore, I can still drop into it at a moment s notice and be perfectly comfortable. I feel it was a fundamentally valuable experience, and that it would be very hard to become a great programmer without ever having lived and breathed something like C, where memory and pointers must be managed manually. Having said that, it is probably possible to become a good coder without ever having touched C. Here, then, is an edited version of some rambly advice I sent to someone recently, where learning OOP was particularly mentioned. I would welcome your comments and suggestions. I may point people that ask to this post in the future. For simply learning how to write code, Dive Into Python has long been a decent resource, though it may assume more experience than some have. I haven t read them myself, but I ve also heard good things about the How to Think Like a Computer Scientist series from Green Tea Press. They re all available as free PDF downloads, too! Eric S. Raymond s The Art of Unix Programming is another work I ve heard good things about, despite having never read it myself. A quick glance at the table of contents makes me think that even if people don t wind up working on Unix, the lessons and philosophy should be informative. It seems that many Computer Science programs are using Java for the core of their instruction, or even almost exclusively. Whether that is good or bad, I m not completely sure. It certainly gets people into OOP more deeply, but I m a right tool for the job kind of person. Despite the hype, OO like everything else isn t the right tool for every job. It is fine for people to dive straight into OO and become good programmers/engineers. However, I think it would be difficult to become a great programmer/engineer without ever having a solid understanding of a more low-level language, such as C in particular. I did my CS work when it was mostly based in C, and am glad for it. If someone never has to manage memory or pointers, I suspect they will be at a disadvantage in the long run for not being able to understand or work with the system at a more fundamental level. If a person knows C, plus some concepts of OO and Functional Programming (FP), it should be easy to pick up just about any other language out there. I used to think Python was a great first language, but during the 2.x series they added so much fluff and so many special cases that I m less enthusiastic now, though I don t know how much of that got cleaned up in 3.x. I am not too keen on Java as a first language, because too many things that should be simple aren t. I have a fondness for Haskell, and its close relationship to mathematics could make it a great first language or maybe a poor one, depending on your perspective. One other thing I think it s important for good programmers to have experience with all three major models of programming (procedural, OO, functional.) Even if a person winds up working mostly in one universe, knowledge of and experience with the others is important and informative and, in my experience, leads to better algorithms and architecture all around.

15 July 2012

Debian Med: Debian Med Bits: Report from LSM Geneva by Andreas Tille

In this report from LSM 2012 in Geneva I will report about
  1. Medical imaging using Debian
  2. Debian Med packaging workshop
  3. Integration of VistA into Debian
  4. Other interesting talks
Medical imaging using Debian There were about 10 attendees basically upstream developers of medical imaging software. The talk got some attention and the message to include even more medical imaging software into Debian was well percived. Thanks to Mathieu Malaterre there was some live demonstration which was way easier for him as a medical imaging expert than it would have been for me.
Debian Med packaging workshop Due to my advertising in the talk yesterday three students (two of them from one medical imaging project, one from an other project) attended the workshop. Thanks to Axel Beckert who helped me out surviving the challenge to walk on unexplored ground.

The idea of the workshop was to ask the attendees to name a package of their own and just package this. Because two of the attendees were upstream developers of CreaTools we decided to go on for packaging this. After circumeventing some pitfalls in the beginning it went rather smoothly and after about 2.5 hours we were able to commit some initial packaging to the Debian Med Git repository which comes quite close to a ready package (perhaps some split into a library and a development package needs to be done and for sure testing is needed).
Quoting Frederic Cervenansky, upstream of CreaTools
Thanks for your work. Your workshop was very interesting and didactic: a relevant discussion between Claire and me for the future of Creatools has emerged from the difficulties you encountered to package creatools. I will try, before the end of the month, to fully package creatools. And for sure, I will contact the debian-med mailing list.
Integration of VistA into Debian I had the good chance to directly address some issues of Claudio Zaugg the speaker in the talk Implementing open source Health Information Systems in Low- and Middle Income Countries a practical review directly before mine. It turned out that by using Debian packaged software might help simplifying the issues they had in supporting health care workers in Low- and Middle Income Countries.
My talk was partly repeating some basic ideas about Debian Med from the talk on Monday because the audience was completely different. Than I tried to explain in detail how we tried hard to establish good contacts to upstream developers and why this is essential to finalise the goal to include hospital information systems straight into Debian any by doing so open the doors of hospitals for large scale Debian installations.
There is also video recording of this talk. Other interesting talks OpenEMR, a multi-language free open source electronic health record for international use Just discussed the packaging of OpenEMR which is prepared for Debian Med as it can be seen on our tasks page. The contact to the creator of some inofficial package will be established to finalise this task.
OpenFovea : when open-source and biophysical research get married Just another target for Debian Med popped up in this talk to further enhance Debian Med in covering all issues of medical care on one hand and on the other hand helping upstream authors to distribute their code
more effectively.
Collaborative software development for nanoscale physics The talk would have fit very nicely into the Debian Science workshop at ESRF (European Synchrotron Radiation Facility) in Grenoble because it was about ETRF (European Theoretical Radiation Facility). At previous LSM events I had just talked with Yann and the work to include their software into Debian is on its way.
Free software and High Performance Computing This talk was not directly connected to my Debian work but I simply enjoyed to see how "two people" had a really entertaining talk about Top 500 computers. Vittoria, you made my last day at LSM.

4 July 2012

Stefano Zacchiroli: bits from the DPL for June 2012

Monthly DPL bits, fresh from the oven^W^W^W hot from DebConf12, and just posted to d-d-a.
Howdy from DebConf12. It's hot, but it's also time to bother you again with a (not so) brief DPL activity report, this time for June 2012. Time-based freeze: DONE, short freeze: TODO Two highlights for this month. First, you've probably noticed Wheezy is now frozen, YAY. This is huge achievement for the release, but also for the project. It's the first time we do a time-based freeze, and it took some quite heated discussion at the beginning of the release cycle to decide to do this. And we did it properly: respecting the planned month and narrowing down the period later. This exercise has hopefully helped both DDs in their package planning and our upstreams in targeting Wheezy with stable releases of their software. Kudos to the release team for their coordination work! Now we've the second part still TODO: releasing Wheezy, without RC bugs, with a freeze period as short as possible. See the beginning of my last "bits from the DPL" mail for my usual song and dance :-P on how to deliver that, together. DebConf12 A lot of us will attend DebConf12. Enjoy it! ... and take the chance to both have fun and make great plans for Debian's future. But remember that "if it didn't happen on a mailing list, it didn't happen". Not all of us will be lucky enough to attend DebConf (in person or remotely). Make sure that those who don't can take part in your team decisions and get informed of what is going to happen here. Politics Zack's spring tour I spent a significant part of June doing Debian talks ins some sort of "spring tour" between Italy and France. In particular: Many thanks to the organizers of these events for inviting and sponsoring me (as well as other Debian people, in the ESRF case) and for their interest in Debian. Sprints Assets Discussions Some relevant discussions for project evolution has been going on in June and I took part into them. You might want to have a look at them: Misc Cheers.
PS the boring day-to-day activity log for June is available at master:/srv/leader/news/bits-from-the-DPL.txt.201206

9 June 2012

Stefano Zacchiroli: bits from the DPL for May 2012

Just posted to d-d-a, here are the monthly DPL bits.
Dear project members,
here's the periodic report of what has happened in DPL land, this time during May 2012. It's briefer than usual, as this year I've enjoyed the lovely French habit of frequent holidays during the month of May. Highlight First highlight for this month is an invitation to us all. We're now in June and the Wheezy freeze is literally a few days away. The RC bugs count is moving in the right direction, but it's still stellar if we want to ensure a short freeze. And a short freeze is of paramount importance: it'll reduce the time during which we can't implement great plans for the future, increase the "freshness" of software we'll ship with Wheezy, and reduce the inconveniences for those who run the testing suite due to its nice "rolling" feature. So please set out some regular time to do RC bug squashing, by providing patches and doing NMUs. Releasing Wheezy is not something that could be outsourced to the Release Team, it's a collective responsibility that kicks in as soon as our own packages are RC bug free (which they already are, right? :-)) The second highlight is more on the internal structure camp. As mentioned last month, I've discussed with the tech-ctte insisting a bit to set up periodic IRC meetings, to ensure outstanding issues get periodically reviewed. At the end of May the first IRC meeting has happened, and has been very productive. See the minutes. Another one has been scheduled, trying to setup a monthly cadence, for the end of June. Many thanks to all tech-ctte members who have took part in and helped with the meeting organization. Communication I've given an interview to iTWire, answering a number of questions about several past and future Debian challenges. Discussions The ongoing discussion to harmonize packaging of multimedia software between the official Debian archive and the unofficial debian-multimedia.org archive (dmo) has progressed. I've tried to help the two groups reaching an agreement on which packages belong where, so that both duplicate packaging efforts and user inconveniences are minimized. That seems not to have worked and dmo maintainers have simply announced that they will move away from the current domain name to a new one that does not include "debian" in its name. Sprints There will be a Debian Science sprint in June, co-located with the broader Debian Science event organized by European Synchrotron Radiation Facility (ESRF) in Grenoble. I've confirmed my attendance for the opening talk of the conference day. ESRF organizers have kindly sponsored travel for all Debian attendees, many thanks to them! Another sprint will happen next week-end in Paris, this time by the i18n/l10n team. I've approved the corresponding tentative budget for travel sponrship for ~2'000 EUR. Other expenses Hardware replacement plans go on. We've ordered SSDs (for ~3'000 CAD) for recently bought machines meant to replace bugs-mirror, bugs-master, and udd. On the "small emergencies" front, we also had to replace failing disks on wagner (1/2 of alioth), for as little as 100 GBP. Miscellanea Happy Wheezy freeze,
and RC bugs squashing!
PS the boring day-to-day activity log for May is available at master:/srv/leader/news/bits-from-the-DPL.txt.201205

2 June 2012

Mike Hommey: Iceweasel ESR in squeeze-backports

In case this went unnoticed, Iceweasel ESR has been available in squeeze-backports for a few weeks, now. I highly recommend anyone using Iceweasel on the Debian stable release to upgrade to that version, at the very least. Even newer versions are available through the pkg-mozilla archive.

Next.

Previous.