The recent AWS introduction of the Elastic File System gives you an automatic grow-and-shrink capability as an NFS mount, an exciting option that takes away the previous overhead in creating shared block file systems for EC2 instances. However it should be noted that the same auto-management of capacity is not true in the EC2 instance s Elastic Block Store (EBS) block storage disks; sizing (and resizing) is left to the customer. With current 2015 EBS, one cannot simply increase the size of an EBS Volume as the storage becomes full; (as at June 2015) an EBS volume, once created, has fixed size. For many applications, that lack of resize function on its local EBS disks is not a problem; many server instances come into existence for a brief period, process some data and then get Terminated, so long term managment is not needed. However for a long term data store on an instance (instead of S3, which I would recommend looking closely at from a durability and pricing fit), and where I want to harness the capacity to grow (or shrink) disk for my data, then I will need to leverage some slightly more advanced disk management. And just to make life interesting, I wish to do all this while the data is live and in-use, if possible. Enter: Logical Volume Management, or LVM. It s been around for a long, long time: LVM 2 made a debut around 2002-2003 (2.00.09 was Mar 2004) and LVM 1 was many years before that so it s pretty mature now. It s a powerful layer that sits between your raw storage block devices (as seen by the operating system), and the partitions and file systems you would normally put on them. In this post, I ll walk through the process of getting set up with LVM on Debian in the AWS EC2 environment, and how you d do some basic maintenance to add and remove (where possible) storage with minimal interruption. Getting Started First a little prep work for a new Debian instance with LVM. As I d like to give the instance its own ability to manage its storage, I ll want to provision an IAM Role for EC2 Instances for this host. In the AWS console, visit IAM, Roles, and I ll create a new Role I ll name EC2-MyServer (or similar), and at this point I ll skip giving it any actual privileges (later we ll update this). As at this date, we can only associate an instance role/profile at instance launch time. Now I launch a base image Debian EC2 instance launched with this IAM Role/Profile; the root file system is an EBS Volume. I am going to put data that I ll be managing on a separate disk from the root file system. First, I need to get the LVM utilities installed. It s a simple package to install: the lvm2 package. From my EC2 instance I need to get root privileges (sudo -i) and run:

apt update && apt install lvm2

After a few moments, the package is installed. I ll choose a location that I want my data to live in, such as /opt/. I want a separate disk for this task for a number of reasons:

1. Root EBS volumes cannot currently be encrypted using Amazon s Encrypted EBS Volumes at this point in time. If I want to also use AWS encryption option, it ll have to be on a non-root disk. Note that instance-size restrictions also exist for EBS Encrypted Volumes.
2. It s possibly not worth make a snapshot of the Operating System at the same time as the user content data I am saving. The OS install (except the /etc/ folder) can almost entirely be recreated from a fresh install. so why snapshot that as well (unless that s your strategy for preserving /etc, /home, etc).
3. The type of EBS volume that you require may be different for different data: today (Apr 2015) there is a choice of Magnetic, General Purpose 2 (GP2) SSD, and Provisioned IO/s (PIOPS) SSD, each with different costs; and depending on our volume, we may want to select one for our root volume (operating system), and something else for our data storage.
4. I may want to use EBS snapshots to clone the disk to another host, without the base OS bundled in with the data I am cloning.

I will create this extra volume in the AWS console and present it to this host. I ll start by using a web browser (we ll use CLI later) with the EC2 console. The first piece of information we need to know is where my EC2 instance is running. Specifically, the AWS Region and Availability Zone (AZ). EBS Volumes only exist within the one designated AZ. If I accidentally make the volume(s) in the wrong AZ, then I won t be able to connect them to my instance. It s not a huge issue, as I would just delete the volume and try again. I navigate to the Instances panel of the EC2 Console, and find my instance in the list:

A (redacted) list of instance from the EC2 console.

Here I can see I have located an instance and it s running in US-East-1A: that s AZ A in Region US-East-1. I can also grab this with a wget from my running Debian instance by asking the MetaData server:

wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone

The returned text is simply: us-east-1a . Time to navigate to Elastic Block Store , choose Volumes and click Create :

Creating a volume in AWS EC2: ensure the AZ is the same as your instance

You ll see I selected that I wanted AWS to encrypt this and as noted above, at this time that doesn t include the t2 family. However, you have an option of using encryption with LVM where the customer looks after the encryption key see LUKS. What s nice is that I can do both have AWS Encrypted Volumes, and then use encryption on top of this, but I have to manage my own keys with LUKS, and should I lose them, then I can keep all the cyphertext! I deselected this for my example (with a t2.micro), and continue; I could see the new volume in the list as creating , and then shortly afterwards as available . Time to attach it: select the disk, and either right-click and choose Attach , or from the menu at the top of the list, chose Actions -> Attach (both do the same thing).

Attaching a volume to an instance: you ll be prompted for the compatible instances in the same AZ.

At this point in time your EC2 instance will now notice a new disk; you can confirm this with dmesg tail , and you ll see something like:

[1994151.231815]  xvdg: unknown partition table

(Note the time-stamp in square brackets will be different). Previously at this juncture you would format the entire disk with your favourite file system, mount it in the desired location, and be done. But we re adding in LVM here between this raw device, and the filesystem we are yet to make . Marking the block device for LVM Our first operation with LVM is to put a marker on the volume to indicate it s being use for LVM so that when we scan the block device, we know what it s for. It s a really simple command:

pvcreate /dev/xvdg

The device name above (/dev/xvdg) should correspond to the one we saw from the dmesg output above. The output of the above is rather straight forward:

  Physical volume "/dev/xvdg" successfully created

Checking our EBS Volume We can check on the EBS volume which LVM sees as a Physical Volume using the pvs command.

# pvs
  PV  VG  Fmt  Attr PSize PFree
  /dev/xvdg  lvm2 ---  5.00g 5.00g

Here we see the entire disk is currently unused. Creating our First Volume Group Next step, we need to make an initial LVM Volume Group which will use our Physical volume (xvdg). The Volume Group will then contain one (or more) Logical Volumes that we ll format and use. Again, a simple command to create a volume group by giving it its first physical device that it will use:

# vgcreate  OptVG /dev/xvdg
  Volume group "OptVG" successfully created

And likewise we can check our set of Volume Groups with vgs :

# vgs
  VG  #PV #LV #SN Attr  VSize VFree
  OptVG  1  0  0 wz--n- 5.00g 5.00g

The Attribute flags here indicate this is writable, resizable, and allocating extents in normal mode. Lets proceed to make our (first) Logical Volume in this Volume Group:

# lvcreate -n OptLV -L 4.9G OptVG
  Rounding up size to full physical extent 4.90 GiB
  Logical volume "OptLV" created

You ll note that I have created our Logical Volume as almost the same size as the entire Volume Group (which is currently one disk) but I left some space unused: the reason for this comes down to keeping some space available for any jobs that LVM may want to use on the disk and this will be used later when we want to move data between raw disk devices. If I wanted to use LVM for Snapshots, then I d want to leave more space free (unallocated) again. We can check on our Logical Volume:

# lvs
  LV  VG  Attr  LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  OptLV OptVG -wi-a----- 4.90g

The attribytes indicating that the Logical Volume is writeable, is allocating its data to the disk in inherit mode (ie, as the Volume Group is doing), and that it is active. At this stage you may also discover we have a device /dev/OptVG/OptLV, and this is what we re going to format and mount. But before we do, we should review what file system we ll use.

---

Filesystems

Popular Linux file systems

Name	Shrink	Grow	Journal	Max File Sz	Max Vol Sz
btrfs	Y	Y	N	16 EB	16 EB
ext3	Y off-line	Y	Y	2 TB	32 TB
ext4	Y off-line	Y	Y	16 TB	1 EB
xfs	N	Y	Y	8 EB	8 EB
zfs*	N	Y	Y	16 EB	256 ZB

For more details see Wikipedia comparison. Note that ZFS requires 3rd party kernel module of FUSE layer, so I ll discount that here. BTRFS only went stable with Linux kernel 3.10, so with Debian Jessie that s a possibility; but for tried and trusted, I ll use ext4. The selection of ext4 also means that I ll only be able to shrink this file system off-line (unmounted). I ll make the filesystem:

# mkfs.ext4 /dev/OptVG/OptLV
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 1285120 4k blocks and 321280 inodes
Filesystem UUID: 4f831d17-2b80-495f-8113-580bd74389dd
Superblock backups stored on blocks:
  32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

And now mount this volume and check it out:

# mount /dev/OptVG/OptLV /opt/
# df -HT /opt
Filesystem  Type  Size  Used Avail Use% Mounted on
/dev/mapper/OptVG-OptLV ext4  5.1G  11M  4.8G  1% /opt

Lastly, we want this to be mounted next time we reboot, so edit /etc/fstab and add the line:

/dev/OptVG/OptLV /opt ext4 noatime,nodiratime 0 0

With this in place, we can now start using this disk. I selected here not to update the filesystem every time I access a file or folder updates get logged as normal but access time is just ignored. Time to expand After some time, our 5 GB /opt/ disk is rather full, and we need to make it bigger, but we wish to do so without any downtime. Amazon EBS doesn t support resizing volumes, so our strategy is to add a new larger volume, and remove the older one that no longer suits us; LVM and ext4 s online resize ability will allow us to do this transparently. For this example, we ll decide that we want a 10 GB volume. It can be a different type of EBS volume to our original we re going to online-migrate all our data from one to the other. As when we created the original 5 GB EBS volume above, create a new one in the same AZ and attach it to the host (perhaps a /dev/xvdh this time). We can check the new volume is visible with dmesg again:

[1999786.341602]  xvdh: unknown partition table

And now we initalise this as a Physical volume for LVM:

# pvcreate /dev/xvdh
  Physical volume "/dev/xvdh" successfully created

And then add this disk to our existing OptVG Volume Group:

# vgextend OptVG /dev/xvdh
  Volume group "OptVG" successfully extended

We can now review our Volume group with vgs, and see our physical volumes with pvs:

# vgs
  VG  #PV #LV #SN Attr  VSize  VFree
  OptVG  2  1  0 wz--n- 14.99g 10.09g
# pvs
  PV  VG  Fmt  Attr PSize  PFree
  /dev/xvdg  OptVG lvm2 a--  5.00g 96.00m
  /dev/xvdh  OptVG lvm2 a--  10.00g 10.00g

There are now 2 Physical Volumes we have a 4.9 GB filesystem taking up space, so 10.09 GB of unallocated space in the VG. Now its time to stop using the /dev/xvgd volume for any new requests:

# pvchange -x n /dev/xvdg
  Physical volume "/dev/xvdg" changed
  1 physical volume changed / 0 physical volumes not changed

At this time, our existing data is on the old disk, and our new data is on the new one. Its now that I d recommend running GNU screen (or similar) so you can detach from this shell session and reconnect, as the process of migrating the existing data can take some time (hours for large volumes):

# pvmove /dev/sdb1 /dev/sdd1
  /dev/xvdg: Moved: 0.1%
  /dev/xvdg: Moved: 8.6%
  /dev/xvdg: Moved: 17.1%
  /dev/xvdg: Moved: 25.7%
  /dev/xvdg: Moved: 34.2%
  /dev/xvdg: Moved: 42.5%
  /dev/xvdg: Moved: 51.2%
  /dev/xvdg: Moved: 59.7%
  /dev/xvdg: Moved: 68.0%
  /dev/xvdg: Moved: 76.4%
  /dev/xvdg: Moved: 84.7%
  /dev/xvdg: Moved: 93.3%
  /dev/xvdg: Moved: 100.0%

During the move, checking the Monitoring tab in the AWS EC2 Console for the two volumes should show one with a large data Read metric, and one with a large data Write metric clearly data should be flowing off the old disk, and on to the new. A note on disk throughput The above move was a pretty small, and empty volume. Larger disks will take longer, naturally, so getting some speed out of the process maybe key. There s a few things we can do to tweak this:

• EBS Optimised: a launch-time option that reserves network throughput from certain instance types back to the EBS service within the AZ. Depending on the size of the instance this is 500 MB/sec up to 4GB/sec. Note that for the c4 family of instances, EBS Optimised is on by default.
• Size of GP2 disk: the larger the disk, the longer it can sustain high IO throughput but read this for details.
• Size and speed of PIOPs disk: if consistent high IO is required, then moving to Provisioned IO disk may be useful. Looking at the (2 weeks) history of Cloudwatch logs for the old volume will give me some idea of the duty cycle of the disk IO.

Back to the move Upon completion I can see that the disk in use is the new disk and not the old one, using pvs again:

# pvs
  PV  VG  Fmt  Attr PSize  PFree
  /dev/xvdg  OptVG lvm2 ---  5.00g 5.00g
  /dev/xvdh  OptVG lvm2 a--  10.00g 5.09g

So all 5 GB is now unused (compare to above, where only 96 MB was PFree). With that disk not containing data, I can tell LVM to remove the disk from the Volume Group:

# vgreduce OptVG /dev/xvdg
  Removed "/dev/xvdg" from volume group "OptVG"

Then I cleanly wipe the labels from the volume:

# pvremove /dev/xvdg
  Labels on physical volume "/dev/xvdg" successfully wiped

If I really want to clean the disk, I could choose to use shred(1) on the disk to overwrite with random data. This can take a lng time Now the disk is completely unused and disassociated from the VG, I can return to the AWS EC2 Console, and detach the disk:

Detach an EBS volume from an EC2 instance

Wait for a few seconds, and the disk is then shown as available ; I then chose to delete the disk in the EC2 console (and stop paying for it). Back to the Logical Volume it s still 4.9 GB, so I add 4.5 GB to it:

# lvresize -L +4.5G /dev/OptVG/OptLV
  Size of logical volume OptVG/OptLV changed from 4.90 GiB (1255 extents) to 9.40 GiB (2407 extents).
  Logical volume OptLV successfully resized

We now have 0.6GB free space on the physical volume (pvs confirms this). Finally, its time to expand out ext4 file system:

# resize2fs /dev/OptVG/OptLV
resize2fs 1.42.12 (29-Aug-2014)
Filesystem at /dev/OptVG/OptLV is mounted on /opt; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/OptVG/OptLV is now 2464768 (4k) blocks long.

And with df we can now see:

# df -HT /opt/
Filesystem  Type  Size  Used Avail Use% Mounted on
/dev/mapper/OptVG-OptLV ext4  9.9G  12M  9.4G  1% /opt

Automating this The IAM Role I made at the beginning of this post is now going to be useful. I ll start by adding an IAM Policy to the Role to permit me to List Volumes, Create Volumes, Attach Volumes and Detach Volumes to my instance-id. Lets start with creating a volume, with a policy like this:

 
  "Version": "2012-10-17",
  "Statement": [
   
  "Sid": "CreateNewVolumes",
  "Action": "ec2:CreateVolume",
  "Effect": "Allow",
  "Resource": "*",
  "Condition":  
  "StringEquals":  
  "ec2:AvailabilityZone": "us-east-1a",
  "ec2:VolumeType": "gp2"
   ,
  "NumericLessThanEquals":  
  "ec2:VolumeSize": "250"
   
   
   
  ]
 

This policy puts some restrictions on the volumes that this instance can create: only within the given Availability Zone (matching our instance), only GP2 SSD (no PIOPs volumes), and size no more than 250 GB. I ll add another policy to permit this instance role to tag volumes in this AZ that don t yet have a tag called InstanceId:

 
  "Version": "2012-10-17",
  "Statement": [
   
  "Sid": "TagUntaggedVolumeWithInstanceId",
  "Action": [
  "ec2:CreateTags"
  ],
  "Effect": "Allow",
  "Resource": "arn:aws:ec2:us-east-1:1234567890:volume/*",
  "Condition":  
  "Null":  
  "ec2:ResourceTag/InstanceId": "true"
   
   
   
  ]
 

Now that I can create (and then tag) volumes, this becomes a simple procedure as to what else I can do to this volume. Deleting and creating snapshots of this volume are two obvious options, and the corresponding policy:

 
  "Version": "2012-10-17",
  "Statement": [
   
  "Sid": "CreateDeleteSnapshots-DeleteVolume-DescribeModifyVolume",
  "Action": [
  "ec2:CreateSnapshot",
  "ec2:DeleteSnapshot",
  "ec2:DeleteVolume",
  "ec2:DescribeSnapshotAttribute",
  "ec2:DescribeVolumeAttribute",
  "ec2:DescribeVolumeStatus",
  "ec2:ModifyVolumeAttribute"
  ],
  "Effect": "Allow",
  "Resource": "*",
  "Condition":  
  "StringEquals":  
  "ec2:ResourceTag/InstanceId": "i-123456"
   
   
   
  ]
 

Of course it would be lovely if I could use a variable inside the policy condition instead of the literal string of the instance ID, but that s not currently possible. Clearly some of the more important actions I want to take are to attach and detach a volume to my instance:

 
  "Version": "2012-10-17",
  "Statement": [
     
      "Sid": "Stmt1434114682836",
      "Action": [
        "ec2:AttachVolume"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:ec2:us-east-1:123456789:volume/*",
      "Condition":  
        "StringEquals":  
          "ec2:ResourceTag/InstanceID": "i-123456"
         
       
     ,
     
      "Sid": "Stmt1434114745717",
      "Action": [
        "ec2:AttachVolume"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:ec2:us-east-1:123456789:instance/i-123456"
     
  ]
 

Now with this in place, we can start to fire up the AWS CLI we spoke of. We ll let the CLI inherit its credentials form the IAM Instance Role and the polices we just defined.

AZ= wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone 
Region= wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone rev cut -c 2- rev 
InstanceId= wget -q -O - http://169.254.169.254/latest/meta-data/instance-id
VolumeId= aws ec2 --region $ Region  create-volume --availability-zone $ AZ  --volume-type gp2 --size 1 --query "VolumeId" --output text 
aws ec2 --region $ Region  create-tags --resource $ VolumeID  --tags Key=InstanceId,Value=$ InstanceId 
aws ec2 --region $ Region  attach-volume --volume-id $ VolumeId  --instance-id $ InstanceId 

and at this stage, the above manipulation of the raw block device with LVM can begin. Likewise you can then use the CLI to detach and destroy any unwanted volumes if you are migrating off old block devices.

What happened about the reproducible builds effort for this week: Presentations On May 26th,Holger Levsen presented reproducible builds in Debian at CCC Berlin for the Datengarten 52. The presentation was in German and the slides in English. Audio and video recordings are available. Toolchain fixes

• Dmitry Shachnev uploaded pyqt5/5.4.1+dfsg-3 which makes pyuic output imports in stable order. Original patch by Reiner Herrmann.
• Lunar uploaded mozilla-devscripts/0.41 which uses the UTC timezone when calling zip or unzip.

Niels Thykier fixed the experimental support for the automatic creation of debug packages in debhelper that being tested as part of the reproducible toolchain. Lunar added to the reproducible build version of dpkg the normalization of permissions for files in control.tar. The patch has also been submitted based on the main branch. Daniel Kahn Gillmor proposed a patch to add support for externally-supplying build date to help2man. This sparkled a discussion about agreeing on a common name for an environment variable to hold the date that should be used. It seems opinions are converging on using SOURCE_DATE_UTC which would hold a ISO-8601 formatted date in UTC) (e.g. 2015-06-05T01:08:20Z). Kudos to Daniel, Brendan O'Dea, Ximin Luo for pushing this forward. Lunar proposed a patch to Tar upstream adding a --clamp-mtime option as a generic solution for timestamp variations in tarballs which might also be useful for dpkg. The option changes the behavior of --mtime to only use the time specified if the file mtime is newer than the given time. So far, upstream is not convinced that it would make a worthwhile addition to Tar, though. Daniel Kahn Gillmor reached out to the libburnia project to ask for help on how to make ISO created with xorriso reproducible. We should reward Thomas Schmitt with a model upstream trophy as he went through a thorough analysis of possible sources of variations and ways to improve the situation. Most of what is missing with the current version in Debian is available in the latest upstream version, but libisoburn in Debian needs help. Daniel backported the missing option for version 1.3.2-1.1. akira submitted a new issue to Doxygen upstream regarding the timestamps added to the generated manpages. Packages fixed The following 49 packages became reproducible due to changes in their build dependencies: activemq-protobuf, bnfc, bridge-method-injector, commons-exec, console-data, djinn, github-backup, haskell-authenticate-oauth, haskell-authenticate, haskell-blaze-builder, haskell-blaze-textual, haskell-bloomfilter, haskell-brainfuck, haskell-hspec-discover, haskell-pretty-show, haskell-unlambda, haskell-x509-util, haskelldb-hdbc-odbc, haskelldb-hdbc-postgresql, haskelldb-hdbc-sqlite3, hasktags, hedgewars, hscolour, https-everywhere, java-comment-preprocessor, jffi, jgit, jnr-ffi, jnr-netdb, jsoup, lhs2tex, libcolor-calc-perl, libfile-changenotify-perl, libpdl-io-hdf5-perl, libsvn-notify-mirror-perl, localizer, maven-enforcer, pyotherside, python-xlrd, python-xstatic-angular-bootstrap, rt-extension-calendar, ruby-builder, ruby-em-hiredis, ruby-redcloth, shellcheck, sisu-plexus, tomcat-maven-plugin, v4l2loopback, vim-latexsuite. The following packages became reproducible after getting fixed:

• afterstep/2.2.12-6 uploaded by Robert Luberda, original patch by Juan Picca.
• birdfont/2.8.0-2 by Hideki Yamane.
• bzr/2.6.0+bzr6602-1 by Jelmer Vernooij.
• cassiopee/1.0.3+dfsg-2 uploaded by Olivier Sallou, original patch by akira.
• dhcp-helper/1.1-3 by Simon Kelley.
• elog/3.1.0-2-1 by Roger Kalt.
• flashcache/3.1.3+git20150513-1 uploaded by Liang Guo, original patch by Reiner Herrmann.
• fonts-kiloji/1:2.1.0-21 by Hideki Yamane.
• inspircd/2.0.20-2 by Guillaume Delacour.
• jd/1:2.8.9-150226-3 by Hideki Yamane.
• jruby-joni/2.1.6-2 by Hideki Yamane.
• libaqbanking/5.6.0beta-1 by Micha Lenk.
• libencode-hanextra-perl/0.23-4 fixed and uploaded by Niko Tyni.
• libencode-jis2k-perl/0.02-3 by Niko Tyni.
• libjide-oss-java/3.6.9+dfsg-1 by Markus Koschany.
• librpc-xml-perl/0.78-3 upload by Niko Tyni, original patch by Reiner Herrmann.
• libterm-readkey-perl/2.32-2 by Niko Tyni.
• mkgmap-splitter/0.0.0+svn421-1 by Bas Couwenberg.
• mod-authn-webid/0~20110301-3 by Clint Adams, original patch by Chris Lamb.
• ossim/1.8.16-4 uploaded by Bas Couwenberg, original patch by Juan Picca.
• psfex/3.17.1+dfsg-2 by Ole Streicher.
• socket-wrapper/1.1.3-2 uploaded by Laszlo Boszormenyi, original patch by Jelmer Vernooij.
• stiff/2.4.0-2 by Ole Streicher.
• testng/6.9.4-2 by Eugene Zhukov.
• wcwidth/0.1.4-2 by Sebastian Ramacher.

Some uploads fixed some reproducibility issues but not all of them:

• ant/1.9.5-1 by Emmanuel Bourg.
• gcin/2.8.3+dfsg1-2 by ChangZhuo Chen.
• grcompiler/4.2-5 by Hideki Yamane.
• python2.7/2.7.10-2 uploaded by Matthias Klose, based on a patch by Lunar.
• python3.4/3.4.3-7 uploaded by Matthias Klose, based on a patch by Lunar.
• python3.5/3.5.0~b2-1 uploaded by Matthias Klose, based on a patch by Lunar.
• wml/2.0.12ds1-9 by Axel Beckert.

Patches submitted which did not make their way to the archive yet:

• #787327 on vim by Reiner Herrmann: set a constant user and set modified-by option.
• #787650 on lush by Daniel Kahn Gillmor: remove __DATE__ and __TIME__ macros from source.
• #787669 on cloc by Daniel Kahn Gillmor: use time of latest debian/changelog entry in manpage.
• #787675 on ricochet by Daniel Kahn Gillmor: patch configure to allow an external build date and set it to the time of latest debian/changelog entry.
• #787804 on clipper by akira: set HTML_TIMESTAMP to NO in Doxygen configuration.
• #787829 on colobot by akira: set HTML_TIMESTAMP to NO in Doxygen configuration.
• #787865 on fastjet by akira: call Doxygen with HTML_TIMESTAMP set to NO.
• #787916 on cal3d by akira: remove $datetime from the file api_footer.html.
• #787918 on dime by akira: remove $datetime from the file footer.html.

Daniel Kahn Gilmor also started discussions for emacs24 and the unsorted lists in generated .el files, the recording of a PID number in lush, and the reproducibility of ISO images in grub2. reproducible.debian.net Notifications are now sent when the build environment for a package has changed between two builds. This is a first step before automatically building the package once more. (Holger Levsen) jenkins.debian.net was upgraded to Debian Jessie. (Holger Levsen) A new variation is now being tested: $PATH. The second build will be done with a /i/capture/the/path added. (Holger Levsen) Holger Levsen with the help of Alexander Couzens wrote extra job to test the reproducibility of coreboot. Thanks James McCoy for helping with certificate issues. Mattia Rizollo made some more internal improvements. strip-nondeterminism development Andrew Ayer released strip-nondeterminism/0.008-1. This new version fixes the gzip handler so that it now skip adding a predetermined timestamp when there was none. Holger Levsen sponsored the upload. Documentation update The pages about timestamps in manpages generated by Doxygen, GHC .hi files, and Jar files have been updated to reflect their status in upstream. Markus Koschany documented an easy way to prevent Doxygen to write timestamps in HTML output. Package reviews 83 obsolete reviews have been removed, 71 added and 48 updated this week. Meetings A meeting was held on 2015-06-03. Minutes and full logs are available. It was agreed to hold such a meeting every two weeks for the time being. The time of the next meeting should be announced soon.

There s a lot of things I d like to blog about. The last version of OpenStack, the OpenStack Liberty design summit, Kilo in the official jessie-backports repositories, etc. Maybe the most interesting part of this blog post is the last bit at the end, about a major change in the packaging workflow for OpenStack in Debian. Please read on OpenStack release names reminder
Just a reminder to make it easier for the average Debian reader who may know Debian well, but not OpenStack. OpenStack 2014.1, is Icehouse, and is the version in Jessie. 2014.2 is Juno and was released right before the freeze of Jessie. 2015.1.0 is what has been released just right after jessie, on the 30th of April. Liberty, which probably will be called 12 (as this will be the 12th release of OpenStack), and not 2015.2 (this has been discussed in Vancouver), will be released in about 5 months form now. The last summit, in Vancouver, BC, Canada, was the Liberty summit, as the OpenStack conventions are always named after the next release (since we are discussing what we will be doing during the next development cycle). OpenStack 2015.1.0, aka Kilo, release in Debian
5 days after the release of Jessie, OpenStack 2015.1.0, aka Kilo, was released. Since I couldn t upload to unstable during the freeze, I was holding a lot of packages, and when I did upload them, there was about 20 packages of mine in the FTP master s NEW queue. Though, since the DSA want to use OpenStack for the Debian infrastructure, the 20 packages were fast track into Sid, thanks to the work of Paultag (thanks man!). OpenStack Kilo in the official Jessie backports
Previously, I was only uploading OpenStack packages to Debian unstable, and maintaining a non-official Debian repositories for backports to Debian stable. However, for multiple reasons, this wasn t satisfying. Then, after packages migrated to Stretch, I started to upload to Debian backports. And right before the summit, almost everything went in. Only python-pysaml2 was missing (as I discovered too late that version 2.0.0 breaks Keystone which needs version 2.4.0). In fact, the last bits of the Kilo release reached jessie-backports in the middle of the OpenStack Liberty summit. Removal of the Debian install-guide from the official site
As there was not enough efforts working on the documentation, unfortunately, the link to the Debian install-guide has been removed from docs.openstack.org. IMO, this is mostly due to a bad communication between myself and the doc team, and also because one person who promised to work on the Debian side of the install-guide failed to warn everyone that he finally couldn t (as his managers assigned him to something else). I hope this will soon be reverted. During the Vancouver summit, I had the opportunity to discuss with the doc team about re-inclusion of the Debian install-guide. Unfortunately, as they are moving away from the XML source format to a more standard RST-based system, the current documentation is frozen, so it seems more realistic to hold on until all of the install-guide is switched to RST. OpenStack Debian image listed on apps.openstack.org
There s a new area on the openstack.org where images and apps for OpenStack are listed. Under the glance image tab, you will see that both the Jessie and the weekly testing image are listed. There s also a nice, easily identifiable Debian logo to link to these images. Also, as there are trademark problems with the Ubuntu images which makes them harder to redistribute, the Murano project (which is shipping a system to automatically install apps that to installed within a few clicks on an OpenStack cloud) decided to switch to Debian for their base image. Debian listed in the OpenStack market place
On the openstack.org site, there s a section called Marketplace. In there, vendors supporting OpenStack are listed. To get there, a vendor needs to 1/ have a defined set of OpenStack project supported by the distribution (Debian already has a way more than the required set), 2/ sign some kind of agreement with the OpenStack foundation, and 3/ pay some sponsoring money. During the summit, I discussed this with Jonathan Bryce, from the OpenStack foundation, and he agreed that Debian would not have to pay for this (since we aren t a big company with big money). I have put Jonathan and Neil (our Debian Project Leader) in touch so that signing the document may happen, though since we were all busy with the summit, I do not expect Jonathan to send the documents right away. Hopefully, this will be fixed before the end of this month of May 2015. Debian (and Ubuntu) packages collaboratively maintained upstream
Since about forever (forever is 5 years in the OpenStack world ), I pushed for more collaboration on OpenStack packaging between Debian package maintainers and Canonical. However, for some reasons which I do not wish to expand on in this blog post, it has been socially hard to do so. Also, Canonical always used BZR, which wasn t to the tastes of everyone. But during the Liberty summit, some very good things happened. First of all, Launchpad is now able to support Git (it s been a few weeks it does in fact). Even though it will take a bit of time before the Canonical server team switches to it, we can consider that this problem is already out of the way. Then it looks like Canonical are now more open than before for collaboration with Debian on the OpenStack packaging. Note that we actually did some work together already, but now we both would like a full alignment of *all* of our packages. I have discussed this with James Page, who is the head of Canonical s server team. We will first start to do so on the dependencies: this includes all of the python-*client libraries, but also all of python-oslo.* (the Oslo libs are use by all of the projects and are kind of unifying the project), plus all the third party dependencies the project relies on. James already pushed new versions of some Oslo libraries to Experimental (in order to not overwrite Kilo), which are adding transition packages needed for Ubuntu. We wont need those in Debian, but we want to welcome them to keep the same source packages. We will then later try to merge the core projects if we can. Unfortunately, since the packaging of the core projects (ie: Nova, Neutron, Cinder, Glance, etc.) was forked, merging probably will be a bit painful. We will have to make some decisions on how this happen. I am however confident that it will be done during the Liberty release cycle. Move of the packaging to upstream Gerrit
A few weeks after the summit, I wrote a proposal to upstream OpenStack dev list, with as subject: Adding packaging as an OpenStack project . What it means is that I have proposed to have Debian/Ubuntu packaging to happen in upstream infrastructure, using Gerrit, and building packages using upstream cloud. We will add all the tests we can, like building with unit tests, lintian, piuparts, adequate, but also maybe a full installation of the packages with functional tests. My proposal is here: http://lists.openstack.org/pipermail/openstack-dev/2015-May/064848.html As everything, this translates into a Gerrit review process: https://review.openstack.org/#/c/185187/ As you can read in the above thread, Fedora/RDO people, which have used a Gerrit work-flow for a long time already, also would like to join. But it looks like we ll be doing 2 teams: one for RPMs and one for debs. The proposal is currently under review by the OpenStack technical committee, which will accept (or not) if the packaging project can be fully considered as an OpenStack project. I expect a final answer next Tuesday. Note that if they deny, we can still use the stackforge namespace instead, their decision is just about the TC blessing the project as being OpenStack or not. What s very nice about this, is that not only we will have a better collaboration between Debian & Ubuntu, better automated testing and Q/A, this also opens contributions to potentially anyone. Especially, we welcome operation people, those who are doing actual big deployments. Sure, it was possible before, but I often had the feedback that many were scared to break anything when trying to contribute. Thanks to the CI/CD form upstream infra, and the Gerrit peer review process, it wont be a problem anymore. So we do expect operation people to contribute more. I will also push more upstream packaging within Mirantis, so that MOS (Mirantis OpenStack) aligns fully with Debian & Ubuntu as well. Another good thing, is that it will be easier for the puppet team to support Debian (they historically were more Ubuntu oriented), and it s going to be super easy for them to request for packaging fixes. I hope we will be able to work hand-to-hand with them, adding puppet deployment checks in the packaging repo, and packaged deployments within the puppet Gerrit review process.

Earlier this week I allowed some of my colleagues at work to try the Kimchi and one of them (James) liked it enough to ask for a tub of it just for himself. Being a lovely person I obliged. My darling Rob decided that what he really wanted was Kimchiguk or Kimchijijae. Being both wonderful and lacking in ingredients, I set to do achieving something. I read a few recipes, worked out that I lacked critical ingredients and so set about simulating Kimchiguk. I grabbed two and a bit cupfuls of kimchi from the box in the fridge, chopped it up more finely, popped it into a pot along with about a pound of cubed pork belly, two teaspoons of hot pepper flakes, two teaspoons of sugar, and two teaspoons of cornflour. I then topped that off with five cups of water, mixed it up, brought it to the boil and simmered it for 30 minutes. Then I cubed some tofu, popped that in and simmered it for another 10 minutes while some rice cooked. Served in a bowl over rice, my approximation of Kimchiguk turned out pretty well. (Note it fed both of us and there was a serving left over.) With a do-over, I'd try harder to get pork shoulder (belly is quite fatty) and given the option I'd wait until I had hot pepper paste because the malty fermented sweetness of hot pepper paste is pretty much impossible to emulate otherwise.

Is the locked books room at McMahon Hall at UW a metaphor for DRM in the academy? Could it be, like so many things in Seattle, sponsored by Amazon? Mika noticed the room several weeks ago but felt that today s International Day Against DRM was a opportune time to raise the questions in front of a wider audience.

For the 2015 International Day Against DRM, I wrote a short essay on DRM for streaming services posted on the Defective by Design website. I m republishing it here. Between 2003 and 2009, most music purchased through Apple s iTunes store was locked using Apple s FairPlay digital restrictions management (DRM) software, which is designed to prevent users from copying music they purchased. Apple did not seem particularly concerned by the fact that FairPlay was never effective at stopping unauthorized distribution and was easily removed with publicly available tools. After all, FairPlay was effective at preventing most users from playing their purchased music on devices that were not made by Apple. No user ever requested FairPlay. Apple did not build the system because music buyers complained that CDs purchased from Sony would play on Panasonic players or that discs could be played on an unlimited number of devices (FairPlay allowed five). Like all DRM systems, FairPlay was forced on users by a recording industry paranoid about file sharing and, perhaps more importantly, by technology companies like Apple, who were eager to control the digital infrastructure of music distribution and consumption. In 2007, Apple began charging users 30 percent extra for music files not processed with FairPlay. In 2009, after lawsuits were filed in Europe and the US, and after several years of protests, Apple capitulated to their customers complaints and removed DRM from the vast majority of the iTunes music catalog. Fundamentally, DRM for downloaded music failed because it is what I ve called an antifeature. Like features, antifeatures are functionality created at enormous cost to technology developers. That said, unlike features which users clamor to pay extra for, users pay to have antifeatures removed. You can think of antifeatures as a technological mob protection racket. Apple charges more for music without DRM and independent music distributors often use DRM-free as a primary selling point for their products. Unfortunately, after being defeated a half-decade ago, DRM for digital music is becoming the norm again through the growth of music streaming services like Pandora and Spotify, which nearly all use DRM. Impressed by the convenience of these services, many people have forgotten the lessons we learned in the fight against FairPlay. Once again, the justification for DRM is both familiar and similarly disingenuous. Although the stated goal is still to prevent unauthorized copying, tools for stripping DRM from services continue to be widely available. Of course, the very need for DRM on these services is reduced because users don t normally store copies of music and because the same music is now available for download without DRM on services like iTunes. We should remember that, like ten years ago, the real effect of DRM is to allow technology companies to capture value by creating dependence in their customers and by blocking innovation and competition. For example, DRM in streaming services blocks third-party apps from playing music from services, just as FairPlay ensured that iTunes music would only play on Apple devices. DRM in streaming services means that listening to music requires one to use special proprietary clients. For example, even with a premium account, a subscriber cannot listen to music from their catalog using an alternative or modified music player. It means that their television, car, or mobile device manufacturer must cut deals with their service to allow each paying customer to play the catalog they have subscribed to. Although streaming services are able to capture and control value more effectively, this comes at the cost of reduced freedom, choice, and flexibility for users and at higher prices paid by subscribers. A decade ago, arguments against DRM for downloaded music focused on the claim that users should have control over the music they purchase. Although these arguments may not seem to apply to subscription services, it is worth remembering that DRM is fundamentally a problem because it means that we do not have control of the technology we use to play our music, and because the firms aiming to control us are using DRM to push antifeatures, raise prices, and block innovation. In all of these senses, DRM in streaming services is exactly as bad as FairPlay, and we should continue to demand better.

Am I missing something? I installed docker and noticed that it created a virtual interface named docker0 with the IP address 172.17.42.1. This behavior is consistent with the Docker networking documentation. However, I was confused by this statement:

It randomly chooses an address and subnet from the private range defined by RFC 1918 that are not in use on the host machine, and assigns it to docker0. Docker made the choice 172.17.42.1/16 when I started it a few minutes ago...

It seems like RFC 1918 defines:

• 10.0.0.0 - 10.255.255.255 (10/8 prefix)
• 172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
• 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)

How is 172.17.42.1/16 from the private ranges listed above? Is 172.17.42.1 a potentially public IP address?

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 26.25 hours on Debian LTS. In that time I did the following:

• CVE triage: I pushed 52 commits to the security tracker. I finished a new helper script (bin/lts-cve-triage.py) that builds on the JSON output that Holger implemented recently. It helps to triage more quickly some issues based on the triaging work already done by the Debian Security team.
• I filed #783005 to clarify the situation of libhtp and suricata in unstable (discovered this problem while triaging issues affecting those packages).
• I reviewed and sponsored DLA-197-1 for Nguyen Cong fixing 5 CVE on libvncserver.
• I released DLA-199-1 fixing one CVE on libx11. I also used codesearch.debian.net to identify all packages that had to be rebuilt with the fixed macro and uploaded them all (there was 11 of them).
• I sponsored DLA-207-1 for James McCoy fixing 7 CVE on subversion.
• I released DLA-210-1 fixing 5 CVE on qt4-x11.
• I released DLA-213-1 fixing 7 CVE on openjdk-6.
• I released DLA-214-1 fixing 1 CVE on libxml-libxml-perl.
• I released DLA-215-1 fixing 1 CVE on libjson-ruby. This backport was non-trivial but luckily included some non-regression tests.
• I filed #783800 about the security-tracker not handling correctly squeeze-lts/non-free.

Now, still related to Debian LTS, but on unpaid hours I did quite a few other things:

• I wrote a talk on Debian LTS that I gave during the Mini-DebConf in Lyon. I took quite some time to collect some statistics about the last 10 months of work within the team.
• I helped to draft a press release announcing our plans for Wheezy LTS and seeking more help at the same time.
• I ensured that the Jessie press release will include a sentence saying that it would be supported for 5 years too.

Other Debian work Feature request in update-alternatives. After a discussion with Josselin Mouette during the Mini-DebConf in Lyon, I filed #782493 to request the possibility to override at a system-wide level the default priority of alternatives recorded in update-alternatives. This would make it easier for derivatives to make different choices than Debian. Sponsored a dnsjava NMU. This NMU introcuded a new upstream version which is needed by jitsi. And I also notified the MIA team that the dnsjava maintainers have disappeared. python-crcmod bug fix and uploads to *-backports. A member of the Google Cloud team wanted this package (with its C extension) to be available to Wheezy users so I NMUed the package in unstable (to fix #782379) and prepared backports for wheezy-backports and jessie-backports (the latter only once the release team rejected a fix in jessie proper, see #782766). Old and new PTS updates for Jessies s release. I took care to update tracker.debian.org and packages.qa.debian.org to take into account Jessie s release (which, most notably, introduced the oldoldstable suite as the new name for Squeeze until its end of life). Received thanks with pleasure. This is not something that I did but I enjoyed reading so many spontaneous thanks in response to Guillem s terse and thankless notification of me stepping down from dpkg maintenance. I love the Debian community. Thank you. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let's see if we can keep everyone up-to-date on the developments. Before the release of Jessie The story goes back a long way but a formal announcement to the project has only been sent in February 2015. Since then, too much work has happened to make a complete report, but to give some highlights:

• New variations are now tested: umask, kernel version, domain name, and timezone. We might only be missing CPU type and current date now.
• Many improvements to the test system on jenkins.debian.net and the pages showing the results.
• Now not only packages from unstable are tested but also those in testing and experimental.
• When rescheduling packages for testing, the build products can be kept and the IRC channel gets a notification when its over.
• binutils version 2.25-6 is now built with the --enable-deterministic-archives flag. Making ar, strip and others create deterministic static libraries.
• Number of identified issues has grown from about 80 to 123 today.

Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon. This past week It seems changes were pilling behind the curtains given the amount of activity that happened in just one week. Toolchain fixes

• Niels Thykier uploaded debhelper/9.20150501 which includes fixes to dh_makeshlibs (#774100), dh_icons (#774102), dh_usrlocal (#775020). Patches written by Lunar.
• Helmut Grohne uploaded doxygen/1.8.9.1-3 which will not generate timestamps in HTML by default. Kudos to akira for bringing the issue upstream.
• Kenneth J. Pronovici uploaded epydoc/3.0.1+dfsg-6 adding a --no-include-build-time option. Patch by Jelmer Vernooij.
• David Pr vot uploaded php-apigen/2.8.1+dfsg-2 which now has reproducible output.
• C dric Boutillier uploaded ruby-prawn/2.0.1+dfsg-1 which now produce a deterministic output when using gradients. Patch by Lunar.
• Jelmer Vernooij uploaded samba/2:4.1.17+dfsg-4 which contains a patch by Matthieu Patou making the output of pidl (from libparse-pidl-perl) reproducible.
• Dmitry Shachnev uploaded sphinx/1.3.1-1 in experimental which should produce deterministic output. The original patch from Chris Lamb has inspired the upstream fix.
• gregor herrmann uploaded libextutils-depends-perl/0.404-1 which makes ExtUtils::Depends output deterministic. Original patch by Reiner Herrmann.
• Niko Tyni uploaded perl/5.20.2-4 which makes the output of Pod::Man reproducible. Nice team work visible on #780259.

We also rebased the experimental version of debhelper twice to merge the latest set of changes. Lunar submitted a patch to add a -creation-date to genisoimage. Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc. Juan Picca submitted a patch to add a --use-date flag to texi2html. Packages fixed The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons-math3, devscripts, disruptor, ehcache, ftphs, gtk2hs-buildtools, haskell-abstract-deque, haskell-abstract-par, haskell-acid-state, haskell-adjunctions, haskell-aeson, haskell-aeson-pretty, haskell-alut, haskell-ansi-terminal, haskell-async, haskell-attoparsec, haskell-augeas, haskell-auto-update, haskell-binary-conduit, haskell-hscurses, jsch, ledgersmb, libapache2-mod-auth-mellon, libarchive-tar-wrapper-perl, libbusiness-onlinepayment-payflowpro-perl, libcapture-tiny-perl, libchi-perl, libcommons-codec-java, libconfig-model-itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt-perl, libdatetime-timezone-perl, libdbd-firebird-perl, libdbix-class-resultset-recursiveupdate-perl, libdbix-profile-perl, libdevel-cover-perl, libdevel-ptkdb-perl, libfile-tail-perl, libfinance-quote-perl, libformat-human-bytes-perl, libgtk2-perl, libhibernate-validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl-perl, liblog-any-perl, libmail-imapclient-perl, libmocked-perl, libmodule-build-xsutil-perl, libmodule-extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia-perl, libnet-frame-layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql-abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar-perl, libterm-shellui-perl, libtest-dir-perl, libtest-log4perl-perl, libtext-context-eitherside-perl, libtime-warp-perl, libtree-simple-perl, libwww-shorten-simple-perl, libwx-perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl-tools, quodlibet, r-cran-rjags, r-cran-rjson, r-cran-sn, r-cran-statmod, ruby-nokogiri, sezpoz, skksearch, slurm-llnl, stellarium. The following packages became reproducible after getting fixed:

• berkeley-abc/1.01+20141105hg5b5af75+dfsg-3 uploaded by Ruben Undheim, original patch by Johann Klammer.
• binutils/2.25-7 by Matthias Klose.
• bitmap-mule/8.5+0.20030825.0433-16 uploaded by Tatsuya Kinoshita, original patch by Chris Lamb.
• blobby/1.0-2 by Felix Geyer.
• codelite/7.0+dfsg-2 by James Cowgill.
• conkeror/1.0~~pre-1+git150409-1 by Axel Beckert.
• convmv/1.15-1 by Christian Perrier.
• cpl/6.6~b-1 by Ole Streicher.
• deheader/1.1-2 by Reiner Herrmann.
• dict-foldoc/20150318-1 uploaded by Iustin Pop, original patch by Chris Lamb.
• ding/1.8-1 by Roland Rosenfeld.
• doc-rfc/20150425-1 by Iustin Pop.
• doxygen/1.8.9.1-3 by Helmut Grohne.
• eureka/1.07-1 by Fabian Greffrath.
• eximdoc4/4.85-2 uploaded by Andreas Metzler, original patch by Chris Lamb.
• flashproxy/1.7-3 by Ximin Luo.
• heimdal/1.6~rc2+dfsg-10 by Jelmer Vernooij.
• init-system-helpers/1.23 by Martin Pitt original patch by Lunar.
• ldb/2:1.1.20-2 by Jelmer Vernooij.
• libalien-wxwidgets-perl/0.67+dfsg-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libcairo-perl/1.105-1 by intrigeri.
• libclass-methodmaker-perl/2.24-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libcommon-sense-perl/3.73-3 uploaded by gregor herrmann, original patch by Chris Lamb.
• libelixirfm-perl/1.1.976-4 uploaded by gregor herrmann, original patch by Chris Lamb.
• libgnome2-perl/1.045-3 by intrigeri.
• libjs-jcrop/0.9.12+dfsg-2 uploaded by David Pr vot, original patch by Chris Lamb.
• liblas/1.8.0-2 by Bas Couwenberg.
• libopengl-perl/0.6704+dfsg-1 uploaded by gregor herrmann, original patch by Chris Lamb.
• libparse-recdescent-perl/1.967009+dfsg-2 uploaded by gregor herrmann, original patch by Reiner Herrmann.
• librasterlite/1.1g-5 by Bas Couwenberg.
• libterm-size-perl-perl/0.029-2 uploaded by gregor herrmann, original patch by Chris Lamb.
• libvdpau/1.1-1 by Andreas Beckmann.
• libxml-sax-expatxs-perl/1.33-2 uploaded by gregor herrmann, original patch by Chris Lamb.
• lynx-cur/2.8.9dev4-2 by Axel Beckert.
• mapcache/1.2.1-3 by Bas Couwenberg.
• mdbtools/0.7.1-4 by Jean-Michel Nirgal Vourg re.
• mopidy uploaded by Stein Magnus Jodal, fixed by upstream.
• objenesis/2.1-1 by Markus Koschany.
• opendkim/2.10.1-2 uploaded by Scott Kitterman, original patch by Reiner Herrmann.
• pktools/2.6.3-1 by Bas Couwenberg.
• posh/0.12.4 uploaded by Clint Adams, original patch by Chris Lamb.
• prboom-plus/2:2.5.1.4~svn4425+dfsg1-1 by Fabian Greffrath.
• qlandkartegt/1.8.1+ds-1 by Bas Couwenberg.
• readosm/1.0.0d-1~exp2 by Bas Couwenberg.
• robocode/1.9.2.4-1 by Markus Koschany.
• ruby-prawn/2.0.1+dfsg-1 uploaded by C dric Boutillier, original patch by Lunar.
• sane-backends/1.0.25+git20150425-1 by J rg Frings-F rst.
• scoop/0.7.1-3 by Daniel Stender.
• springlobby/0.218-1 by Markus Koschany.
• subvertpy/0.9.2-1 by Jelmer Vernooij.
• t-prot/3.4-2 by Axel Beckert.
• talloc/2.1.2-3 by Jelmer Vernooij.
• tdb/1.3.4-2 by Jelmer Vernooij, patch now applied by upstream.
• tk-html3/3.0~fossil20110109-5 by Ole Streicher.
• tox/1.9.2-2 uploaded by Barry Warsaw original patch by Reiner Herrmann.
• trove3/3.0.3-2 by Erich Schubert.
• txt2man/1.5.6-2 uploaded by Joao Eriberto Mota Filho, initial patch by Jonathan Wiltshire.
• units/2.11-2 by Stephen Kitt.
• win32-loader/0.7.10 upload by Didier Raboud, original patch by Lunar.
• zec/0.12-3 uploaded by Clint Adams, original patch by Chris Lamb.
• zomg/0.8-1 uploaded by Clint Adams, original patch by Chris Lamb.

Some uploads fixed some reproducibility issues but not all of them:

• ada-reference-manual/1:2012.2-4 by Nicolas Boulenguez.
• adacontrol/1.16r11-3 by Nicolas Boulenguez.
• aspectj/1.8.5-1 by Emmanuel Bourg.
• binutils-m68hc1x/1:2.18-4 uploaded by Riku Voipio, original patch by Chris Lamb.
• debianutils/4.5) uploaded by Clint Adams, original patch by Lunar.
• ioquake3/1.36+u20150412+dfsg1-1 by Simon McVittie.
• libsdl1.2/1.2.15-11 uploaded by Manuel A. Fernandez Montecelo, original patch by Chris Lamb; currently FTBFS.
• libsdl2/2.0.2+dfsg1-7 by Manuel A. Fernandez Montecelo, original patch by Chris Lamb; currently FTBFS.
• nedit/1:5.6a-2 by Paul Gevers.
• openarena-085-data/0.8.5split-4 by Simon McVittie.
• openarena-data/0.8.5split-4 by Simon McVittie.
• openarena/0.8.8-12 by Simon McVittie.
• openchange/1:2.2-6 by Jelmer Vernooij.
• puredata/0.46.6-1 by IOhannes m zm lnig.
• pyexiv2/0.3.2-8 by Michal iha ; currently FTBFS.
• quakespasm/0.90.0-2 by Stephen Kitt.
• sed/4.2.2-5 by Clint Adams, patch by Lunar.
• v4l2loopback/0.8.0-5 uploaded by IOhannes m zm lnig, original patch by Chris Lamb.
• vim/2:7.4.712-1 uploaded by James McCoy, original patch by Reiner Herrmann.
• zookeeper/3.4.6-4 by Emmanuel Bourg.

Patches submitted which did not make their way to the archive yet:

• #783882 on jodconverter by Reiner Herrmann: tell javadoc to stop writing timestamps.
• #783885 on bind9 by Reiner Herrmann: pass -DNO_VERSION_DATE to build system.
• #783688 on serverstats by Chris Lamb: fix permissions.
• #783453 on cdbackup by Chris Lamb: remove build date from version string in binary.
• #783478 on texi2html by Juan Picca: pass --use-date to texi2html.
• #783933 on imagemagick by Reiner Herrmann: remove timestamps from PNG and use deterministic package build date.
• #783515 on memtest86+ by Lunar: fix embedded mtimes and pass -creation-date to genisoimage.
• #783558 on websvn by Chris Lamb: fix permissions.

Improvements to reproducible.debian.net Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content. Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies. Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version. debbindiff development Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster. Documentation update Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc. Package reviews Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build. For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing. 1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West. Summer of code Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let's welcome them! They have until May 25th before coding officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

Yesterday I permanently deleted 15 years of email. It wasn't because I didn't have enough hard disk space to store it. It's because I decided, after 15 years, that the benefits of keeping all this email did not outweigh the risks. Although I have never had my email subpoenaed, I have had many legal interactions due to my involvement with May First/People Link, some of which were about finding the real identities of May First/People Link members. I'd rather not risk compromising anyone or needlessly exposing my networks. Now I have an Inbox, Sent Box, Trash Box and Detected Spam Box. The Inbox I empty manually and the other boxes are automatically purged on a scheduled basis. In this age of surveillance it's sad to see data evaluated based on risk of exposure.

For once you have tasted flight,
You will walk the earth with your eyes turned skyward;
For there you have been,
And there you long to return. Leonardo da Vinci

There is something of a magic to flight, to piloting. I remember the first flight I ever took, after years of dreaming of flying in a plane: my grandma had bought me a plane ticket. In one of the early morning flights, I witnessed a sunrise above cumulus clouds. Although I was just 10 or so at the time, that still is a most beautiful image seared into my memory. I have become meh about commercial flight over the years. The drive to the airport, the security lines, the lack of scenery at 35,000 feet. And yet, there is much more to flight than that. When I purchased what was essentially a flying camera, I saw a whole new dimension of the earth s amazing beauty. All the photos in this post, in fact, are ones I took. I then got a RC airplane, because flying the quadcopter was really way too easy.

It s wonderful to climb the liquid mountains of the sky.
Behind me and before me is God, and I have no fears. Helen Keller

Start talking to pilots, and you notice a remarkable thing: this group of people that tends to be cool and logical, methodical and precise, suddenly finds themselves using language almost spiritual. Many have told me that being a pilot brings home how much all humans have in common, the unifying fact of sharing this beautiful planet together. Many volunteer with organizations such as Angel Flight. And having been up in small planes a few times, I start to glimpse this. Flying over my home at 1000 up, or from lake to lake in Seattle with a better view than the Space Needle, seeing places familiar and new, but from a new perspective, drives home again and again the beauty of our world, the sheer goodness of it, and the wonderful color of the humanity that inhabits it.

The air up there in the clouds is very pure and fine, bracing and delicious. And why shouldn t it be? It is the same the angels breathe. Mark Twain

The view from 1000 feet, or 3000, is often so much more spectacular than the view from 35,000 ft as you get on a commercial flight. The flexibility is too; there are airports all over the country that smaller planes can use which the airlines never touch. Here is one incredible video from a guy that is slightly crazy but does ground-skimming, flying just a few feet off the ground: (try skipping to 9:36) So what comes next is something I blame slightly on my dad and younger brother. My dad helped get me interested in photography as a child, and that interest has stuck. It s what caused me to get into quadcopters ( a flying camera for less than the price of a nice lens! ). And my younger brother started mentioning airplanes to me last year for some reason, as if he was just trying to get me interested. Eventually, it worked. I started talking to the pilots I know (I know quite a few; there seems to be a substantial overlap between amateur radio and pilots). I started researching planes, flight, and especially safety the most important factor. And eventually I decided I wanted to be a pilot. I ve been studying feverishly, carrying around textbooks and notebooks in the car, around the house, and even on a plane. There is a lot to learn. And today, I took my first flight with a flight instructor. Today I actually flew a plane for awhile. Wow! There is nothing quite like that experience. Seeing a part of the world I am familiar with from a new perspective, and then actually controlling this amazing machine I really fail to find the words to describe it. I have put in many hours of study already, and there will be many more studying and flying, but it is absolutely worth it. Here is one final video about one of the most unique places you can fly to in Kansas. And a blog with lots of photos of a flight to Beaumont called Horse on the runway .

Today, Aaron Shaw and I are pleased to announce a new startup. The startup is based around an app we are building called RomancR that will bring the sharing economy directly into your bedrooms and romantic lives. When launched, RomancR will bring the kind of market-driven convenience and efficiency that Uber has brought to ride sharing, and that AirBnB has brought to room sharing, directly into the most frustrating and inefficient domain of our personal lives. RomancR is Uber for romance and sex. Here s how it will work:

• Users will view profiles of nearby RomancR users that match any number of user-specified criteria for romantic matches (e.g., sexual orientation, gender, age, etc).
• When a user finds a nearby match who they are interested in meeting, they can send a request to meet in person. If they choose, users initiating these requests can attach an optional monetary donation to their request.
• When a user receives a request, they can accept or reject the request with a simple swipe to the left or right. Of course, they can take the donation offer into account when making this decision or counter-offer with a request for a higher donation. Larger donations will increase the likelihood of an affirmative answer.
• If a user agrees to meet in person, and if the couple then subsequently spends the night together RomancR will measure this automatically by ensuring that the geolocation of both users phones match the same physical space for at least 8 hours the donation will be transferred from the requester to the user who responded affirmatively.
• Users will be able to rate each other in ways that are similar to other sharing economy platforms.

Of course, there are many existing applications like Tinder and Grindr that help facilitate romance, dating, and hookups. Unfortunately, each of these still relies on old-fashion intrinsic ways of motivating people to participate in romantic endeavors. The sharing economy has shown us that systems that rely on these non-monetary motivations are ineffective and limiting! For example, many altruistic and socially-driven ride-sharing systems existed on platforms like Craigslist or Ridejoy before Uber. Similarly, volunteer-based communities like Couchsurfing and Hospitality Club existed for many years before AirBnB. None of those older systems took off in the way that their sharing economy counterparts were able to! The reason that Uber and AirBnB exploded where previous efforts stalled is that this new generation of sharing economy startups brings the power of markets to bear on the problems they are trying to solve. Money both encourages more people to participate in providing a service and also makes it socially easier for people to take that service up without feeling like they are socially in debt to the person providing the service for free. The result has been more reliable and effective systems for proving rides and rooms! The reason that the sharing economy works, fundamentally, is that it has nothing to do with sharing at all! Systems that rely on people s social desire to share without money projects like Couchsurfing are relics of the previous century. RomancR, which we plan to launch later this year, will bring the power and efficiency of markets to our romantic lives. You will leave your pitiful dating life where it belongs in the dustbin of history! Go beyond antiquated non-market systems for finding lovers. Why should we rely on people s fickle sense of taste and attractiveness, their complicated ideas of interpersonal compatibility, or their sense of altruism, when we can rely on the power of prices? With RomancR, we won t have to! Note: Thanks to Yochai Benkler whose example of how leaving a $100 bill on the bedside table of a person with whom you spent the night can change the nature of the a romantic interaction inspired the idea for this startup.

Today, Aaron Shaw and I are pleased to announce a new startup. The startup is based around an app we are building called RomancR that will bring the sharing economy directly into your bedrooms and romantic lives. When launched, RomancR will bring the kind of market-driven convenience and efficiency that Uber has brought to ride sharing, and that AirBnB has brought to room sharing, directly into the most frustrating and inefficient domain of our personal lives. RomancR is Uber for romance and sex. Here s how it will work:

• Users will view profiles of nearby RomancR users that match any number of user-specified criteria for romantic matches (e.g., sexual orientation, gender, age, etc).
• When a user finds a nearby match who they are interested in meeting, they can send a request to meet in person. If they choose, users initiating these requests can attach an optional monetary donation to their request.
• When a user receives a request, they can accept or reject the request with a simple swipe to the left or right. Of course, they can take the donation offer into account when making this decision or counter-offer with a request for a higher donation. Larger donations will increase the likelihood of an affirmative answer.
• If a user agrees to meet in person, and if the couple then subsequently spends the night together RomancR will measure this automatically by ensuring that the geolocation of both users phones match the same physical space for at least 8 hours the donation will be transferred from the requester to the user who responded affirmatively.
• Users will be able to rate each other in ways that are similar to other sharing economy platforms.

Of course, there are many existing applications like Tinder and Grindr that help facilitate romance, dating, and hookups. Unfortunately, each of these still relies on old-fashion intrinsic ways of motivating people to participate in romantic endeavors. The sharing economy has shown us that systems that rely on these non-monetary motivations are ineffective and limiting! For example, many altruistic and socially-driven ride-sharing systems existed on platforms like Craigslist or Ridejoy before Uber. Similarly, volunteer-based communities like Couchsurfing and Hospitality Club existed for many years before AirBnB. None of those older systems took off in the way that their sharing economy counterparts were able to! The reason that Uber and AirBnB exploded where previous efforts stalled is that this new generation of sharing economy startups brings the power of markets to bear on the problems they are trying to solve. Money both encourages more people to participate in providing a service and also makes it socially easier for people to take that service up without feeling like they are socially in debt to the person providing the service for free. The result has been more reliable and effective systems for proving rides and rooms! The reason that the sharing economy works, fundamentally, is that it has nothing to do with sharing at all! Systems that rely on people s social desire to share without money projects like Couchsurfing are relics of the previous century. RomancR, which we plan to launch later this year, will bring the power and efficiency of markets to our romantic lives. You will leave your pitiful dating life where it belongs in the dustbin of history! Go beyond antiquated non-market systems for finding lovers. Why should we rely on people s fickle sense of taste and attractiveness, their complicated ideas of interpersonal compatibility, or their sense of altruism, when we can rely on the power of prices? With RomancR, we won t have to! Note: Thanks to Yochai Benkler whose example of how leaving a $100 bill on the bedside table of a person with whom you spent the night can change the nature of the a romantic interaction inspired the idea for this startup.

Pictures from the CDSW sessions in Spring 2014

After two successful rounds in 2014, I m helping put on another round of the Community Data Science Workshops. Last year, our 40+ volunteer mentorss taught more than 150 absolute beginners the basics of programming in Python, data collection from web APIs, and tools for data analysis and visualization and we re still in the process of improving our curriculum and scaling up. Once again, the workshops will be totally free of charge and open to anybody. Once again, they will be possible through the generous participation of a small army of volunteer mentors. We ll be meeting for four sessions over three weekends:

• Setup and Programming Tutorial (April 10 evening)
• Introduction to Programming (April 11)
• Importing Data from web APIs (April 25)
• Data Analysis and Visualization (May 9)

If you re interested in attending, or interested in volunteering as mentor, you can go to the information and registration page for the current round of workshops and sign up before April 3rd.

I have just released version 1.9 of Obnam, my backup program. See the website at http://obnam.org for details. The new version is available from git (see http://git.liw.fi) and as Debian packages from http://code.liw.fi/debian. Due to the freeze of Debian for the jessie release, I've not uploaded this version to Debian yet (not experimental and not backports). This is the first Obnam release since May 13, 2014, 313 days ago. That's a long time. I make no excuses: Obnam is a hobby project, which I work on when I have the time and energy. The past year has been very /interesting/ year for me, in all sorts of stressful ways: I've changed jobs, moved to another country, and dealt with the loss of a close relative. Because of this, I've not been able to spend as much time on Obnam as I'd like. The NEWS file extract below gives the highlights of what has happened to Obnam during this time. There's been a lot of things, actually. My plans for Obnam next are mainly centered around performance. This will require developing a new repository format, to allow things that are not possible with the current format. For example, the current format stores each data chunk in its own file in the repository, and that is quite wasteful when live data files (and therefore their chunks) are quite small. As preparation for this work, the silly-looking "simple" format has been added, mostly to make sure the internal code infrastructure is ready to support multiple repository formats in the same Obnam version. Those interested in discussing ways to make Obnam fast should join the obnam-dev mailing list. Version 1.9, released 2015-03-22 New features:

• James Vasile changed Obnam so it can backup an individual file, instead of an entire directory.
• James Vasile added the --include option to Obnam, allowing one to include files that would otherwise be excluded (see --exclude).
• Carlo Teubner changed obnam fsck to remove unused chunks, if the --fsck-fix or --fsck-rm-unused settings are used. He also made it not check for unused chunks when it's useless to do so, because of various --fsck-skip settings are used.
• A start of a French translation of the manual by pedrito2.
• Ian Cambell provided a new Obnam command, obnam kdirstat, which makes the KDE k4dirstat utility be able to show graphically which parts of a backup generation use most space.
• Lars Wirzenius added the simple repository format, which is for demonstration only. It is much too simplistic to be used for real.

Minor changes:

• The manual page and obnam --help are now clearer that the --root setting and command line arguments to obnam backup can be SFTP URLs. Thanks to Simone Piccardi for reporting the issue.
• David Fries filled in the displayed file permission mode bits.
• Grammar and typo fixes for the obnam.1 manual page, from Jean Jordaan.
• Tom Chiverton suggested a clarification to the manual page for "obnam mount" to say that each generation is a subdirectory.
• David Fries changed restore to set the group ownership if possible even when not root. No warnings are issued if the attempt fails.
• Jan Niggemann added a little to the German translation of the Obnam manual.
• Lars Wirzenius added the path to the error message about a missing chunk (R43272X).
• Lars Wirzenius made the message at the end of a backup report more statistics about transfers during the backup.

Bug fixes:

• The Obnam SFTP plugin would loop infinitely if it lost the connection to the SSH server while creating a temporary file. Itamar Turner-Trauring provided a fix for this.
• Will Dyson fixed a bug about locking while removing checkpoint generations.
• Michel Alexandre Salim fixed a Python 2.6 compatibility problem in the unit tests (use of assertRaises as a context manager).
• Lars Kruse fixed a bug with backing up of overlapping backup roots (e.g., / and /boot), given a test case by Adrien Clerc.
• Thomas Eschenbacher fixed a bug in the format 6 repository code that would crash when there is an obscure problem and a B-tree code can't be found in the tree.
• Tom Chiverton pointed out that the manual page was using "obnam restore" instead of "obnam mount" in an example for "obnam mount".
• The yarn test suite now runs FUSE tests (obnam mount) when fusermount is available, rather than checking for membership in the group fuse. The latter is a Debianism (fixed in Debian jessie).
• Thomas Waldmann noticed that obnam verify didn't notice that a file had new data, when the modification time was the same. Obnam now notices this.
• Thomas Waldmann fixed many typos and minor bugs in the source code.
• Laurence Perkins reported that the Tahoe-LAFS SFTP server returned some stat fields as None. Fixed to change those to be 0 instead.
• Lars Wirzenius fixed double-downloading of chunks during restores.

"As long as people still think of people like me as "a dude in a dress" there is a lot work to do to fight transphobia and gain tolerance and acceptance."

This line in Rhonda's most recent blogpost broke my heart a little, and sparked an interesting conversation with her about the (perceived?) value of clothes, respect and identity. So, guess what? Here's a pic of a "girl in trousers". Just because. (Sorry for the quality: couldn't find my camera and had to use a phone. Also, I don't own a binder, so I used a very light binding)

Mika recently brought up the Japanese modern legend of Kuchisake-onna ( ). For background, I turned to the English Wikipedia article on Kuchisake-onna which had the following to say about the figure (the description matches Mika s memory):

According to the legend, children walking alone at night may encounter a woman wearing a surgical mask, which is not an unusual sight in Japan as people wear them to protect others from their colds or sickness. The woman will stop the child and ask, Am I pretty? If the child answers no, the child is killed with a pair of scissors which the woman carries. If the child answers yes, the woman pulls away the mask, revealing that her mouth is slit from ear to ear, and asks How about now? If the child answers no, he/she will be cut in half. If the child answers yes, then she will slit his/her mouth like hers. It is impossible to run away from her, as she will simply reappear in front of the victim.

To help anyone who is not only frightened, but also confused, Mika and I made the following decision tree of possible conversations with Kuchisake-onna and their universally unfortunate outcomes.

Decision tree of conversations with Kuchisake-onna.

Of course, we uploaded the SVG source for the diagram to Wikimedia Commons and used the diagram to illustrate the Wikipedia article.

November 2014:
Sunday 2nd:
Travel from Moscow to Paris Monday 3rd to Sunday 8th:
Summit in Paris Monday 10th:
Uploaded python-rudolf to Sid (needed by Fuel)
Uploaded python-invoke and python-invocations (needed to run fabric s unit tests)
Uploaded python-requests-kerberos/0.5-2 fixing CVE-2014-8650: failure to handle mutual authentication. Asked the release team for unblock.
Uploaded openstack-pkg-tools version 19 fixing startup with systemd in Jessie (added RuntimeDirectory directive). Asked the release team for unblock.
Opened ticket to remove TripleO, Tuskar and Ironic packages from Jessie. I don t consider them ready for a Debian stable release, and there s no long term support from upstream.
Fixed Designate Juno dbsync process which prevented it from being installed.
Fixed Ironic Juno unowned files after purge (policy 6.8, 10.8): /var/lib/ironic/ cache, ironicdb (eg: purging these folders on purge) Thuesday 11:
Fixed nova-api CVE-2014-3708: Nova network DoS through API filtering in both the Juno and Icehouse release. Asked the release team to unblock the Icehouse version for Jessie. See: https://bugs.debian.org/769163
Uploaded Cinder with Duch debconf translation fix and pt.po
Uploaded python-django-pyscss with upstream patch for Django 1.7 support instead of the Debian one that I wrote 2 months ago. Asked the release team to unblock which they did. Wednesday 12:
Uploaded fix for horizon (see #769101) unowned files after purge (policy 6.8, 10.8). Now purging /usr/share/openstack-dashboard/openstack_dashboard on purge.
Uploaded Ironic with Duch translations of debconf
Uploaded Designate with Duch translations of Debconf screens
Uploaded openstack-trove with Duch translations of Debconf screens
Uploaded Tuskar with Duch translations of Debconf screens
Updated python-oslotest in Experimental to version 1.2.0 Thursday 13:
Uploaded new packages: python-oslo.middleware and python-oslo.concurrency.
Opened a new packaging branch for Nova Kilo, and updated (build-)depends.
Uploaded fix for Icehouse Cinder: delete volume failed due to unicode problems , and asked for unblock.
Uploaded new package: python-pygit2 and python-xmlbuilder, needed for fuel-agent-ci.
Uploaded sheepdog with Duch debconf translation.
Uploaded python-daemonize to Sid (in FTP master NEW queue).
Re-uploaded python-invoke after FTP master rejection (missing copyright information) Friday 14:
Uploaded liberasurecode & python-pyeclib to Sid, now in the FTP masters NEW queue waiting for approval. This will soon be needed by Swift. Monday 17:
Worked on the Cobbler packaging (all day long ) Tuesday 18:
Worked on backporting all of Fuel packages to Wheezy. Done with fuelclient already.
Uploaded ruby-cstruct and ruby-rethtool to Sid (needed by nailgun-agent) Wednesday 19:
Uploaded pyeclib again, with fixes for the build-depends. Package is still in the NEW queue anyway.
Built a Debian-based bootstrap hardware discovery image for Fuel, and it seems that it works already (to be checked )! \o/
To be added as packages in the ISO:
* nailgun-mcagents
* nailgun-net-check
* fuel-agent
* python-tasklib Thursday 20:
Uploaded python-tasklib to Sid (now in NEW queue )
Continued working on the discovery bootstrap ISO Friday 21:
Documented Sahara procedure in Debian in the official install-guide: https://review.openstack.org/136237
Fixed oslo.messaging so it doesn t use PROTOCOL_SSLv3 because its support has been removed from Debian (due to possible protocol downgrade attacks): https://review.openstack.org/136278 and uploaded fixed packages for Sid and Experimental.
Uploaded fixed Neutron packages for CVE-2014-7821 in both Sid and Experimental (eg: Icehouse and Juno) Monday 24:
Uploaded new package: python-os-client-config (in NEW queue)
Installed new Xen server to be used as my new Jenkins build machine
Moved the juno-wheezy VM to it
Finished to package python-pymysql and uploaded to Sid. It s now running all unit tests successfully! \o/ Tuesday 25:
Uploaded fix for openstack-debian-images to add the -o compat=1.0 option when building an image with Qemu > 1.0. Opened bug to the release team to have it unblocked.
Continued working on unit tests for fuel-nailgun. Wednesday 26:
Uploaded python-os-net-config to Sid (new package)
Worked briefly on python-cassandra-driver. It needs cassandra to be in, which is a LOT of work.
Found a (not useable) hack to run nailgun unit tests. It works, however, it doesn t seem like fuel-nailgun is designed to be able to use unix socket for the postgres connection in its unit tests.
Uploaded python-pykmip to Sid (new package)
Updated the Debian wheezy backport repository for libvirt to version 1.2.9 from official wheezy-backports. Removed policykit-1 and libusb from there too, as it broke stuff to use a backported version (X and usb were not useable on my Wheezy laptop when using it ). Thursday 27 & Friday 28:
Uploaded new Javascript packages or dependencies for Fuel: libjs-autonumeric, libjs-backbone-deep-model, libjs-backbone.stickit, libjs-cocktail, libjs-i18next, libjs-require-css, libjs-requirejs, libjs-requirejs-text Sunday 30:
Uploaded debian/copyright fixes for libjs-backbone-deep-model, libjs-backbone.stickit and libjs-cocktail after the packages were accepted by the FTP masters and they gave remarks about copyright. DECEMBER 2014 Monday 01:
Uploaded new Debian image to MOX, after I unerstood the issue was about the architecture field that I was wrongly filling. I ll be able to use that for Tempest checking on my dev account. Tuesday 02:
Uploaded python-q-text-as-data to Sid (new awesome package!)
Uploaded Horizon with some triggers mechanisms to start the compress when one of its JS depends is updated. That s very important for security!
Uploaded a fixed version of heat-cfntools to Sid (it was missing the /usr/lib/python* folder). Asked the release team for an unblock so it can reach Jessie.
Fixed unit tests in fuel-nailgun, thanks to a patch from Sebastian Kalinowski. Now all unit tests are passing but one (for which I opened a launchpad bug: tests are trying to write in /var/log/nailgun, which is impossible at package build time). Wednesday 03:
Uploaded fixed version of ruby-rethtool after FTP master s rejection and upstream correction of licensing files.
Uploaded fixed version of libjs-require-css after FTP master s rejection
Fixed (in Git only) python-sysv-ipc missing build-depends on dh-python as per bug opened by James Page (this is not so important, but I did it still).
Continued working on the tempest-ci scripts.
Added to the image-guide docs about openstack-debian-images: https://review.openstack.org/#/c/138743/ Thursday 04:
Uploaded new package: python-proliantutils. Send patch to upstream about an issue in indentation (mix-up with space and tabs) which made the package uninstallable with Python 3.4. Friday 05:
Worked on the package CI. Monday 07:
Worked on the package CI. All works now, up to all of the Tempest tests for Keystone. Now need to fix the neutron config. Thuesday 08:
Continued working on the CI. Wednesday 09:
Uploaded fix for FTBFS of python-tasklib (Closes: #772606)
Uploaded fix for libjerasure-deb missing dependency on libgf-complete-dev, package already unblocked and will migrate to Jessie.
Uploaded fix for Designate Juno fail to upgrade from Icehouse: this was due to the database_connection directive renamed to connection =.
Uploaded fix for Designate purge in Sid (Icehouse release of Designate).
Commited to git updates of the German debconf translation in both Icehouse and Juno.
Updated nova to use libvirtd as init script dependency instead of libvirt-bin (this was renamed in the libvirt-daemon-system package).
Do not touch the db connection directive if user didn t ask for db handling by the package. Thursday 10 to Saturday 13:
Finally understood the issues with systemd service files not being activated by default. Fixed openstack-pkg-tools, and uploaded version 20 to Sid, after the release team accepted the changes. Sunday 14:
Uploaded Juno 2014.2.1 to Experimental: ceilometer, cinder, glance, python-glance-store, heat, horizon, keystone Monday 15:
Finished uploading Juno 2014.2.1 to Experimental: Nova, Neutron, Sahara Tuesday 16:
Added crontab to flush tokens in Icehouse Keystone
Some more CI work Wednesday 17:
Uploaded keystone with systemd fix and crontab to flush the token table in Sid (eg: Icehouse).
Uploaded nova Icehouse with a bunch of fixes in Sid. Thursday 18:
Updated some issues in Nova Icehouse (Sid/Jessie) Friday 19:
Started building a new Jenkins instance for building Kilo packages Monday 22:
Finished building the new Jenkins instance for building Kilo packages, and rebuilt every packages there, using Jessie as a base. Tuesday 23:
Updated version for the following packages: oslo.utils, oslo.middleware, stevedore, oslo.concurency, pecan, oslo.concurrency, python-oslo.vmware, python-glance-store
Built so far: Ceilometer, Keystone, python-glanceclient, cinder, glance Wednesday 24:
Continued packaging Kilo beta 1. Updated: nova, designate, neutron
Uploaded python-tempest-lib to Debian Unstable (new package) Wednesday 31:
Continued packaging Kilo beta 1. Updated: heat JANUARY 2015 Thursday 01:
Continued packaging Kilo beta 1. Updated: ironic, openstack-trove, openstack-doc-tools, ceilometer Friday 02:
Finished packaging Kilo beta 1. Updated: Sahara, Murano, Murano-dashboard, Murano-agent Sunday 04:
Started testing Kilo beta 1. Fixed a few issues on default configuration for Ceilometer and Glance. Monday 05:
Fixed openstack-pkg-tools which failed to create PID files at boot time, Uploaded to Sid, asked the release team for unblock.
Uploaded ceilometer & cinder to Sid, rebuilt against openstack-pkg-tools 21.
Did more testing of Kilo beta 1, fixed a few more minor issues. Tuesday 06:
Uploaded glance, neutron, nova, designate, keystone, heat, trove to Sid, so that all sysv-rc init scripts are fixed with the new openstack-pkg-tools 21. Designate, heat, keystone and trove contains other minor fixes reported to the Debian BTS. Wednesday 07:
Asked the Debian release team (open bugs with debdiff as attachment) for unblocks of glance, neutron, nova, designate, keystone, heat, trove so they migrate to Jessie.
Fixed a few minor issues tracked in the Debian BTS on various packages. Thesday 08:
James Page from Canonical informed me that they are now using openstack-pkg-tools for maintaining their daemons in Nova, Cinder and Keystone in Ubuntu. That s an awesome news : more QA for both platforms.
James Page found out that dh_installinit *must* be called *after* the call of dh_systemd_enable, otherwise, daemons aren t started automatically at the first install of packages, as the unmask of systemd happens after the invoke-rc.d. Friday 09:
Did some QA checks on the latest upload. Fixed Heat which broke because using the wrong template name (glance instead of heat). Monday 12:
Started re-running the automated openstack-deploy scrip in Icehouse, Juno and Kilo. Found out the issue in Keystone wasn t fixed in Juno (but was fixed in other releases), and fixed it.
Removed the use of ssl.PROTOCOL_SSLv3 from heat (removed form Debian). Uploaded the fixed package to Sid.
All of openstack-deploy (debian/kilo branch) now works and succesfully installs OpenStack again. If dh_installinit is called before, we have:

# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ]   [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start   true
fi
# End automatically added section
# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null   true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null   true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null   true
fi
# End automatically added section

If it s called after dh_systemd_enable, we have:

# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null   true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null   true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null   true
fi
# End automatically added section
# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ]   [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start   true
fi
# End automatically added section

As a consequence, I have to re-upload version 22 of openstack-pkg-tools and also re-upload all OpenStack core packages to Debian Sid. Fixed a number of issues like:
* dbc_upgrade = true check which shouldn t have been there in postinst.
* <project>/configure_db default value is now always false
* db_sync and pkgos_dbc_postinst are now only done if <project>/configure_db is set to true.
Rebuilt all packages in Juno and Kilo with the above changes. Tuesday 13:
Opened unblock bugs for the release team to unblock all fixed packages.
Made more tests in Juno and Kilo to make sure the fixed bugs in Icehouse are fixed there too.
Fixed numerous issues in Trove (missing trove-conductor.conf, wrong trove-api init file, etc.). More work will be needed for it for both Icehouse and newer releases. Wednesday 14:
Did a doc meeting about debconf. Some doc contributors still want to kill the debconf / debian manual, and I have to not agree.
Made a new patch to better document the keystone install procedure:
Did some bug triaging in the doc about Debian.
Uploaded new versions of core packages to Experimental (eg: Juno) built against openstack-pkg-tools >= 22~, and some fixes forward ported from Icehouse: Keystone, Ceilometer, Cinder, Glance, Heat, Ironic, Murano, Neutron, Nova, Saraha and Murano-agent. All where rebuilt in Juno (Wheezy + Trusty) and Kilo (Jessie only) on my Jenkins. Thuesday 15:
Succesfully booted a live-build Debian live image containing mcollective and nailgun-agent as a Debian replacement for the hardware discovery / boostrap image of Fuel. Now, I need to find a way to use just a kernel + initramfs Friday 16 to Tuesday 20:
Worked on the packaging CI. Wednesday 21:
Fixed https://bugs.debian.org/775636 (Horizon failed to build due to a Moscow timezone change and wrong test). Uploaded to Sid, asked for unblock.
Fixed https://bugs.debian.org/775926: CVE-2015-1195: Glance still allows users to download and delete any file in glance-api server (applied upstream patch). Uploaded to Sid, asked for unblock. Uploaded Juno version to Experimental.
Uploaded openstack-trove with the remaining fixes, asked release team for unblock.
Uploaded python-glanceclient 0.15.0 (Juno) to Experimental because it fixes an issue with HTTPS. Added to it a patch from James Page not yet merged, which fixes unit test with Python 2.7.9 (7 failures otherwise).
Uploaded python-xstatic-d3 as it can t be installed anymore in Sid after a new version of d3 was uploaded. Thursday 22:
Uploaded python-xstatic-smart-table and libjs-angularjs-smart-table to Sid (new packages, now in NEW queue). Friday 23:
Ask for the removal of the below list of packages from Jessie:
python-xstatic
python-xstatic-angular
python-xstatic-angular-cookies
python-xstatic-angular-mock
python-xstatic-bootstrap-datepicker
python-xstatic-bootstrap-scss
python-xstatic-d3
python-xstatic-font-awesome
python-xstatic-hogan
python-xstatic-jasmine
python-xstatic-jquery
python-xstatic-jquery-migrate
python-xstatic-jquery-ui
python-xstatic-jquery.bootstrap.wizard
python-xstatic-jquery.quicksearch
python-xstatic-jquery.tablesorter
python-xstatic-jsencrypt
python-xstatic-qunit
python-xstatic-rickshaw
python-xstatic-spin
libjs-jsencrypt
libjs-spin.js
libjs-twitter-bootstrap-datepicker
libjs-twitter-bootstrap-wizard They are used only in OpenStack Horizon starting on 2014.2 (aka Juno), and Jessie is shipped with Icehouse, so it s IMO best to not carry the burden of maintaining these packages for the life of Jessie. Monday 26:
Enhanced and review requested changes for https://review.openstack.org/147296 (ie: Keystone install with more details about what the package does).
Finished testing network on the CI install. Now need to automate all. Tuesday 27:
Closed all bugs on the rabbitmq-server package (2 correction, one bug triage).
Uploaded a fix for the missing conntrack dependency in neutron-l3-agent.
Restarted working on CI setup of Juno after success with manual install in a Xen domU.
Uploaded fix to make sheepdog build reproducible (patch from the Debian BTS). Thursday 28:
Fixed and uploaded to Sid openstack-debian-images 2 bugs reported by Steve McIntire. Official Debian images for OpenStack are now available at:
http://cdimage.debian.org/cdimage/openstack/ \o/
Note that this is the weekly build of testing. We wont get Debian Stable images before Jessie is out.
Documented the new image thing here: http://docs.openstack.org/image-guide/content/ch_obtaining_images.html#debian-images as a new patch: https://review.openstack.org/#/c/151015/
Fixed my patch for keystone debconf doc at: https://review.openstack.org/#/c/147296/ Wednesday 29:
Continued working on packaging CI Thursday 30:
Fixed CVE on Neutron (Juno): L3 agent denial of service with radvd 2.0+
Fixed CVE on Glance (Icehouse + Juno): Glance user storage quota bypass. Asked release team for unblock.
Fixed the image-guide patch after review (ie: https://review.openstack.org/151015)

---

---