Search Results: "cas"

13 November 2016

Andrew Cater: Debian MiniConf, ARM, Cambridge 11/11/16 - Day 2 post 2

It's raining cats and dogs in Cambridge.

Just listening to Lars Wirzenius - who shared an office with Linus Torvalds, owned the computer that first ran Linux, founded the Linux Documentation Project. Living history in more than one sense :)

Live streaming is also happening.

Building work is also happening - so there may be random noise happening occasionally.

Andrew Cater: Debian Miniconf, ARM Cambridge 10/11/16 - post 3

Quiet room with occasional outbursts of talking. One of my problems with radio gear down to broken cable so that's OK-ish - I have spares.

And suddenly it's 1720 and we need to leave at about 1730.

2 x radio receivers tried: 2 x transmitters working ready for talk on Saturday.

Spare huge bag of cables appreciated by others - it's amazing how often you find someone else needs stuff :)





Andrew Cater: Debian Miniconf ARM, Cambridge 10/11/16 - post 2

Now about 30 people here: the video team are chasing down power leads and cables in readiness for the weekend.

One large open plan room with about 30 small quadrilateral tables - two make a hexagon. Very quiet for open plan: periodically the room falls completely silent - lots of developers quietly coding / reading screens.

Lunch was very good curry: ARM caterers feed us very well indeed :D

The radio I'm trying to program refuses to play with the software: the maintainer is at the back of the room and has offered to sort out a backport to Debian stable. Debian can occasionally seem like a dysfunctional family but it's good to be a member.

The Cubietruck I bought last year is sitting with another developer as I speak - he's going to try adding multiple disks for a RAID array on a machine that only draws 5W or so.

11 November 2016

Jonathan Dowland: Vinyl is killing Vinyl (but that's ok)

I started buying vinyl records about 16 years ago, but recently I've become a bit uncomfortable being identified as a "vinyl lover". The market is ascendant, with vinyl album sales growing for 8 consecutive years, at least in the UK. So why am I uncomfortable about it? A quick word about audio fidelity/quality here. I don't subscribe to the school of thought that audio on vinyl is inherently better than digital audio, far from it. I'm aware of its limitations. For recordings that I love, I try to seek out the best quality version available, which is almost always digital. Some believe that vinyl is immune to the "loudness war" brickwall mastering plaguing some modern releases, but for some of the worst offenders (Depeche Mode's Playing The Angel; Red Hot Chili Pepper's Californication) I haven't found the vinyl masterings to sound any different. 16 years ago Let's go back to why I started buying vinyl. Back when I started, the world was a very different place to what it is today. You could not buy most music in a digital form: it was 3 more years before the iTunes Store was opened, and it was Mac-only at first, and the music it sold was DRM-crippled for the first 5 or so years afterwards. The iPod had not been invented yet and there was no real market for personal music players. Minidiscs were still around, but Net-MD (the only sanctioned way to get digital music onto them from a computer) was terrible.
old-ish LPs old-ish LPs
Buying vinyl 16 years ago was a way to access music that was otherwise much harder to reach. There were still plenty of albums, originally recorded and released before CDs, which either had not been re-issued digitally at all, or had been done so early, and badly. Since vinyl was not fashionable, the second hand market was pretty cheap. I bought quite a lot of stuff for pennies at markets and car boot sales. Some music such as b-sides and 12" mixes and other mixes prepared especially for the format remains unavailable and uncollected on CD. (I'm a big fan of the B-side culture that existed prior to CDs. I might write more about that one day.) 10 years ago
modern-ish 7 inches modern-ish 7 inches
Fast forward to around 10 years ago. Ephemeral digital music is now much more common, the iPod and PMPs are well established. High-street music stores start to close down, including large chains like MOS, Our Price, and Virgin. Streaming hasn't particularly taken off yet, attempts to set up digital radio stations are fought by the large copyright owners. Vinyl is still not particularly fashionable, but it is still being produced, in particular for singles for up-and-coming bands in 7" format. You can buy a 7" single for between 1 and 4, getting the b-side with it. The b-side is often exclusive to the 7" release as an incentive to collectors. I was very prepared to punt 1-2 on a single from a group I was not particularly familiar with just to see what they were like. I discovered quite a lot of artists this way. One of the songs we played at our wedding was such an exclusive: a recording of the Zutons' covering Jackie Wilson's "Higher and Higher", originally broadcast once on Colin Murray's Evening Session radio show. Now
Vangelis - Blade Runner OST An indulgence
So, where are we now? Vinyl album sales are a huge growth market. They are very fashionable. Many purchasers are younger people who are new to the format; it's believed many don't have the means to play the music on the discs. Many (most?) albums are now issued as 12" vinyl in parallel with digital releases. These are usually exactly the same product (track listing, mixes, etc.) and usually priced at exactly twice that of the CD (with digital prices normally a fraction under that). The second hand market for 12" albums has inflated enormously. Gone are the bargains that could be had, a typical second hand LP is now priced quite close to the digital price for a popular/common album in most places. The popularity of vinyl has caused a huge inflation in the price of most 7" singles, which average somewhere between 8- 10 each, often without any b-side whatsoever. The good news is from my observations the 2nd hand market for 7" singles hasn't been affected quite as much. I guess they are not as desirable to buyers. The less said about Record Store Day, the better. So, that's all quite frustrating. But most of the reasons I used to buy vinyl have gone away anyway. Many of the rushed-to-market CD masterings have been reworked and reissued, correcting the earlier problems. B-side compilations are much more common so there are far fewer obscure tracks or mixes, and when the transfer has been done right, you're getting those previously-obscure tracks in a much higher quality. Several businesses exist to sell 2nd hand CDs for rock bottom prices, so it's still possible to get popular music very cheaply. The next thing to worry about is probably streaming services.

9 November 2016

Enrico Zini: On SPF

I woke up this morning with some Django server error mails in my inbox: 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 9: ordinal not in range(128)
 
 'REMOTE_USER': u' -guest@users.alioth.debian.org',
I did what one does in cases like these, I tried to fix the bug and mailed -guest@users.alioth.debian.org asking them to try again and let me know if it works. I get a bounce: 
  <Actual user's email>
    (generated from  -guest@users.alioth.debian.org)
    SMTP error from remote mail server after MAIL FROM:<enrico@enricozini.org> SIZE=3948:
    host   [ ]: 550 Please see http://www.openspf.org/Why?id=enrico%40enricozini.org&ip=2001%3a41c8%3a1000%3a21%3a%3a21%3a21&receiver=bq :
    Reason: mechanism
I resent the mail to the actual user's address, and it went through. Job done, at least until they get back to me telling me that my fix didn't work. Lessons learnt:

Daniel Pocock: Understanding what lies behind Trump and Brexit

As the US elections finish, many people are scratching their heads wondering what it all means. For example, is Trump serious about the things he has been saying, or is he simply saying whatever was most likely to make a whole bunch of really stupid people crawl out from under their rocks to vote for him? Was he serious about winning at all, or was it just the ultimate reality TV experiment? Will he show up for work in 2017, or like Australia's billionaire Clive Palmer, will he set a new absence record for an elected official? Ironically, Palmer and Trump have both been dogged by questions over their business dealings, will Palmer's descent towards bankruptcy be replicated in the ongoing fraud trial against Trump University and similar scandals? While the answer to those questions may not be clear for some time, some interesting observations can be made at this point. The world has been going racist. In the UK, for example, authorities have started putting up anti-Muslim posters with an eery resemblance to Hitler's anti-Jew propaganda. It makes you wonder if the Brexit result was really the "will of the people", or were the people deliberately whipped up into a state of irrational fear by a bunch of thugs seeking political power? Who thought The Man in the High Castle was fiction? In January 2015, a pilot of The Man in the High Castle, telling the story of a dystopian alternative history where Hitler has conquered America, was the most-watched original series on Amazon Prime. It appears Trump supporters have already been operating US checkpoints abroad for some time, achieving widespread notoriety when they blocked a family of British Muslims from visiting Disneyland in 2015. Ambushing them at the last moment as they were about to board their flight, it is unthinkable how anybody could be so cruel. When you reflect on statements made by Trump and the so-called "security" practices around the world, this would appear to be only a taste of things to come though. Is it a coincidence that Brexit and Trump both happened in the same year that the copyright on Mein Kampf expired? Ironically, in the chapter on immigration Hitler specifically singles out the U.S.A. for his praise, is that the sort of rave review that Trump aspires to when he talks about making America great again? US voters have traditionally held concerns about the power of the establishment. The US Federal Reserve has been in the news almost every week since the financial crisis, but did you know that the very concept of central banking was thrown out the window four times in America's history? Is Trump the type of hardliner who will go down this path again, or will it be business as usual? In his book Rich Dad's Guide to Investing in Gold & Silver, Robert Kiyosaki and Michael Maloney encourage people to consider putting most of their wealth into gold and silver bullion. Whether you like the politics of Trump and Brexit or not, are we entering an era where it will be prudent for people to keep at least ten percent of net wealth in this asset class again? Online dealers like BullionVault in Europe already appear to be struggling under the pressure as people rush to claim the free grams of bullion credited to newly opened accounts. The Facebook effect In recent times, there has been significant attention on the question of how Facebook and Google can influence elections, some European authorities have even issued alerts comparing this threat to terrorism. Yet in the US election, it was simple email that stole the limelight (or conveniently diverted attention from other threats), first with Clinton's private email server and later with Wikileaks exposing the entire email history of Clinton's chief of staff. The Podesta emails, while being boring for outsiders, are potentially far more damaging as they undermine the morale of Clinton's grass roots supporters. These people are essential for knocking on doors and distributing leaflets in the final phase of an election campaign, but after reading about Clinton's close relationship with big business, many of them may well have chosen to stay home. Will future political candidates seek to improve their technical competance, or will they simply be replaced by candidates who are born hackers and fluent in the language of a digital world?

8 November 2016

Uwe Kleine-K nig: Installing Debian Stretch on an Omnia Turris

Recently I got "my" Omnia Turris and it didn't take long to replace the original firmware with Debian. If you want to reproduce, here is what you have to do: Open the case of the Omnia Turris, connect the hacker pack (or an RS232-to-TTL adapter) to access the U-Boot prompt (see Turris Omnia: How to use the "Hacker pack"). Then download the installer and device tree: 
# cd /srv/tftp
# wget https://d-i.debian.org/daily-images/armhf/daily/netboot/vmlinuz
# wget https://d-i.debian.org/daily-images/armhf/daily/netboot/initrd.gz
# wget https://www.kleine-koenig.org/tmp/armada-385-turris-omnia.dtb
(The latter is not included yet in Debian, but I'm working on that.) and after connecting the Omnia Turris's WAN to a dhcp managed network start it in U-Boot: 
dhcp
setenv serverip 192.168.1.17
tftpboot 0x01000000 vmlinuz
tftpboot 0x02000000 armada-385-turris-omnia.dtb
tftpboot 0x03000000 initrd.gz
bootz 0x01000000 0x03000000:$filesize 0x02000000
With 192.168.1.17 being the IPv4 of the machine you have the tftp server running. I suggest to use btrfs as rootfs because that works well with U-Boot. Before finishing the installation put the dtb in the rootfs as /boot/dtb. To then boot into Debian do in U-Boot: 
setenv mmcboot=btrload mmc 0 0x01000000 boot/vmlinuz\; btrload mmc 0 0x02000000 boot/dtb\; btrload mmc 0 0x03000000 boot/initrd.img\; bootz 0x01000000 0x03000000:$filesize 0x02000000
setenv bootargs console=ttyS0,115200 rootfstype=btrfs rootdelay=2 root=/dev/mmcblk0p1 rootflags=commit=5 rw
saveenv
boot
Known issues: If you have problems, don't hesitate to contact me.

Jonathan Carter: A few impressions of DebConf 16 in Cape Town

DebConf16 Group Photo

DebConf16 Group Photo by Jurie Senekal.

DebConf16 Firstly, thanks to everyone who came out and added their own uniqueness and expertise to the pool. The feedback received so far has been very positive and I feel that the few problems we did experience was dealt with very efficiently. Having a DebConf in your hometown is a great experience, consider a bid for hosting a DebConf in your city! DebConf16 Open Festival (5 August) The Open Festival (usually Debian Open Day) turned out pretty good. It was a collection of talks, a job fair, and some demos of what can be done with Debian. I particularly liked Hetzner s stand. I got to show off some 20 year old+ Super Mario skills and they had some fun brain teasers as well. It s really great to see a job stand that s so interactive and I think many companies can learn from them. The demo that probably drew the most attention was from my friend Georg who demoed some LulzBot Mini 3D Printers. They really seem to love Debian which is great! DebConf (6 August to 12 August) If I try to write up all my thoughts and feeling about DC16, I ll never get this post finished. Instead, here as some tweets from DebConf that other have written:


Day Trip We had 3 day trips: Brought to you by
orga

DebConf16 Orga Team.

See you in Montr al! DebConf17 dates: The DC17 sponsorship brochure contains a good deal of information, please share it with anyone who might be interested in sponsoring DebConf! Media

7 November 2016

Daniel Pocock: Quickstart SDR with gqrx, GNU Radio and the RTL-SDR dongle

Software Defined Radio (SDR) provides many opportunities for both experimentation and solving real-world problems. It is not exactly a new technology but it has become significantly more accessible due to the increases in desktop computing power (for performing the DSP functions) and simultaneous reduction in the cost of SDR hardware. Thanks to the availability of a completely packaged gqrx and GNU Radio solution, you can now get up and running in less than half an hour and spending less than fifty dollars/pounds/euros. We provided a full demo of the Debian Hams gqrx solution at Mini DebConf Vienna (video) and hope to provide a similar demo at MiniDebConf Cambridge on the coming weekend of 12-13 November. gqrx is also available for Fedora users. Choosing hardware There are many different types of hardware, ranging from the low-cost RTL-SDR USB dongles to full duplex multi-transceiver systems. My recommendation is to start with an RTL-SDR dongle due to extremely low cost, this will give you an opportunity to reflect on the opportunities of this technology before putting money into one of the transceivers and their accessories. The RTL-SDR dongle also benefits from being a small self-contained solution that you can carry around and experiment with or demo just about anywhere. Important: Don't buy the cheapest generic RTL TV/radio receivers. It is absolutely essential to buy one of the units that has been explicitly promoted for SDR. These typically have a temperature compensated crystal oscillator (TCXO) which is absolutely essential for the reception of narrowband voice and digital signals. Without this, it is only possible to receive wideband broadcash FM radio and TV channels. For those who want to try it out with us at MiniDebConf Cambridge, Technofix has UK stock (online ordering), they are about 26. Getting gqrx up and running fast Note: to avoid the wrong kernel module being loaded automatically, it is recoemmended that you don't connect the RTL-SDR dongle before you install the packages. If you did already connect it, you may need to reboot or rmmod dvb_usb_rtl28xxu. If you are using a Debian jessie system, you can get all the necessary packages from jessie-backports. If you haven't already enabled backports, you can do so with a command like this:
$ sudo echo "deb http://ftp.ch.debian.org/debian jessie-backports main" >> /etc/apt/sources.list
 Make sure your local index is updated and then install the necessary packages:
$ sudo apt-get update
$ sudo apt-get install -t jessie-backports gqrx-sdr rtl-sdr
 Running it for the first time Once the packages are installed, connect the RTL-SDR dongle to the computer and then start the gqrx GUI from a terminal:
$ gqrx
 If the GUI fails to appear, look carefully at the error messages. It may be that the wrong kernel module has been loaded. The properties window appears, select the RTL-SDR dongle: Now the main screen will appear. Choose the wideband FM mode "WFM (mono)" and change the frequency to a value in the FM broadcast band such as 100MHz. Click the "Power on" button in the top left corner, just under the "File" menu, to start reception. Click in the middle of a strong signal to tune to that station. If you don't hear anything, check the squelch setting (it should be more negative than the signal strength value) and increase the Gain control at the bottom right hand side of the window. Looking for ham / amateur radio signals A popular band for hams is between 144 - 148 MHz (in some countries only a subset of this band is used). This is referred to as the two-meter band, as that is the wavelength at this frequency. Hams often use the narrowband FM mode in this band, especially with repeater stations. Change the "Mode" setting from "WFM" to "Narrow FM" and change the frequency to a value in the middle of the band. Look for signals in the radio spectrum and click on them to hear them. If you are not sure which part of the band to look in, search for the two-meter band plan for your country/region and look for the repeater output frequencies in the band plan.

Jaldhar Vyas: New Laptop / Problems with Windows part 896,324

I had mentioned previously that I had been forced to purchase a new laptop. I decided that I didn't want another Thinkpad. The Lenovo ones no longer have the high quality they had in the IBM days and while support is still pretty good by todays dismal standards it's not worth the premium price. (If I'm buying it with my own money that is.) I had heard good thing about Dells' Linux support so I looked into their offerings and ended up buying a Precision 7510. Mind you this model came with Windows 7 installed but I didn't mind. As I wanted to install Debian according to my own specs anyway, I was ok with just knowing that the hardware would be compatible. So I prepared a Jessie USB installation stick (This model doesn't have a CD/DVD drive.) and shrunk down the Windows installation (but not deleted it altogether for reasons to be explained below.) At this point it is traditional to give a long, tortured account of how Heaven and Earth had to be moved to get Linux installed. But that is a thing of the past. The combination of good hardware and the excellent work of the debian-installer team, made the setup a breeze with only a couple of minor bumps in the road. One is that the kernel on the Jessie cd was not quite up to snuff. Downloading 4.6.0 from backports did the trick. Post-install, to get the most out of my nifty new 4K display, I needed the latest, alas non-free, nvidia-drivers. And for stable wifi (I always install over ethernet for this reason) I had to install the firmware-iwlwifi package. Everything else even my printer either "just worked" or needed only minor fiddling around. Having used this machine for a while, the biggest problem I have is with the keyboard. It is nowhere near as tactile and comfortable to use as the old IBM Thinkpads. Even Lenovo Thinkpad keyboards are better. I'm a hunt-and-peck type myself but it is annoying. I think a real touch typist would hate it. The cursor and home, end, page up, page down etc. keys are in the wrong place and home and end are actually function keys. There is a pointer and a trackpad and two sets of mouse buttons which seems like a waste of space. In fact much space is wasted everywhere, space which could be used to improve the keyboard. Other than that I like it. The battery life is not the best but fairly good. It's a bit heavier than I was used to but I've gotten used to it. Although I didn't go with the SSD option, it is not that noisy; again you can get used to it. All in all, I think it is worth it for the price.
I installed Debian but I only really use it as a base to run VMWare Workstation. I occasionally have to support software across multiple platforms but I don't want the hassle or expense of multiple computers so I have Windows (the original installation upgraded to Windows 10) and Mac OS X running in VMs. Plus I have another VM running Kubuntu LTS for my day to day computing, another Debian install running sid for packaging, and Minix. Backups are as simple as making a snapshot of the VM. If something accidently gets screwed up, I can easily revert it back to a known good state. Ideally, I would like to replace VMWare with a free solution such as qemu or virtualbox etc. but as far as I know VMWare is far ahead in emulation capabilities (OpenGL support for example.) which is vital for efficiently using the proprietary OS's. Things were going swimmingly until a few days ago which brings me to part two of this post. I booted into the Windows 10 VM only to be greeted by a message from the Windows boot manager that "A component of the operating system has expired." I tried going back to a snapshot from September (when this definitely was working) but I still got the same thing. A bit of googling revealed this has happened to others and the advice seemed to be to reset the computers date and reinstall Windows 10. It took several tries but I finally got that done, completed the task I needed to do and shut it down. At the end of the day I shut the whole laptop down and thought no more of it. The next day I boot up and...where is grub? It seems that during the Windows reinstall, it had overwritten grub with the Windows boot loader. And while grub is nice enough to add an entry for Windows when detected, Windows does not extend the same courtesy to Linux. Ok time to bring out my trusty USB stick again and reinstall grub. Oops I've wiped it off to store other things. No matter, download another image and do it again. Reboot and...back in Windows. Fiddle around in the EFI settings until I can get it to boot from USB. Now i'm in the shell provided by debian-installer so I can mount and chroot my Linux partition and reinstall grub. Except no I can't because it is Luks encrypted. Ok apt-get install cryptsetup, open it with my passphrase and now I can mount the partion, chroot into and reinstall grub. Except no I can't because it is a logical volume group. Back to apt-get, install lvm2, vgscan (because of course I've forgotten the name of the group,) vgchange and now I can mount, chroot, etc. etc. Except no I can't. 

# mount /dev/mapper/vg00-root /mnt
# chroot /mnt
# grub-install /dev/sda
error: cannot find a device for /boot/grub (is /dev mounted?).
sigh 

# mount /dev/sda5 /boot
special device /dev/sda5 does not exist.
Well, /dev is mounted but it does indeed not contain a device called sda5. 

# /etc/init.d/udev start
udev requires a mounted procfs.  not started.
Very well then. 

# mount -t proc none /proc
# /etc/init.d/udev start
Nope. proc needs sysfs. 

# mount -t sysfs none /sys
# /etc/init.d/udev start
Still no. You get a warning about how it is a bad idea to run udev from an interactive shell and there is still not /dev/sda5. Time to start googling again. It turns out what I should have done is open another shell from the installer environment and do... 

# mount --bind /dev/ /mnt/dev
Now I can mount /boot/grub and reinstall grub and it should all work right? I should be so lucky. Ok back to square one. I now did what I should have done in the first place and searched the Debian wiki. Sure enough there is a page which deals exactly with my predicament. Finally I get everything installed correctly and triumphantly reboot into Linux. Of course now Windows doesn't work again...

Reproducible builds folks: Reproducible Builds: week 80 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday October 30 and Saturday November 5 2016: Upcoming events Reproducible work in other projects Bugs filed Reviews of unreproducible packages 81 package reviews have been added, 14 have been updated and 43 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated: 1 issue type has been removed: 1 issue type has been updated: Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by: diffoscope development buildinfo.debian.net development tests.reproducible-builds.org Reproducible Debian: Misc. Also with thanks to Profitbricks sponsoring the "hardware" resources, Holger created a 13 core machine with 24GB RAM and 100GB SSD based storage so that Ximin can do further tests and development on GCC and other software on a fast machine. This week's edition was written by Chris Lamb, Ximin Luo, Vagrant Cascadian, Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

Petter Reinholdtsen: How to talk with your loved ones in private

A few days ago I ran a very biased and informal survey to get an idea about what options are being used to communicate with end to end encryption with friends and family. I explicitly asked people not to list options only used in a work setting. The background is the uneasy feeling I get when using Signal, a feeling shared by others as a blog post from Sander Venima about why he do not recommend Signal anymore (with feedback from the Signal author available from ycombinator). I wanted an overview of the options being used, and hope to include those options in a less biased survey later on. So far I have not taken the time to look into the individual proposed systems. They range from text sharing web pages, via file sharing and email to instant messaging, VOIP and video conferencing. For those considering which system to use, it is also useful to have a look at the EFF Secure messaging scorecard which is slightly out of date but still provide valuable information. So, on to the list. There were some used by many, some used by a few, some rarely used ones and a few mentioned but without anyone claiming to use them. Notice the grouping is in reality quite random given the biased self selected set of participants. First the ones used by many: Then the ones used by a few. Then the ones used by even fewer people And finally the ones mentioned by not marked as used by anyone. This might be a mistake, perhaps the person adding the entry forgot to flag it as used? Given the network effect it seem obvious to me that we as a society have been divided and conquered by those interested in keeping encrypted and secure communication away from the masses. The finishing remarks from Aral Balkan in his talk "Free is a lie" about the usability of free software really come into effect when you want to communicate in private with your friends and family. We can not expect them to allow the usability of communication tool to block their ability to talk to their loved ones. Note for example the option IRC w/OTR. Most IRC clients do not have OTR support, so in most cases OTR would not be an option, even if you wanted to. In my personal experience, about 1 in 20 I talk to have a IRC client with OTR. For private communication to really be available, most people to talk to must have the option in their currently used client. I can not simply ask my family to install an IRC client. I need to guide them through a technical multi-step process of adding extensions to the client to get them going. This is a non-starter for most. I would like to be able to do video phone calls, audio phone calls, exchange instant messages and share files with my loved ones, without being forced to share with people I do not know. I do not want to share the content of the conversations, and I do not want to share who I communicate with or the fact that I communicate with someone. Without all these factors in place, my private life is being more or less invaded.

6 November 2016

Shirish Agarwal: The long tail in a common s man journey to debconf16 1

I was going to put a technical post but saw the discussion of one of the meetings of the debconf meets and decided to share a novice s travel experience. Before I start here s the discussion log http://meetbot.debian.net/debconf-team/2016/debconf-team.2016-10-20-20.01.log.html and specifically this part which hit me (using fake names for discussion as haven t taken permission from the folks to cite them by name.)
20:36:52 abcd: $100 CAD is a lot for some, but you d only need it if you won t sleep in sponsored accom, which arguably is acceptable.
20:37:04 it would, efgh, fixed sponsorship sum for everybody and allocation of rooms completely decoupled. Hotel gets the money from everybody and the base fee from DebConf.
20:37:15 people who can t afford also have special needs and may be uncomfortable in sharing rooms. That s quite frequently in our community. Managing each case will be much more complicated.
20:37:31 hijk: we could set aside budget for such special needs, for sure.
20:37:43 I m talking about managing each case
20:37:46 hijk: yes, but we ll have the special cases no matter what.
20:37:48 yes, and the special cases need to be catered for regardless of how everybody else is housed
20:38:06 hijk: room allocation already includes this.
20:38:19 people having to expose their personal problems to have us permitting them staying in the hotel
20:38:23 that s just too weird
It just goes on. I dunno whether I m weird or not or the experience I would share is just normal, this I would leave for you to decide. As have shared before, some friends of mine from the free software community had cajoled me last year to apply for debconf bursary (debconf15), which surprisingly got approved, but as it was late and my pre-conceived myths/notions of visas taking a looooong time decided not to go further. Many things take a long time to happen in the Indian bureaucratic maze. For instance have been in a civil case for almost a decade now among other things so know and accept that things take their own sweet time otherwise known as Indian patience Did the application and again, surprise, surprise this time too I was approved. Luckily, had done the application for bursary early so was a bit positive on the visa-front. There was a goof-up at the embassay but thanks to people at travel.stackexchange.com where I asked quite a few questions, I was a bit informed and travel was relatively hassle-free. Internally though, I was nervous as hell. I had been feeling like a conman or a fraud or being an imposter because I knew before-hand that the project is so huge and had done the mistake of putting up a talk and a workshop where the big guns would be, which again was accepted (not good). The only thing I was thinking of as a saving grace is that there might be some newbies who don t know about the project at all (on Open Day) and hopefully I could help with that but as you will see, even there I was fully inadequate. I live in Pune which is around 3.5 hours from Mumbai (BOM) from where international flights take off. While Pune has an Airport, due to defence considerations, there cannot be much improvement either for domestic or International carriers. There have been attempts to have an exclusive civilian Airport for a long time (almost a decade) and would still take a decade or more. Hence had decided to take an early morning train from Pune to Mumbai, change couple of locals and finally land up at the Mumbai International Airport. Hind-sight as they say is 50:50, while I do have friends in Mumbai, I also found about a homestay which is closer to the Airport and still relatively budget-friendly. Anyways, met few friends but as was paranoid about missing connections found myself in front of the Airport at 20:00 hrs. with about 7 hours + to go before my flight. While there is nothing to do around the airport rather than hanging around, just hung around outside the airport as knew that inside the airport will be chilling and once you go in, you cannot come out or at least it s an inconvenience to the security therein. The International Airport in on three levels, the basement is for vehicles, the first level to receive International and Domestic passengers and the upper-most level exclusively for people flying internationally. This again, came to know when I tried to enter into the ones meant for Domestic and International Passengers coming into the city. Came to the check-in counter at around 02:00 hrs, did the security thing and just had to wait as the flight was of 0400 hrs (from my limited search experience, the cheapest flights are at such times when nobody else (i.e.civilized people) wants to fly). Entered Doha around 5:15 Doha time and saw a much much bigger airport than either the Mumbai International Airport or/and the Delhi International Airport . While I have written some negative stuff about Doha, there were two positives that I am sure, I had forgotten to share a. There were no transit Visa Fees that I had to pay. Most countries and airports I researched have something called transit visa and that can really get expensive, so saved money on that. b. The free ride into the city and back with voluntary tipping the driver or/and guide (approx. 3-4 hours) While the second from what I could tell/know is a gimmick, this is something I wish other countries and airports emulate. There are hotels in the airport and I could have had hotel accommodation if I had booked a slightly more expensive ticket, roughly INR 5k/- each way which would have given me a bit more legroom as well as stay as my layover was more than 24 hours. But this information was known at last minute. Qatar Airways has just a toll-free number and trying more than a few times gave up. They don t have an office in the city. When I reached the check-in counter they said if I had upgraded to Y class I would have had the hotel thing. Changing tickets at the last moment was too expensive and anyways for hotel accommodation for layovers they required at least 24 hours notice. Had to make do with recliners and chairs which are not really comfortable. There were only a couple of waiting rooms on air-side which had a view of the aircraft and hence were a bit more pleasing than those which were on the land-side and were fully blocked without a view. I wish there was a map of the Airport from within the Airport as even with the single terminal it is really easy to get lost. Somehow the day and night went by and took my second flight and reached Cape Town, South Africa. Throughout the journey had been stressed as had to be awake at all times and make sure that nothing gets stolen. Having attendants at toilets were also good so that there is no possibility of any violence there. So it had been 2 days, no shower and no sleep. Later also came to know about Airport Sleeping Pods and shower stalls but these also seem to be less in number, at few airports and there always be a bit of premium attached to them as airports are a monopoly business. Anyways, reached the venue. Throughout the travel there was quite a bit of unnamed fear which I later came to know after seeing Dr. Ramanujan s The Man who knew Infinity . It was/is the fear or unknown, while in the movie it is articulated as fear of crossing seven seas, symbolically it is the fear or unknown. Now while I was dead tired, I still pushed myself as I didn t want to have the effects of jet lag interfere with the normal sleeping and waking patterns. I did freshen myself but didn t allow myself the luxury of the bath-tub as I knew that if I went in, I would not come out that day. Met all the people, learnt who s who, where things are happening etc. and slowly night came. Night came and I was so-looking forward to sleep but sleep was not to be. I later learnt it could be either of the two reasons, it could either have been travel-induced insomnia or/and what is known as the first night effect . It was only on the second day when I was in bath-tub for about 2/3 hours I could feel the tension leaving my body. I finally realized that I am in Cape Town, South Africa and could enjoy and be surprised at seeing birds within few feet of me . Now I don t know whether I m the only weird/paranoid one, I do know that it would not have been easier for me at least for the first night as I was turning and twisting throughout the night. I opened the lights, read for some time hoping for sleep to take over but that didn t work. Tried quite a few things but sleep didn t come. If I had been sleeping with other people I dunno how they would have reacted. I myself am a light sleeper (most of the time) and if I had sleep coming and somebody else acted or been the way I was, I wouldn t been able to sleep. However much you try, whatever is the natural reaction is, will be. There are still some bits to share but that would be in part 2.
Filed under: Miscellenous Tagged: #air-travel, #Debconf16, #paranoia, #sleep

Niels Thykier: Improvements in apt-file 3.1.2

Yesterday, I just uploaded apt-file 3.1.2 into unstable, which comes with a few things I would like to highlight. You can also set defaults in the config file if you want to always search in unstable, simply do: 
# echo 'apt-file::Search-Filter::Suite "unstable";' >> /etc/apt/apt-file.conf
For the suite filter, either a code name ( sid ) or a suite name ( unstable ) will work. Please note that the filters are case-sensitive suites/code names generally use all lowercase, whereas origins appear to use title-case (i.e. unstable vs. Debian ).
Filed under: apt-file, Debian

Russell Coker: Is a Thinkpad Still Like a Rolls-Royce

For a long time the Thinkpad has been widely regarded as the Rolls-Royce of laptops . Since 2003 one could argue that Rolls-Royce is no longer the Rolls-Royce of cars [1]. The way that IBM sold the Think business unit to Lenovo and the way that Lenovo is producing both Thinkpads and cheaper Ideapads is somewhat similar to the way the Rolls-Royce trademark and car company were separately sold to companies that are known for making cheaper cars. Sam Varghese has written about his experience with Thinkpads and how he thinks it s no longer the Rolls-Royce of laptops [2]. Sam makes some reasonable points to support this claim (one of which only applies to touchpad users not people like me who prefer the Trackpoint), but I think that the real issue is whether it s desirable to have a laptop that could be compared to a Rolls-Royce nowadays. Support The Rolls-Royce car company is known for great reliability and support as well as features that other cars lack (mostly luxury features). The Thinkpad marque (both before and after it was sold to Lenovo) was also known for great support. You could take a Thinkpad to any service center anywhere in the world and if the serial number indicated that it was within the warranty period it would be repaired without any need for paperwork. The Thinkpad service centers never had any issue with repairing a Thinkpad that lacked a hard drive just as long as the problem could be demonstrated. It was also possible to purchase an extended support contract at any time which covered all repairs including motherboard replacement. I know that not everyone had as good an experience as I had with Thinkpad support, but I ve been using them since 1998 without problems which is more than I can say for most hardware. Do we really need great reliability from laptops nowadays? When I first got a laptop hardly anyone I knew owned one. Nowadays laptops are common. Having a copy of important documents on a USB stick is often a good substitute for a reliable laptop, when you are in an environment where most people own laptops it s usually not difficult to find someone who will let you use theirs for a while. I think that there is a place for a laptop with RAID-1 and ECC RAM, it s a little known fact that Thinkpads have a long history of supporting the replacement of a CD/DVD drive with a second hard drive (I don t know if this is still supported) but AFAIK they have never supported ECC RAM. My first Thinkpad cost $3,800. In modern money that would be something like $7,000 or more. For that price you really want something that s well supported to protect the valuable asset. Sam complains about his new Thinkpad costing more than $1000 and needing to be replaced after 2.5 years. Mobile phones start at about $600 for the more desirable models (IE anything that runs Pokemon Go) and the new Google Pixel phones range from $1079 to $1,419. Phones aren t really expected to be used for more than 2.5 years. Phones are usually impractical to service in any way so for most of the people who read my blog (who tend to buy the more expensive hardware) they are pretty much a disposable item costing $600+. I previously wrote about a failed Nexus 5 and the financial calculations for self-insuring an expensive phone [3]. I think there s no way that a company can provide extended support/warranty while making a profit and offering a deal that s good value to customers who can afford to self-insure. The same applies for the $499 Lenovo Ideapad 310 and other cheaper Lenovo products. Thinkpads (the higher end of the Lenovo laptop range) are slightly more expensive than the most expensive phones but they also offer more potential for the user to service them. Features My first Thinkpad was quite underpowered when compared to desktop PCs, it had 32M of RAM and could only be expanded to 96M at a time when desktop PCs could be expanded to 128M easily and 256M with some expense. It had a 800*600 display when my desktop display was 1280*1024 (37% of the pixels). Nowadays laptops usually start at about 8G of RAM (with a small minority that have 4G) and laptop displays start at about 1366*768 resolution (51% of the pixels in a FullHD display). That compares well to desktop systems and also is capable of running most things well. My current Thinkpad is a T420 with 8G of RAM and a 1600*900 display (69% of FullHD), it would be nice to have higher resolution but this works well and it was going cheap when I needed a new laptop. Modern Thinkpads don t have some of the significant features that older ones had. The legendary Butterfly Keyboard is long gone, killed by the wide displays that economies of scale and 16:9 movies have forced upon us. It s been a long time since Thinkpads had some of the highest resolution displays and since anyone really cared about it (you only need pixels to be small enough that you can t see them). For me one of the noteworthy features of the Thinkpads has been the great keyboard. Mechanical keys that feel like a desktop keyboard. It seems that most Thinkpads are getting the rubbery keyboard design made popular by Apple. I guess this is due to engineering factors in designing thin laptops and the fact that most users don t care. Matthew Garrett has blogged about the issue of Thinkpad storage configured as RAID mode without any option to disable it [4]. This is an annoyance (which incidentally has been worked around) and there are probably other annoyances like it. Designing hardware and an OS are both complex tasks. The interaction between Windows and the hardware is difficult to get right from both sides and the people who design the hardware often don t think much about Linux support. It has always been this way, the early Thinkpads had no Linux support for special IBM features (like fan control) and support for ISA-PnP was patchy. It is disappointing that Lenovo doesn t put a little extra effort into making sure that Linux works well on their hardware and this might be a reason for considering another brand. Service Life I bought my curent Thinkpad T420 in October 2013 [5] It s more than 3 years old and has no problems even though I bought it refurbished with a reduced warranty. This is probably the longest I ve had a Thinkpad working well, which seems to be a data point against the case that modern Thinkpads aren t as good. I bought a T61 in February 2010 [6], it started working again (after mysteriously not working for a month in late 2013) and apart from the battery lasting 5 minutes and a CPU cooling problem it still works well. If that Thinkpad had cost $3,800 then I would have got it repaired, but as it cost $796 (plus the cost of a RAM upgrade) and a better one was available for $300 it wasn t worth repairing. In the period 1998 to 2010 I bought a 385XD, a 600E, a T21, a T43, and a T61 [6]. During that time I upgraded laptops 4 times in 12 years (I don t have good records of when I bought each one). So my average Thinkpad has lasted 3 years. The first 2 were replaced to get better performance, the 3rd was replaced when an employer assigned me a Thinkpad (and sold it to be when I left), and 4 and 5 were replaced due to hardware problems that could not be fixed economically given the low cost of replacement. Conclusion Thinkpads possibly don t have the benefits over other brands that they used to have. But in terms of providing value for the users it seems that they are much better than they used to be. Until I wrote this post I didn t realise that I ve broken a personal record for owning a laptop. It just keeps working and I hadn t even bothered looking into the issue. For some devices I track how long I ve owned them while thinking can I justify replacing it yet , but the T420 just does everything I want. The battery still lasts 2+ hours which is a new record too, with every other Thinkpad I ve owned the battery life has dropped to well under an hour within a year of purchase. If I replaced this Thinkpad T420 now it will have cost me less than $100 per year (or $140 per year including the new SSD I installed this year), that s about 3 times better than any previous laptop! I wouldn t feel bad about replacing it as I ve definitely got great value for money from it. But I won t replace it as it s doing everything I want. I ve just realised that by every measure (price, reliability, and ability to run all software I want to run) I ve got the best Thinkpad I ve ever had. Maybe it s not like a Rolls-Royce, but I d much rather drive a 2016 Tesla than a 1980 Rolls-Royce anyway.
Related posts:
  1. Thinkpad T420 I ve owned a Thinkpad T61 since February 2010 [1]. In...
  2. PC prices drop again! A few weeks ago Dell advertised new laptops for $849AU,...
  3. I Just Bought a new Thinkpad and the Lenovo Web Site Sucks I ve just bought a Thinkpad T61 at auction for $AU796....

3 November 2016

Simon Josefsson: Why I don t Use 2048 or 4096 RSA Key Sizes

I have used non-standard RSA key size for maybe 15 years. For example, my old OpenPGP key created in 2002. With non-standard key sizes, I mean a RSA key size that is not 2048 or 4096. I do this when I generate OpenPGP/SSH keys (using GnuPG with a smartcard like this) and PKIX certificates (using GnuTLS or OpenSSL, e.g. for XMPP or for HTTPS). People sometimes ask me why. I haven t seen anyone talk about this, or provide a writeup, that is consistent with my views. So I wanted to write about my motivation, so that it is easy for me to refer to, and hopefully to inspire others to think similarily. Or to provoke discussion and disagreement that s fine, and hopefully I will learn something. Before proceeding, here is some context: When building new things, it is usually better to use the Elliptic Curve technology algorithm Ed25519 instead of RSA. There is also ECDSA which has had a comparatively slow uptake, for a number of reasons that is widely available and is a reasonable choice when Ed25519 is not available. There are also post-quantum algorithms, but they are newer and adopting them today requires a careful cost-benefit analysis. First some background. RSA is an asymmetric public-key scheme, and relies on generating private keys which are the product of distinct prime numbers (typically two). The size of the resulting product, called the modulus n, is usually expressed in bit length and forms the key size. Historically RSA key sizes used to be a couple of hundred bits, then 512 bits settled as a commonly used size. With better understanding of RSA security levels, the common key size evolved into 768, 1024, and later 2048. Today s recommendations (see keylength.com) suggest that 2048 is on the weak side for long-term keys (5+ years), so there has been a trend to jump to 4096. The performance of RSA private-key operations starts to suffer at 4096, and the bandwidth requirements is causing issues in some protocols. Today 2048 and 4096 are the most common choices. My preference for non-2048/4096 RSA key sizes is based on the simple and na ve observation that if I would build a RSA key cracker, there is some likelihood that I would need to optimize the implementation for a particular key size in order to get good performance. Since 2048 and 4096 are dominant today, and 1024 were dominent some years ago, it may be feasible to build optimized versions for these three key sizes. My observation is a conservative decision based on speculation, and speculation on several levels. First I assume that there is an attack on RSA that we don t know about. Then I assume that this attack is not as efficient for some key sizes than others, either on a theoretical level, at implementation level (optimized libraries for certain characteristics), or at an economic/human level (decision to focus on common key sizes). Then I assume that by avoiding the efficient key sizes I can increase the difficulty to a sufficient level. Before analyzing whether those assumptions even remotely may make sense, it is useful to understand what is lost by selecting uncommon key sizes. This is to understand the cost of the trade-off. A significant burden would be if implementations didn t allow selecting unusual key sizes. In my experience, enough common applications support uncommon key sizes, for example GnuPG, OpenSSL, OpenSSH, FireFox, and Chrome. Some applications limit the permitted choices; this appears to be rare, but I have encountered it once. Some environments also restrict permitted choices, for example I have experienced that LetsEncrypt has introduced a requirement for RSA key sizes to be a multiples of 8. I noticed this since I chose a RSA key size of 3925 for my blog and received a certificate from LetsEncrypt in December 2015 however during renewal in 2016 it lead to an error message about the RSA key size. Some commercial CAs that I have used before restrict the RSA key size to one of 1024, 2048 or 4096 only. Some smart-cards also restrict the key sizes, sadly the YubiKey has this limitation. So it is not always possible, but possible often enough for me to be worthwhile. Another cost is that RSA signature operations are slowed down. This is because the exponentiation function is faster than multiplication, and if the bit pattern of the RSA key is a 1 followed by several 0 s, it is quicker to compute. I have not done benchmarks, but I have not experienced that this is a practical problem for me. I don t notice RSA operations in the flurry of all of other operations (network, IO) that is usually involved in my daily life. Deploying this on a large scale may have effects, of course, so benchmarks would be interesting. Back to the speculation that leads me to this choice. The first assumption is that there is an attack on RSA that we don t know about. In my mind, until there are proofs that the currently known attacks (GNFS-based attacks) are the best that can be found, or at least some heuristic argument that we can t do better than the current attacks, the probability for an unknown RSA attack is therefor, as strange as it may sound, 100%. The second assumption is that the unknown attack(s) are not as efficient for some key sizes than others. That statement can also be expressed like this: the cost to mount the attack is higher for some key sizes compared to others. At the implementation level, it seems reasonable to assume that implementing a RSA cracker for arbitrary key sizes could be more difficult and costlier than focusing on particular key sizes. Focusing on some key sizes allows optimization and less complex code. At the mathematical level, the assumption that the attack would be costlier for certain types of RSA key sizes appears dubious. It depends on the kind of algorithm the unknown attack is. For something similar to GNFS attacks, I believe the same algorithm applies equally for a RSA key size of 2048, 2730 and 4096 and that the running time depends mostly on the key size. Other algorithms that could crack RSA, such as some approximation algorithms, does not seem likely to be thwarted by using non-standard RSA key sizes either. I am not a mathematician though. At the economical or human level, it seems reasonable to say that if you can crack 95% of all keys out there (sizes 1024, 2048, 4096) then that is good enough and cracking the last 5% is just diminishing returns of the investment. Here I am making up the 95% number. Currently, I would guess that more than 95% of all RSA key sizes on the Internet are 1024, 2048 or 4096 though. So this aspect holds as long as people behave as they have done. The final assumption is that by using non-standard key sizes I raise the bar sufficiently high to make an attack impossible. To be honest, this scenario appears unlikely. However it might increase the cost somewhat, by a factor or two or five. Which might make someone target a lower hanging fruit instead. Putting my argument together, I have 1) identified some downsides of using non-standard RSA Key sizes and discussed their costs and implications, and 2) mentioned some speculative upsides of using non-standard key sizes. I am not aware of any argument that the odds of my speculation is 0% likely to be true. It appears there is some remote chance, higher than 0%, that my speculation is true. Therefor, my personal conservative approach is to hedge against this unlikely, but still possible, attack scenario by paying the moderate cost to use non-standard RSA key sizes. Of course, the QA engineer in me also likes to break things by not doing what everyone else does, so I end this with an ObXKCD.

Jan Wagner: Container Orchestration Thoughts

Container Orchestration ThoughtsSince some time everybody (read developer) want to run his new microservice stacks in containers. I can understand that building and testing an application is important for developers.
One of the benefits of containers is, that developer (in theory) can put their new version of applications into production on their own. This is the point where operations is affected and operations needs to evaluate, if that might evolve into better workflow. For yolo^WdevOps people there are some challenges that needs to be solved, or at least mitigated, when things needs to be done in large(r) scale.

Orchestration Engine Running Docker, which is actual the most preferred container solution, on a single host with docker command line client is something you can do, but there you leave the gap between dev and ops.

UI For Docker Since some time there is UI For Docker available for visualizing and managing containers on a single docker node. It's pretty awesome and the best feature so far is the Container Network view, which also shows the linked container. Container Orchestration Thoughts

Portainer Portainer is pretty new and it can be deployed as easy as UI For Docker. But the (first) great advantage: it can handle Docker Swarm. Beside that it has many other great features. Container Orchestration Thoughts

Rancher Rancher describes themselves as 'container management platform' that 'supports and manages all of your Kubernetes, Mesos, and Swarm clusters'. This is great because this are all of the relevant docker cluster orchestrations at the market actually. Container Orchestration Thoughts For the use cases, we are facing, Kubernetes and Mesos seems both like bloated beasts. Usman Ismail has written a really good comparison of Orchestration Engine options which goes into details. Container Orchestration Thoughts

Docker Swarm As there is actually no clear defacto standard/winner of the (container) orchestration wars, I would prevent to be in a vendor lock-in situation (yet). Docker swarm seems to be evolving and is getting more nice features other competitors doesn't provide.
Due the native integration into the docker framework and great community I believe Docker Swarm will be the Docker Orchestration of the choice on the long run. This should be supported by Rancher 1.2 which is not released yet.
From this point of view it looks very reasonable that Docker Swarm in combination with Rancher (1.2) might be a good strategy to maintain your container farms in the future. If you think to put Docker Swarm into production in the actual state, I recommend to read Docker swarm mode: What to know before going live on production by Panjamapong Sermsawatsri.

Persistent Storage While it is a best practice to use data volume container these days, providing persistent storage across multiple hosts for shared volumes seems to be tricky. In theory you can mount a shared-storage volume as a data volume and there are several volume plugins which supports shared storage. For example you can use the convoy plugin which gives you:
  • thin provisioned volumes
  • snapshots of volumes
  • backup of snapshots
  • restore volumes
As backend you can use:
  • Device Mapper
  • Virtual File System(VFS)/Network File System(NFS)
  • Amazon Elastic Block Store(EBS)
The good thing is, that convoy is integrated into Rancher. For more information I suggest to read Setting Up Shared Volumes with Convoy-NFS, which also mentions some limitations. If you want test Persistent Storage Service, Rancher provides some documentation. Actually I did not evaluate shared-storage volumes yet, but I don't see a solution I would love to use in production (at least on-premise) without strong downsides. But maybe things will go further and there might be a great solution for this caveats in the future.

Keeping base images up-to-date Since some time there are many projects that tries to detect security problems in your container images in several ways.
Beside general security considerations you need to deal somehow with issues in your base images that you build your applications on. Of course, even if you know you have a security issue in your application image, you need to fix it, which depends on the way how you based your application upon.

Ways to base your application image
  • You can build your application image entire from scratch, which leaves all the work to your development team and I wouldn't recommend it that way.
  • You also can create one (or more) intermediate image(s) that will be used by your development team.
  • The development team might ground their work on images in public available or private (for example the one bundled to your gitlab CI/CD solution) registries.

Whats the struggle with the base image? If you are using images being not (well) maintained by other people, you have to wait for them to fix your base image. Using external images might also lead into trust problems (can you trust those people in general?).
In an ideal world, your developers have always fresh base images with fixed security issues. This can probably be done by rebuilding every intermediate image periodically or when the base image changes.

Paradigm change Anyway, if you have a new application image available (with no known security issues), you need to deploy it to production. This is summarized by Jason McKay in his article Docker Security: How to Monitor and Patch Containers in the Cloud:
To implement a patch, update the base image and then rebuild the application image. This will require systems and development teams to work closely together.
So patching security issues in the container world changes workflow significant. In the old world operation teams mostly rolled security fixes for the base systems independent from development teams.
Now hitting containers the production area this might change things significant.

Bringing updated images to production Imagine your development team doesn't work steady on a project, cause the product owner consider it feature complete. The base image is provided (in some way) consistently without security issues. The application image is build on top of that automatically on every update of the base image.
How do you push in such a scenario the security fixes to production? From my point of view you have two choices:
  • Let the development team require to test the resulting application image and put it into production
  • Push the new application image without review by the development team into production
The first scenario might lead into a significant delay until the fixes hit production created by the probably infrequent work of the development team. The latter one brings your security fixes early to production by the notable higher risk to break your application. This risk can be reduced by implementing massive tests into CI/CD pipelines by the development team. Rolling updates provided by Docker Swarm might also reduce the risk of ending with a broken application. When you are implementing an update process of your (application) images to production, you should consider Watchtower that provides Automatic Updates for Docker Containers.

Conclusion Not being a product owner or the operations part of an application that is facing a widely adopted usage that would compensate the actual tradeoffs we are still facing I tend not to move large scale production projects into a container environment.
This means not that this might be a bad idea for others, but I'd like to sort out some of the caveats before. I'm still interested to put smaller projects into production, being not scared to reimplement or move them on a new stack.
For smaller projects with a small number of hosts Portainer looks not bad as well as Rancher with the Cattle orchestration engine if you just want to manage a couple of nodes. Things are going to be interesting if Rancher 1.2 supports Docker swarm cluster out of the box. Let's see what the future will bring us to the container world and how to make a great stack out of it.

Update I suggest to read Docker in Production: A History of Failure and the answer Docker in Production: A retort to understand the actual challenges when running Docker in larger scale production environments.

2 November 2016

Reproducible builds folks: Reproducible Builds: week 79 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday October 23 and Saturday October 29 2016: Upcoming events The second Reproducible Builds World Summit will be held from December 13th-15th in Berlin! See the link for more details. Other events: Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle, USA on November 12th, 2016. Reproducible Debian Hackathon - A small hackathon organized in Boston, USA on December 3rd and 4th. If you are interested in attending, contact Valerie Young - spectranaut in the #debian-reproducible IRC channel on irc.oftc.net. IRC meeting The next IRC meeting will be held on 2016-11-01 at 18:00 UTC. The meeting after that will be held on 2016-11-15 at 18:00 UTC. Reproducible work in other projects Ximin Luo has had his fix to bug 77985 accepted into GCC. This is needed to be able to write test cases for patches to make GCC produce debugging symbols that are reproducible regardless of the build path. There was continued discussion on the mailing list regarding our build path proposals. It has now been decided to use an environment variable SOURCE_PREFIX_MAP instead of the older proposal SOURCE_ROOT_DIR. This would be similar to GCC's existing -fdebug-prefix-map option, which allows for better disambiguation between paths from different packages. mandoc's makewhatis is now reproducible. It is used by all the BSDs, including FreeBSD, as well as Alpine Linux and Void Linux. Packages reviewed and fixed, and bugs filed Chris Lamb: Reiner Herrmann: Reviews of unreproducible packages 145 package reviews have been added, 608 have been updated and 94 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated: Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by: tests.reproducible-builds.org Debian: General: diffoscope development Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

1 November 2016

Jonathan Wiltshire: Reflecting on a year of regular, public IRC meetings

The release team first started holding a regular, public planning and status meeting a little over a year ago, in September 2015. At that time, FTP masters had experimented along similar lines and I took some inspiration from that, including the keeping of proper minutes that anyone can look at. I wanted to open up our discussion processes and allow other developers and users to see (and perhaps influence) our plans for the release taking shape month by month, and how we reached certain decisions with a lot of mature discussion and not just on a whim. A secondary aim, since we are quite geographically distributed and getting together for same-room meetings is hard, was to bring more accountability to ourselves when we decided something ought to happen; if it s in the minutes, there s no escaping someone asking so what happened to ? . That s worked better for us on some topics than on others. Finally, public minutes mean that anyone who might be interested in joining the team can see easily what we re up to and how we shape the release throughout the cycle. That might help lower the barrier to entry, which can only be good for the team. I had hoped that regular meetings would inspire other teams to do similar; I haven t seen any indication of that to date (though perhaps it s just down to awareness). The Reproducible Builds contributors held fortnightly meetings for a period in 2015, though not inspired by ours, and I heard recent talk of starting those again. I still think that there is plenty of scope to improve the transparency of core teams in general in Debian, but also that regular meetings aren t going to work for every team. A regular slot which is not varied except when absolutely necessary, is essential for avoiding the temptation to just push it back another week when things are busy. In our office we have an allegedly-regular Thursday afternoon slot for technical demonstrations, which has suffered from exactly that problem for a long time now, and I wanted to avoid that issue. We have a calendar to remind us when each meeting is due, along with other important events like freeze milestones. Our slot is the fourth Wednesday of the month, a fairly arbitrary choice which seems to have worked out quite well. Time zones are more of an issue, even within Europe. We have mostly used a European evening time, but that s not very helpful further West where it s in the middle of the working day, or the middle of the night if you re further East (that one fortunately isn t an issue for us so far). Even within Europe it s difficult, as we have to try and balance commuting time in the UK with dinner on the continent, or dinner with late evening, or adjust for saving changes, or you get the idea. If we gained a far-eastern team member one day, this would be a real issue. We use Meetbot for recording the minutes. I have heard criticism that it publicly archives IRC logs to the web essentially forever, but for us that s the whole point. With a little practice and discipline it does generate really nice minutes, with a bullet summary of the important parts, a summary of actions agreed and a log of the conversation for detailed reference. Anybody reading them can see how we reached a conclusion, and I m of the view that goes some way to avoiding a reputation for cabal-ism. It does pay to use the #info, #agree and #action tags liberally, but other things are slightly unnatural like always remembering to use a URL at the beginning of a line and not in the middle of a sentence, or Meetbot doesn t notice it. Practice goes a long way. I ve naturally fallen into chairing most meetings, for better or worse the consistency seems beneficial, but I worry that I m dominating the discussion sometimes. Discipline in making sure everybody has been included is something I ve had to get better at. It s essential to have a public agenda and to stick to it, and it should include some stock items at the start and end (including making sure the URL to the previous minutes has been given, reviewing outstanding actions, and arranging the next meeting before ending the current one). There is some skill in judging the agenda length and deciding which items can be deferred to make sure it doesn t drag on too late we ve found anything more than an hour is far too long, and between 45 and 60 minutes is pushing it. Getting some easy topics out of the way before starting one which is more contentious can be helpful to avoid having to defer them later. I circulate the URL to the minutes and the date of the next meeting publicly on the mailing list immediately after each meeting, or as soon as possible. With little feedback, I have no idea if our meetings are helpful to those outside the team or not. We do still hold in-person meetings from time to time when we re all together, because they re useful for some circumstances (like some genuinely private topics we occasionally need to discuss, or for sprinting). I would hope that public meetings inspire confidence that we re on top of the release process, that they show we have a mature and transparent decision making process (for example, in deciding to move the freeze date to accommodate an external release schedule as a one-off, and subsequently deciding to not move it back when circumstances changed), and mostly that other teams might benefit for the same reasons. But I can also see that they make more sense in a team with a defined project cycle than they might in one which is more administrative or where work is more sporadic (no point holding a meeting for the sake of it, after all).

31 October 2016

Steve McIntyre: Twenty years...

So, it's now been twenty years since I became a Debian Developer. I couldn't remember the exact date I signed up, but I decided to do some forensics to find out. First, I can check on the dates on my first Debian system, as I've kept it running as a Debian system ever since! 
jack:~$ ls -alt /etc
...
-rw-r--r--   1 root   root     6932 Feb 10  1997 pine.conf.old
-rw-r--r--   1 root   root     6907 Dec 29  1996 pine.conf.old2
-rw-r--r--   1 root   root    76739 Dec  7  1996 mailcap.old
-rw-r--r--   1 root   root     1225 Oct 20  1996 fstab.old
jack:~$
I know that I did my first Debian installation in late October 1996, migrating over from my existing Slackware installation with the help of my friend Jon who was already a DD. That took an entire weekend and it was painful, so much so that several times that weekend I very nearly bailed and went back. But, I stuck with it and after a few more days I decided I was happier with Debian than with the broken old Slackware system I'd been using. That last file (fstab.old) is the old fstab file from the Slackware system, backed up just before I made the switch. I was already a software developer at the time, so of course the first thing I wanted to do once I was happy with Debian was to become a DD and take over the Debian maintenance of mikmod, the module player I was working on at the time. So, I mailed Bruce to ask for an account (there was none of this NM concept back then!) and I think he replied the next day. Unfortunately, I don't have the email in my archives any more due to a disk crash back in the dim and distant past. But I can see that the first PGP key I generated for the sake of joining Debian dates from October 30th 1996 which gives me a date of 31st October 1996 for joining Debian. Twenty years, wow... Since then, I've done lots in the project. I'm lucky enough to been to 11 DebConfs, hosted all around the world. I'm massively proud to have been voted DPL for two of those twenty years. I've worked on a huge number of different things in Debian, from the audio applications I started with to the installer (yay, how things come back to bite you!), from low-level CD and DVD tools (and making our CD images!) to a wiki engine written in python. I've worked hard to help make the best Operating System on the planet, both for my own sake and the sake of our users. Debian has been both excellent fun and occasionally a huge cause of stress in my life for the last 20 years, but despite the latter I wouldn't go back and change anything. Why? Through Debian, I've made some great friends: in Cambridge, in the UK, in Europe, on every continent. Thanks to you all, and here's to (hopefully) many years to come!

Next.

Previous.