Search Results: "will"

10 August 2015

Mirco Bauer: Smuxi 1.0 "Finally" Release

RSS Atom
Smuxi - IRC Client Smuxi 1.0 with message persistence, counters
And here we go again! We're proud to announce the new version of Smuxi, release 1.0 "Finally". During the development, 20 bug reports and 10 feature requests in 285 commits were worked on.

Finally 1.0 Smuxi is celebrating its 10th anniversary! 10 years ago, Mirco Bauer made the first commit to the Smuxi source code repository and is still very committed to it. He started the Gnosmirc project in 2005 when the only way a 24/7 "always-on" experience with IRC meant you had to use a console based IRC client like bitchx, irssi or epic combined with screen and SSH. This looks very practical at first and is a powerful Unix-ish way of accomplishing that job, but it has the big downside that it doesn't integrate with a desktop environment like GNOME. A bit later the Gnosmirc project was renamed to Smuxi when the new code architecture allowed other frontend implementations besides the GNOME one. The ncurses/STFL based text frontend was later implemented and is considered stable and useful enough for day to day use, but still has some rough edges. WinForms and WPF frontends also exist but need more work to reach a usable state. At this point Smuxi 1.0 contains all features that we could have imagined and even goes beyond with very advanced features like message patterns or language agnostic scripting.

Changes since Smuxi 0.11

Message Persistence One of the biggest drawbacks of the IRC protocol ever was that messages can't be retrieved from the IRC server because the IRC server is simply relaying messages to the connected clients. So, if an IRC client is freshly started and connects it starts to receive new messages, but all message you had received before are no longer available. This always made IRC in a way "volatile" unlike other communication systems like email where messages are relayed and stored on the client side. One common approach for IRC clients is to store log files in a text file. This is a simple feature and gives the user the possibility to read older conversations. Smuxi also supports text file logging like other IRC clients but it has a big user experience drawback as you need to open the file from the disk outside of the IRC client. In Smuxi 1.0 messages sent and received are now stored on the disk in a way they can automatically be retrieved/loaded when you restart Smuxi. It is like you have never closed Smuxi! This feature was already available in Smuxi for some time as a technical preview and it used the Db4o object database, but we were never happy about the performance neither with the stability so it always stayed an optional feature you need to enable. This year we tried a new message buffer backend using the famous SQLite database and it works much faster and stable as a rock. So finally we can enable this feature by default because it just works and enhanced your experience. We hope you enjoy it. Documentation of how you can change Smuxi message buffer backend and behavior can be found here. For instructions how to convert your existing db4o history to SQLite can be found in the "smuxi-message-buffer tool" section.

User Interface Enhancements
  • Synced message markers: the position of of the seen/unseen messages marker is pushed to the smuxi-server and remembered when the frontend reconnects. (Sebastian Poeplau)
  • Persistent message markers: the message marker position is also remembered across Smuxi(-server) restarts.
  • Message Counter: in addition to the highlight counter next to a chat new/unseen messages are also counted. This makes it easy to identify chats with much traffic.
  • Single application instance support. If you start Smuxi again from the menu it will bring the existing instance into the foreground. This makes the Ubuntu Messaging Menu much nicer.
  • The command/message entry is alignment with the messages. (Lex Berezhny)

Text Frontend Enhancements
  • The console background color can now be configured using: /config set STFL/Interface/TerminalBackgroundColor = #000000 (Ond ej Ho ek)
  • The text color contrast if nicks with the background is now ensured (Ond ej Ho ek) #1033
  • Messages containing images will not be skipped but their alternative text is shown instead (Ond ej Ho ek) #1035

New smuxi-message-buffer tool This is a new commandline tool that allows you to convert and export the message history of Smuxi message buffer files. This can be used to convert your existing Db4o history to SQLite like this for example: 
for DB_DB4O in $HOME/.local/share/smuxi/buffers/*/*/*/*.db4o; do
    DB_SQLITE=$ DB_DB4O/.db4o/.sqlite3 
    smuxi-message-buffer convert $DB_DB4O $DB_SQLITE
done
Smuxi shouldn't be running when using this tool.

Scripting Enhancements

New Hook Points Smuxi 1.0 supports with the following new hook points:
  • engine/protocol-manager/on-presence-status-changed/ This hook point is raised when the presence status of a protocol manager changes. This happens for example when an IRC connection toggles the away state.
  • engine/session/on-event-message/ This hook point raises event messages that usually begin with "-!-". This can be useful to track state changes that are shown as a message without having a dedicated hook point for it.
  • engine/session/command-$cmd/ This hook point is raised on the engine side for commands, e.g. /some_command that isn't handled by the frontend or engine built-in commands. This is useful for commands that should be available for all frontends and isn't specific to the frontend environment.

New Plugins The following new plugins are supported by Smuxi 1.0:
  • topic-diff: Shows the word differences of the topic after topic changes. (meebey)
  • away-nick: Automatically appends and removes $AWAY_SUFFIX to/from the nick name when you go away using the /away command or by disconnecting all frontends from the smuxi-server. (meebey)
  • system-info: Shows system info. Includes system kernel version, distro name, and CPU vendor information. (AK0)
  • now-playing: This plugin is not new but was rewritten in Python to get rid of the spaghetti code monster which was written in Bash. (jamesaxl)

IRC Enhancements
  • NICKSERV support Notices from Nick/ChanServ are no longer shown on all channels as they like to send greeting messages and other spam which is annoying to see on all channels. #868
  • Automatic rejoin of channels protected with a key works as expected again
  • Connecting to irc.gitter.im is now supported. Gitter's IRCd implementation has a bug in the IRC protocol which is now tolerated.

Twitter Enhancements
  • The /search command shows tweets as live stream
  • Added /delete, /favorite and /unfavorite commands

Behind the Scenes
  • Re-licensed smuxi-common from GPLv2 to MIT/X11

Contributors Contributors to this release are the following people:
  • Mirco Bauer (199 commits)
  • Carlos Mart n Nieto (15 commits)
  • Andr s G. Aragoneses (14 commits)
  • Piotr Dr g (12 commits)
  • Ond ej Ho ek (11 commits)
  • Oliver Schneider (5 commits)
  • Calvin B (4 commits)
  • Victor Seva (3 commits)
  • Will Johansson (2 commits)
  • Sebastian Poeplau (2 commits)
  • Julian Taylor (2 commits)
  • James Axl (2 commits)
  • Daniel Mustieles (2 commits)
  • Christopher James Halse Rogers (2 commits)
  • . Uzun (1 commit)
  • Lex Berezhny (1 commit)
  • Kalle Kaitala (1 commit)
  • Jordi Mas (1 commit)
  • Joe Hansen (1 commit)
  • Jimmie Elvenmark (1 commit)
  • Dimitris Spingos (1 commit)
  • Dean Lee (1 commit)
  • Cl ment Bourgeois (1 commit)
  • Carlos Hernandez (1 commit)
Thank you very much for your contributions to Smuxi! Want this? Go here and grab it right now!

Posted Sun Aug 9 17:48:18 2015

9 August 2015

Yves-Alexis Perez: WPS and Network Manager

So, everybody knows that WPS (Wi-Fi Protected Setup) is broken. But sometimes, you don't own the access point, and you'd just want the wireless to work. That happens for example when you're a guest in some place using an Orange Livebox and you don't have the WPA passphrase (usually because it's written somewhere you don't have access too, or because someone forgot to tell you). Liveboxes WPS is the press button thing: you press a button on the front for one second, then any device can connect in the next two minutes. That works fine with Android devices, for example, but it didn't work with my laptop and NetworkManager, which doesn't support WPS at all. Fortunately, the underlying piece of software (wpa_supplicant) does support WPS, and even the push button style. And you can nicely ask it to reveal the passphrase to you with the following trick.
  1. Disconnect NetworkManager from the network, disable the wireless link, stop it; just make sure wpa_supplicant is not running;
  2. Put a stub wpa_supplicant.conf file with only the following content: 
    update_config=1
  3. Start wpa_supplicant in the foreground with your stub config file: 
    wpa_supplicant -iwlan0 -c wpa_supplicant.conf
  4. Start wpa_cli
Inside wpa_cli:
  1. Scan the network: 
    scan
  2. Get the results: 
    scan_results
    and identity the bssid of the Livebox
  3. Press the WPS button on the Livebox
  4. Run 
    wps_pbc <bssid>
    ; some text should appear in the wpa_cli window, and it should eventually connect successfully (at that point you can even run a dhclient on wlan0)
  5. Run 
    save_config
The last command will update your stub configuration file, adding a new network block with the passphrase in the clear. You can then use that passphrase inside Network Manager if it's more convenient for you. There might be something easier, but at least it worked just fine for me during the holidays.

DebConf team: TecKids @ DebConf: Register Until Tuesday (Posted by DebConf Team)

We are happy to announce that TecKids, a non-profit specialising in working with kids, teaching them technonology, and fostering self-sustaining communities amongst them, will be holding a workshop from 15.08. to 18.08. The focus is on kids aged 10-15 years old, but kids aged 8-16 years old are welcome to attend if they can follow the course without supervision of theirs parents. Admission is free of charge, but registration through TecKids web form is mandatory. Registration is open until Tuesday, 11 August. The rough schedule for now is: You will be kept up to date on any changes to the schedule and other details by the TecKids team after registration. It is important that you check mail before heading to DebConf in order to receive any last-minute informations.

Craig Small: procps 3.3.11

I have updated NEWS, bumped the API and tagged in git; procps version 3.3.11 is now released! This release we have fixed many bugs and made procps more robust for those odd corner cases. See the NEWS file for details. The most significant new feature in this release is the support for LXC containers in both ps and top. The source files can be found at both sourceforge and gitlab at: My thanks to the procps co-maintainers, bug reporters and merge/patch authors. What s Next? There has been a large amount of work on the library API. This is not visible to this release as it is on a different git branch called newlib. The library is getting a complete overhaul and will look completely different to the old libproc/libprocps set. A decision hasn t been made when newlib branch will merge into master, but we will do it once we re happy the library and its API have settled. This change will be the largest change to procps library in its 20-odd year history but will mean the library will use common modern practices for libraries.

Petter Reinholdtsen: First paper version of the Norwegian Free Culture book heading my way

Typesetting a book is harder than I hoped. As the translation is mostly done, and a volunteer proof reader was going to check the text on paper, it was time this summer to focus on formatting my translated docbook based version of the Free Culture book by Lawrence Lessig. I've been trying to get both docboox-xsl+fop and dblatex to give me a good looking PDF, but in the end I went with dblatex, because its Debian maintainer and upstream developer were responsive and very helpful in solving my formatting challenges. Last night, I finally managed to create a PDF that no longer made Lulu.com complain after uploading, and I ordered a text version of the book on paper. It is lacking a proper book cover and is not tagged with the correct ISBN number, but should give me an idea what the finished book will look like. Instead of using Lulu, I did consider printing the book using CreateSpace, but ended up using Lulu because it had smaller book size options (CreateSpace seem to lack pocket book with extended distribution). I looked for a similar service in Norway, but have not seen anything so far. Please let me know if I am missing out on something here. But I still struggle to decide the book size. Should I go for pocket book (4.25x6.875 inches / 10.8x17.5 cm) with 556 pages, Digest (5.5x8.5 inches / 14x21.6 cm) with 323 pages or US Trade (6x8 inches / 15.3x22.9 cm) with 280 pages? Fewer pager give a cheaper book, and a smaller book is easier to carry around. The test book I ordered was pocket book sized, to give me an idea how well that fit in my hand, but I suspect I will end up using a digest sized book in the end to bring the prize down further. My biggest challenge at the moment is making nice cover art. My inkscape skills are not yet up to the task of replicating the original cover in SVG format. I also need to figure out what to write about the book on the back (will most likely use the same text as the description on web based book stores). I would love help with this, if you are willing to license the art source and final version using the same CC license as the book. My artistic skills are not really up to the task. I plan to publish the book in both English and Norwegian and on paper, in PDF form as well as EPUB and MOBI format. The current status can as usual be found on github in the archive/ directory. So far I have spent all time on making the PDF version look good. Someone should probably do the same with the dbtoepub generated e-book. Help is definitely needed here, as I expect to run out of steem before I find time to improve the epub formatting. Please let me know via github if you find typos in the book or discover translations that should be improved. The final proof reading is being done right now, and I expect to publish the finished result in a few months.

Simon Kainz: DUCK challenge: week 5

Slighthly delayed, but here are the stats for week 5 of the DUCK challenge: So we had 10 packages fixed and uploaded by 10 different uploaders. A big "Thank You" to you!! Since the start of this challenge, a total of 59 packages, were fixed. Here is a quick overview:
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7
# Packages 10 15 10 14 10 - -
Total 10 25 35 49 59 - -
The list of the fixed and updated packages is availabe here. I will try to update this ~daily. If I missed one of your uploads, please drop me a line. Only 2 more weeks to DebConf15 so please get involved: The DUCK Challenge is running until end of DebConf15! Pevious articles are here: Week 1, Week 2, Week 3, Week 4.

8 August 2015

Matthew Garrett: Difficult social problems are still difficult problems

After less than a week of complaints, the TODO group have decided to pause development of their code of conduct. This seems to have been triggered by the public response to the changes I talked about here, which TODO appear to have been completely unprepared for.

While disappointing in a bunch of ways, this is probably the correct decision. TODO stumbled into this space with a poor understanding of the problems that they were trying to solve. Nikki Murray pointed out that the initial draft lacked several of the key components that help ensure less privileged groups can feel that their concerns are taken seriously. This was mostly rectified last week, but nobody involved appeared to be willing to stand behind those changes in a convincing way. This wasn't helped by almost all of this appearing to land on Github's plate, with the rest of the TODO group largely missing in action[1]. Where were Google in this? Yahoo? Facebook? Left facing an angry mob with nobody willing to make explicit statements of support, it's unsurprising that Github would try to back away from the situation.

But that doesn't remove their blame for being in the situation in the first place. The statement claims
We are consulting with stakeholders, community leaders, and legal professionals, which is great. It's also far too late. If an industry body wrote a new kernel from scratch and deployed it without any external review, then discovered that it didn't work and only then consulted any of the existing experts in the field, we'd never take them seriously again. But when an industry body turns up with a new social policy, fucks up spectacularly and then goes back to consult experts, it's expected that we give them a pass.

Why? Because we don't perceive social problems as difficult problems, and we assume that anybody can solve them by simply sitting down and talking for a few hours. When we find out that we've screwed up we throw our hands in the air and admit that this is all more difficult than we imagined, and we give up. We ignore the lessons that people have learned in the past. We ignore the existing work that's been done in the field. We ignore the people who work full time on helping solve these problems.

We wouldn't let an industry body with no experience of engineering build a bridge. We need to accept that social problems are outside our realm of expertise and defer to the people who are experts.

[1] The repository history shows the majority of substantive changes were from Github, with the initial work appearing to be mostly from Twitter.

comment count unavailable comments

Christoph Egger: What's wrong with the web? -- authentication

The problem One problem to solve when doing web authentication has always been one identity provider, so you don't have to remember which username (or email address) you used for that bugtracker you used three years ago or that website. And tie it to one login ideally. Five years ago this problem seemed to be basically solved. There was OpenID and while it may not have been great it worked. You could have your own provider, your institution (university, company, foss project, ..) could have one and you could use your university-provided ID for all university stuff. Today's state Looking at the problem again today and the situation seems to have changed. To the worse. A lot. People are actively removing OpenID support. There seemed to be a replacement with, at least, proper design goals: Mozilla's persona. However this seems to be a dead end, no-one (almost) actually supports it. Then there is what people call OAuth2. However there does not seem to be such a thing as OAuth2 at all, at least not for logging into websites. So for example phabricator supports 12 different OAuth2 systems. That includes Google, Facebook, Twitter, Amazon Github and a whole bunch of other services. Each with a different implementation in the webapp of course. And of course you can not just have your university/company/.. provide an OAuth2 service for you to use -- you would need to write yet another adapter on the (foreign) website to talk to your implementation and your provider. And the strange thing, people seem to still consider OAuth2 a replacement for OpenID while it does not even provide the core functionality of the older system. Plus there does not seem to be any awareness of that all together. Other features Now of course, OpenID is not (and never was) the ultimate answer to the web authentication problem. The most obvious problem being user tracking. Your identity provider will see every website you log into, will see when you log into it and even be able to log into that website with your credentials. Of course, this problem is fully inherited by OAuth2. And in contrast to OpenID you can no longer run your own provider whom you can fully trust and who already knows about your surfing habits (because it's actually you already). Mozilla's persona might have solved that, they at least intended to. But, again, persona seems quite dead.

Wouter Verhelst: Backing up with tar

The tape archiver, better known as tar, is one of the older backup programs in existence. It's not very good at automated incremental backups (for which bacula is a good choice), but it can be useful for "let's take a quick snapshot of the current system" type of situations. As I'm preparing to head off to debconf tomorrow, I'm taking a backup of my n-1 laptop (which still contains some data that I don't want to lose) so it can be reinstalled and used by the Debconf video team. While I could use a "proper" backup system, running tar to a large hard disk is much easier. By default, however, tar won't preserve everything, so it is usually a good idea to add some extra options. This is what I' mrunning currently: 
sudo tar cvpaSf player.local:carillon.tgz --rmt-command=/usr/sbin/rmt --one-file-system /
which breaks down to create tar archive, verbose output, preserve permissions, automatically determine compression based on file extension, handle Sparse files efficiently, write to a file on a remote host using /usr/sbin/rmt as the rmt program, don't descend into a separate filesystem (since I don't want /proc and /sys etc to be backed up), and back up my root partition. Since I don't believe there's any value to separate file systems on a laptop, this will back up the entire contents of my n-1 laptop to the carillon.tgz in my home directory on player.local.

7 August 2015

Norbert Preining: Japanese-English dictionary for Kobo

Since ever the Kobo firmwares also allowed downloading of a bunch of dictionaries, most of which I don t need. As I am fluent in most languages I read and write, the only real dictionary I would like to see is a Japanese-English (I don t dare asking for a Japanese-German). Unfortunately, Kobo never shipped one. OTOH, starting with firmware 3.16.10 they ships two different English-Japanese dictionaries, one excellent Japanese-Japanese dictionary, but not one Japanese-English. So I took the liberty to write a script that allows everyone to enrich the shipped Japanese-Japanese dictionary with English definitions. Kobo Logo Till now I have used Tshering s excellent Japanese dictionary which was enriched with English definitions. It was based on the older Japanese-Japanese dictionary shipped with firmwares before 3.16.10. I didn t want to loose the more complete dictionary from the new firmware, so here is a script that updates the dictionary for you Prerequisites The script is neither fool-proof nor does everything by itself. Furthermore, it requires a set of programs. In details: Supported dictionaries At the moment the program can use the following two dictionaries as sources: Edict2 and Japanese3. Edict2 The Edict dictionary is a free dictionary which is the base of most other dictionaries. Created by the JMdict/EDICT Project it provides a very complete Japanese-English dictionary. To use the dict with the current program, one need to download the edict2.gz file and unpack it with gunzip edict2.gz. If you put it into the same directory as the Perl script, nothing else needs to be done, otherwise one can use the command line option --edict PATH-TO-EDICT to specify the location of the edict2 file. Japanese3 The Japanese3 is a dictionary application for iOS which provides another very complete Japanese-English dictionary. My feeling is that it is 90% and more based on Edict, so adding it will not buy you much, but still a bit (see below for how much!). If you have purchased this dictionary/application, and you manage to get access to your iOS device (via jailbreaking or some other tools), then you need to get the file Japanese3.db from the application folder, and then generate a file via sqlite3 as follows: 
$ sqlite3 Japanese3.db
.output japanese3-data
select Entry, Furigana, Summary from entries;
If you save the generated file japanese3-data into the same directory as the Perl script, nothing else needs to be done, otherwise one can use the command line option --japanese3 PATH-TO-J3-DATA to specify the location of the Japanese3 data file. Command line options The program supports the following command line options: Note that in case you pass in the option --unpacked, the files should be properly named (encodings are a problem!). Furthermore, note that the unpacked directory contains lots of .gif and .html files that are actually gzip-compressed files, but lacking the .gz extension. If you want to unpack the dictionary, by advised that the file names in the Zip directory are already encoded in UTF-8, but normal programs (unzip, 7z) assume that they are encoded in some other encoding. Thus, if your are using UTF-8 locale, it is necessary to set LC_CTYPE to C to make sure that the encoding is used, as in 
LC_CTYPE=C 7z x ...
Typical run In the following example we use both dict2 and Japanese3 dictionaries, and prefer Edict2: 
$ perl enhance-dictionary.pl --dicts edict2 --dicts japanese3
Using the following dictionaries as source for translations: edict2 japanese3
loading edict2 ... done
loading Japanese3 data ... done
unpacking original dictionary ... done
loading dict files ... done
searching for words and updating ... done
total words 805521, matches: 130563 (edict: 123388, japanese3: 7175)
creating output html ... done
creating update dictionary in dicthtml-jaxxdjs-201508072201.zip ... done
$
Installing the dictionary After having created the enhanced dictionary, one can install the generated file as KOBO/.kobo/dicts/dicthtml-jaxxdjs.zip (where KOBO is the mount point of the eReader). The dictionary should be picked up automatically. And the next lookup should give you something like the following:
kobo-glohd-enhanced-ja-dict There is only one caveat: Syncing with Kobo will re-download the original dictionary and overwrite the enhanced one. There is at least one solution here, which I am employing, see this post on MobileRead. Download and development place I am using the github repository kobo-ja-dict-enhance to develop the program. Please report bugs, and feature requests there. The file for the last released version can be downloaded from here: enhance-dictionary.pl Future plans I am planning to get rid of the 7z dependency and use Archive::Zip for all the unpacking and packing. This would also allow to do everything in memory and thus make it faster. There is a branch on github with ongoing work on this matter. I also think about including translations for Hiragana words by providing the translations of all possible Kanji readings. Conclusions Even if Kobo does not provide us with a decent Japanese-English dictionary, adding at least a huge amount of translations to the current dictionary is now easily possible. For serious students of Japanese who are reading or starting to read Japanese eBooks, this will be of great help. Enjoy, and don t forget to give feedback, suggestions, and improvements either here or better via the Github page.

Christoph Egger: Systemd pitfalls

logind hangs If you just updated systemd and ssh to that host seems to hang, that's just a known Bug (Debian Bug). Don't panic. Wait for the logind timeout and restart logind. restart and stop;start One thing that confused me several times and still confuses people is systemctl restart doing more than systemctl stop ; systemctl start. You will notice the difference once you have a failed service. A restart will try to start the service again. Both stop and start however will just ignore it. Rumors have it this has changed post jessie however. sysvinit-wrapped services and stop While there are certainly bugs with sysvinit services in general (I found myself several times without a local resolver as unbound failed to be started, haven't managed to debug further), the stop behavior of wrapped services is just broken. systemctl stop will block until the sysv initscript finished. It will even note the result of the action in its state. However systemctl will return with exitcode 0 and not output anything on stdout/stderr. This has been reported as Debian Bug. zsh helper I found the following zshrc snipped quite helpful in dealing with non-reported systemctl failures. On root shells it will display a list of failed services as part of the prompt. This will give proper feedback whether your systemctl stop failed, it will give feedback if you still have type=simple services and if the sysv-init script or wrapper is broken. 
precmd ()  
    if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
    then
      use_systemd=true
      systemd_failed=" systemctl --state=failed   grep failed   cut -d \  -f 2   tr '\n' ' ' "
    fi
 
if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
then
  PROMPT=$'% $fg[red]>>  $systemd_failed$reset_color% \n'
else
  PROMPT=""
fi
PROMPT+=whateveryourpromptis
zsh completion Speaking of zsh, there's one problem that bothers me a lot and I don't have any solution for. Tab-completing the service name for service is blazing fast. Tab-completing the service name for systemctl restart takes ages. People traced down to truckloads of dbus communication for the completion but no further fix is known (to me). type=simple services As described in length by Lucas Nussbaum type=simple services are actively harmful. Proper type=forking daemons are strictly superior (they provide feedback of finished initialization and success thereof) and type=notify services are so simple there's no excuse for not using them even for private one-off hacks. Even if you're language doesn't provide libsystemd-daemon bindings: 
(defun sd-notify (event-string)
  (let ((socket (make-instance 'sb-bsd-sockets:local-socket :type :datagram))
        (name (posix-getenv "NOTIFY_SOCKET"))
        (bufferlen (length event-string)))
    (when name
      (sb-bsd-sockets:socket-connect socket name)
      (sb-bsd-sockets:socket-send socket event-string bufferlen))))
This is a stable API guaranteed to not break in the future and implemented in less than ten lines of code with just basic socket functions. And if your language has support it becomes actually trivial: 
    try:
        import systemd
        systemd.daemon.notify("READY=1")
    except ImportError:
        pass
Note that in both cases there is no drawback at all on systemd-free setups. It has the overhead of checking the process' environment for NOTIFY_SOCKET or for the systemd package and behaves like a simple service otherwise. Actually the idea of separating the technical aspect (daemonizing) from the semantic aspect of signalizing "initialization finished, everything's fine" is a pretty good idea and hopefully has the potential to reduce the number of services signalizing the "everything's fine" too early. It could even be ported to non-systemd init systems easily given the API.

Christoph Egger: Systemd pitfalls

logind hangs If you just updated systemd and ssh to that host seems to hang, that's just a known Bug (Debian Bug #770135). Don't panic. Wait for the logind timeout and restart logind. restart and stop;start One thing that confused me several times and still confuses people is systemctl restart doing more than systemctl stop ; systemctl start. You will notice the difference once you have a failed service. A restart will try to start the service again. Both stop and start however will just ignore it. Rumors have it this has changed post jessie however. sysvinit-wrapped services and stop While there are certainly bugs with sysvinit services in general (I found myself several times without a local resolver as unbound failed to be started, haven't managed to debug further), the stop behavior of wrapped services is just broken. systemctl stop will block until the sysv initscript finished. It will even note the result of the action in its state. However systemctl will return with exitcode 0 and not output anything on stdout/stderr. This has been reported as Debian Bug #792045. zsh helper I found the following zshrc snipped quite helpful in dealing with non-reported systemctl failures. On root shells it will display a list of failed services as part of the prompt. This will give proper feedback whether your systemctl stop failed, it will give feedback if you still have type=simple services and if the sysv-init script or wrapper is broken. 
precmd ()  
    if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
    then
      use_systemd=true
      systemd_failed=" systemctl --state=failed   grep failed   cut -d \  -f 2   tr '\n' ' ' "
    fi
 
if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
then
  PROMPT=$'% $fg[red]>>  $systemd_failed$reset_color% \n'
else
  PROMPT=""
fi
PROMPT+=whateveryourpromptis
zsh completion Speaking of zsh, there's one problem that bothers me a lot and I don't have any solution for. Tab-completing the service name for service is blazing fast. Tab-completing the service name for systemctl restart takes ages. People traced down to truckloads of dbus communication for the completion but no further fix is known (to me). type=simple services As described in length by Lucas Nussbaum type=simple services are actively harmful. Proper type=forking daemons are strictly superior (they provide feedback of finished initialization and success thereof) and type=notify services are so simple there's no excuse for not using them even for private one-off hacks. Even if you're language doesn't provide libsystemd-daemon bindings: 
(defun sd-notify (event-string)
  (let ((socket (make-instance 'sb-bsd-sockets:local-socket :type :datagram))
        (name (posix-getenv "NOTIFY_SOCKET"))
        (bufferlen (length event-string)))
    (when name
      (sb-bsd-sockets:socket-connect socket name)
      (sb-bsd-sockets:socket-send socket event-string bufferlen))))
This is a stable API guaranteed to not break in the future and implemented in less than ten lines of code with just basic socket functions. And if your language has support it becomes actually trivial: 
    try:
        import systemd
        systemd.daemon.notify("READY=1")
    except ImportError:
        pass
Note that in both cases there is no drawback at all on systemd-free setups. It has the overhead of checking the process' environment for NOTIFY_SOCKET or for the systemd package and behaves like a simple service otherwise. Actually the idea of separating the technical aspect (daemonizing) from the semantic aspect of signalizing "initialization finished, everything's fine" is a pretty good idea and hopefully has the potential to reduce the number of services signalizing the "everything's fine" too early. It could even be ported to non-systemd init systems easily given the API.

Ben Hutchings: Debian LTS work, July 2015

This was my eighth month working on Debian LTS. I was assigned 14.75 hours of work by Freexian's Debian LTS initiative. linux-2.6 I didn't upload any new version of the kernel this month, but I did send all the recent security fixes to Willy Tarreau who maintains the 2.6.32.y branch at kernel.org. I also spent more time working on a fix for bug #770492 aka CVE-2015-1350, which is not yet fixed upstream. I now have a candidate patch for 2.6.32.y/squeeze, and automated tests covering many of the affected filesystems. Front desk The LTS 'front desk' role is now assigned on a rota, and I was my first turn in the third week of July. I investigated which new CVEs affected LTS-supported packages in squeeze, recorded this in the secure-testing repository, and mailed the package maintainers to give them a chance to handle the updates. groovy Groovy had a single issue (CVE-2015-3253) with a simple fix that I could apply to the version in squeeze. Unfortunately the previous version in squeeze had not been properly updated during the squeeze release cycle and could no longer be built from source. I eventually worked out what the build-dependencies should be, uploaded the fix and issued DLA-274-1. ruby1.9.1 Ruby 1.9.1 also had a single issue (CVE-2014-6438), though the fixes were more complicated and hard to find. (The original bug report for this is still not public in the upstream bug tracker.) I also had to find an earlier upstream change that they depended on. As I've mentioned before, Ruby has an extensive test suite so I could be quite confident in my backported changes. I uploaded and issued DLA-275-1. libidn The GNU library for Internationalized Domain Names, libidn, required applications to pass only valid UTF-8 strings as domain names. The Jabber instant messaging server turned out not to be validating untrusted domain names, leading to a security issue there (CVE-2015-2059). As there are likely to be other applications with similar bugs, this was resolved by adding UTF-8 validation to libidn. The fix for this involved importing additional functions from the GNU portability library, gnulib, and there my difficulties began. Confusingly, libidn has two separate sets of functions imported from gnulib, and due to interdependencies it turned out that I would have to update both of these wholesale rather than just importing the new functions that were wanted. This resulted in a 35,000 line patch. Following that I needed to autoreconf the package (and debug that process when it failed), ending up with another 26,000 line patch. Finally, it turned out that the new gnulib code needed gperf to build a header file for Solaris (when building for Linux? huh?). I ended up adding that with another patch instead. libidn has a decent test suite, so I could at least be confident in the result of my changes. I uploaded and issued DLA-277-1. Dear upstream developers, please use sane libraries instead of gnulib.

John Goerzen: Detailed Smart Card Cryptographic Token Security Guide

After my first post about smartcards under Linux, I thought I would share some information I ve been gathering. This post is already huge, so I am not going to dive into much specific commands, but I am linking to many sources with detailed instructions. I ve reviewed several types of cards. For this review, I will focus on the OpenPGP card and the Yubikey NEO, since the Cardomatic Smartcard-HSM is not supported by the gpg version in Jessie. Both cards are produced by people with strong support for the Free Software ecosystem and have strong cross-platform support with source code. OpenPGP card: Basics with GnuPG The OpenPGP card is well-known as one of the first smart cards to work well on Linux. It is a single-application card focused on use with GPG. Generally speaking, by the way, you want GPG2 for use with smartcards. Basically, this card contains three slots: decryption, signing, and authentication slots. The concept is that the private key portions of the keys used for these items are stored only on the card, can never be extracted from the card, and the cryptographic operations are performed on the card. There is more information in my original post. In a fairly rare move for smartcards, this card supports 4096-byte RSA keys; most are restricted to 2048-byte keys. The FSF Europe hands these out to people and has a lot of good information about them online, including some HOWTOs. The official GnuPG smart card howto is 10 years old, and although it has some good background, I d suggest using the FSFE instructions instead. As you ll see in a bit, most of this information also pertains to the OpenPGP mode of the Yubikey Neo. OpenPGP card: Other uses Of course, this is already pretty great to enhance your GPG security, but there s a lot more that you can do with this card to add two-factor authentication (2FA) to a lot of other areas. Here are some pointers: OpenPGP card: remote authentication with ssh You can store the private part of your ssh key on the card. Traditionally, this was only done by using the ssh agent emulation mode of gnupg-agent. This is still possible, of course. Now, however, the OpenSC project now supports the OpenPGP card as a PKCS#11 and PKCS#15 card, which means it works natively with ssh-agent as well. Try just ssh-add -s /usr/lib/x86_64-linux-gnu/pkcs11/opensc-pkcs11.so if you ve put a key in the auth slot with GPG. ssh-add -L will list its fingerprint for insertion into authorized_keys. Very simple! As an aside: Comments that you need scute for PKCS#11 support are now outdated. I do not recommend scute. It is quite buggy. OpenPGP card: local authentication with PAM You can authenticate logins to a local machine by using the card with libpam-poldi here are some instructions. Between the use with ssh and the use with PAM, we have now covered 2FA for both local and remote use in Unix environments. OpenPGP card: use on Windows Let s move on to Windows environments. The standard suggestion here seems to be the mysmartlogon OpenPGP mini-driver. It works with some sort of Windows CA system, or the local accounts using EIDAuthenticate. I have not yet tried this. OpenPGP card: Use with X.509 or Windows Active Directory You can use the card in X.509 mode via these gpgsm instructions, which apparently also work with Windows Active Directory in some fashion. You can also use it with web browsers to present a certificate from a client for client authentication. For example, here are OpenSC instructions for Firefox. OpenPGP card: Use with OpenVPN Via the PKCS#11 mode, this card should be usable to authenticate a client to OpenVPN. See the official OpenVPN HOWTO or these other instructions for more. OpenPGP card: a note on PKCS#11 and PKCS#15 support You ll want to install the opensc-pkcs11 package, and then give the path /usr/lib/x86_64-linux-gnu/pkcs11/opensc-pkcs11.so whenever something needs the PKCS#11 library. There seem to be some locking/contention issues between GPG2 and OpenSC, however. Usually killing pcscd and scdaemon will resolve this. I would recommend doing manipulation operations (setting PINs, generating or uploading keys, etc.) via GPG2 only. Use the PKCS#11 tools only to access. OpenPGP card: further reading Kernel Concepts also has some nice readers; you can get this card in a small USB form-factor by getting the mini-card and the Gemalto reader. Yubikey Neo Introduction The Yubikey Neo is a fascinating device. It is a small USB and NFC device, a little smaller than your average USB drive. It is a multi-application device that actually has six distinct modes: There is a ton to digest with this device. Yubikey Neo Basics By default, the Yubikey Neo is locked to only a subset of its features. Using the yubikey-personalization tool (you ll need the version in stretch; jessie is too old), you can use ykpersonalize -m86 to unlock the full possibilities of the card. Run that command, then unplug and replug the device. It will present itself as a USB keyboard as well as a PC/SC-compatible card reader. It has a capacitive button, which is used to have it generate keystrokes to input validation information for HOTP or HMAC validation. It has two slots that can be configured with HMAC and HOTP; a short button press selects the default slot #1 and a long press selects slot #2. But before we get into that, let s step back at some basics. opensc-tool list-algorithms claims this card supports RSA with 1024, 2048, and 3072 sizes, and EC with 256 and 384-bit sizes. I haven t personally verified anything other than RSA-2048 though. Yubikey Neo: OpenPGP support In this mode, the card is mostly compatible with the physical OpenPGP card. I say mostly because there are a few protocol differences I ll get into later. It is also limited to 2048-byte keys. Support for this is built into GnuPG and the GnuPG features described above all work fine. In this mode, it uses firmware from the Yubico fork of the JavaCard OpenPGP Card applet. There are Yubico-specific tutorials available, but again, most of the general GPG stuff applies. You can use gnupg-agent to use the card with SSH as before. However, due to some incompatibilities, the OpenPGP applet on this card cannot be used as a PKCS#11 card with either scute or OpenSC. That is not exactly a huge problem, however, as the card has another applet (PIV) that is compatible with OpenSC and so this still provides an avenue for SSH, OpenVPN, Mozilla, etc. It should be noted that the OpenPGP applet on this card can also be used with NFC on Android with the OpenKeychain app. Together with pass (or its Windows, Mac, or phone ports), this makes a nicely secure system for storing passwords. Yubikey Neo: PKCS#11 with the PIV applet There is also support for the PIV standard on the Yubikey Neo. This is supported by default on Linux (via OpenSC) and Windows and provides a PKCS#11-compabible store. It should, therefore, be compatible with ssh-agent, OpenVPN, Active Directory, and all the other OpenPGP card features described above. The only difference is that it uses storage separate from the OpenPGP applet. You will need one of the Yubico PIV tools to configure the key for it; in Debian, the yubico-piv-tool from stretch does this. Here are some instructions on using the Yubikey Neo in PIV mode: A final note: for security, it s important to change the management key and PINs before deploying the PIV mode. I couldn t get this to work with Firefox, but it worked pretty much everywhere else. Yubikey Neo: HOTP authentication This is the default mode for your Yubikey; all other modes require enabling with ykpersonalize. In this mode, a 128-bit AES key stored on the Yubikey is used to generate one-time passwords (OTP). (This key was shared in advance with the authentication server.) A typical pattern would be for three prompts: username, password, and Yubikey HOTP. The user clicks in the Yubikey HOTP field, touches the Yubikey, and their one-time token is pasted in. In the background, the service being authenticated to contacts an authentication server. This authentication server can be either your own (there are several open source implementations in Debian) or the free Yubicloud. Either way, the server decrypts the encrypted part of the OTP, performs validity checks (making sure that the counter is larger than any counter it s seen before, etc) and returns success or failure back to the service demanding authentication. The first few characters of the posted auth contain the unencrypted key ID, and thus it can also be used to provide username if desired. Yubico has provided quite a few integrations and libraries for this mode. A few highlights: You can also find some details on the OTP mode. Here s another writeup. This mode is simple to implement, but it has a few downsides. One is that it is specific to the Yubico line of products, and thus has a vendor lock-in factor. Another is the dependence on the authentication server; this creates a potential single point of failure and can be undesireable in some circumtances. Yubikey Neo: OATH and HOTP and TOTP First, a quick note: OATH and OAuth are not the same. OATH is an authentication protocol, and OAuth is an authorization protocol. Now then Like Yubikey HOTP, OATH (both HOTP and TOTP) modes rely on a pre-shared key. (See details in the Yubico article.) Let s talk about TOTP first. With TOTP, there is a pre-shared secret with each service. Each time you authenticate to that service, your TOTP generator combines the timestamp with the shared secret using a HMAC algorithm and produces a OTP that changes every 30 seconds. Google Authenticator is a common example of this protocol, and this is a drop-in replacement for it. Gandi has a nice description of it that includes links to software-only solutions on various platforms as well. With the Yubikey, the shared secrets are stored on the card and processed within it. You cannot extract the shared secret from the Yubikey. Of course, if someone obtains physical access to your Yubikey they could use the shared secret stored on it, but there is no way they can steal the shared secret via software, even by compromising your PC or phone. Since the Yubikey does not have a built-in clock, TOTP operations cannot be completed solely on the card. You can use a PC-based app or the Android application (Play store link) with NFC to store secrets on the device and generate your TOTP codes. Command-line users can also use the yubikey-totp tool in the python-yubico package. OATH can also use HOTP. With HOTP, an authentication counter is used instead of a clock. This means that HOTP passwords can be generated entirely within the Yubikey. You can use ykpersonalize to configure either slot 1 or 2 for this mode, but one downside is that it can really only be used with one service per slot. OATH support is all over the place; for instance, there s libpam-oath from the OATH toolkit for Linux platforms. (Some more instructions on this exist.) Note: There is another tool from Yubico (not in Debian) that can apparently store multiple TOTP and HOTP codes in the Yubikey, although ykpersonalize and other documentation cannot. It is therefore unclear to me if multiple HOTP codes are supported, and how.. Yubikey Neo: Challenge-Response Mode This can be useful for doing offline authentication, and is similar to OATH-HOTP in a sense. There is a shared secret to start with, and the service trying to authenticate sends a challenge to the token, which must supply an appropriate response. This makes it only suitable for local authentication, but means it can be done fairly automatically and optionally does not even require a button press. To muddy the waters a bit, it supports both Yubikey OTP and HMAC-SHA1 challenge-response modes. I do not really know the difference. However, it is worth noting that libpam-yubico works with HMAC-SHA1 mode. This makes it suitable, for instance, for logon passwords. Yubikey Neo: U2F U2F is a new protocol for web-based apps. Yubico has some information, but since it is only supported in Chrome, it is not of interest to me right now. Yubikey Neo: Further resources Yubico has a lot of documentation, and in particular a technical manual that is actually fairly detailed. Closing comments Do not think a hardware security token is a panacea. It is best used as part of a multi-factor authentication system; you don t want a lost token itself to lead to a breach, just as you don t want a compromised password due to a keylogger to lead to a breach. These things won t prevent someone that has compromised your PC from abusing your existing ssh session (or even from establishing new ssh sessions from your PC, once you ve unlocked the token with the passphrase). What it will do is prevent them from stealing your ssh private key and using it on a different PC. It won t prevent someone from obtaining a copy of things you decrypt on a PC using the Yubikey, but it will prevent them from decrypting other things that used that private key. Hopefully that makes sense. One also has to consider the security of the hardware. On that point, I am pretty well satisfied with the Yubikey; large parts of it are open source, and they have put a lot of effort into hardening the hardware. It seems pretty much impervious to non-government actors, which is about the best guarantee a person can get about anything these days. I hope this guide has been helpful.

6 August 2015

Eduard Sanou: Reproducible builds on Debian for GSoC 2015, 1st update

This is the second blog post I m writing about my experiences contributing to Debian for Google Summer of Code 2015 (check my first post)

Status update

First month It s been two months and a few days since the GSoC started. During the first month I worked on fixing specific packages, mainly concerning issues with timestamps, which is a very common source of unreproducibility. In many cases, during the build process files are compressed into gzip or zip archives, which store the creation time of files in the metadata. This can lead to unreproducible results when there is timezone variation between builds (easily fixed setting the timezone to UTC before the compression happens). In some cases the compressed files are generated during the build, and thus add build times in the metadata of compressed files (in this case the creation date of the files needs to be normalized somehow). As explained in my application, I finished exams on the end of June, that s why I chose to work on small fixes first, so that I could make the most out of my time between studying and finishing university projects and reports. I m happy with my first month, as I have worked as originally planned. Actually, my estimation of the number of bugs I could submit every week was surpassed in reality!

Second month Once the university was over, I started dedicating myself fully to the project. This allowed me to start working on toolchain fixes, following my original plan on working with timestamp related issues. In particular I have been working a lot in implementing a proposal for deterministic timestamps that appeared in the reproducible builds project. The idea is to define an environment variable called SOURCE_DATE_EPOCH that contains a known timestamp in Unix epoch format. With this variable exported, tools that would embed the localtime in their generated or processed files, can use this externally supplied date. This would happen only if SOURCE_DATE_EPOCH is exported, so the behaviour of the tool wouldn t change if the variable is not set. The first package I patched to implement this behaviour was gcc. The reason behind this is that there are about 420 unreproducible packages due to using the __DATE__, __TIME__ and __TIMESTAMP__ C macros. My patch changes the behavior of the macros __DATE__ and __TIME__ if SOURCE_DATE_EPOCH is exported. I submitted this patch to the gcc-patches list. Even though there was some interesting discussions in the list, the patch has not been accepted yet. Seeing how the reproducible builds idea is gaining momentum and becoming widespread, I m positive that at some point the gcc people will be more receptive for such patch. The second work with SOURCE_DATE_EPOCH was in debhelper; I patched this building tool to export the variable with the latest debian/changelog entry timestamp. With this patch, all the tools that run under dh will be able to use it to embed deterministic timestamps. Unfortunately some parts of the build process of some packages don t happen under debhelper, so the variable needs to be exported in a different way. Having submitted the debhelper patch allowed many packages to become reproducible after the tools that embedded timestamps were patched to honour SOURCE_DATE_EPOCH. As of today, the toolchain packages I have patched to do that are: gcc, libxslt, gettext, ghostscript and qt4-x11 (qhelpgenerator). I have also continued working on fixing individual packages affected by timestamps, random orderings (such as the ones from listing hash keys) and locale depending orderings; I have tagged packages in our infrastructure to note what kind of issue makes them unreproducible; I have updated some parts of the Reproducible Builds Wiki.

Impressions about reproducible builds The work I did during the first month felt a bit tedious sometimes: it didn t require much creativity or thinking as most of the fixes where quite mechanical, following a recipe. After I became free from university duties, I started looking into less trivial issues, which require more deep investigation and feel more rewarding once they are fixed. I also worked on toolchain fixes, which need more work and need more care. Fixing toolchain packages feels particularly rewarding because they can cause many packages to become reproducible at once. There is a very active community in the reproducible builds project! It s great to see so many people contributing to this project spending so many hours and putting so much effort. I ve felt very welcome from the beginning and I have gotten kind replies and helpful answers to all the questions and doubts I ve had, both from my mentor and from the other people in the project. I want to add that I m still amazed by the awesome infrastructure set up for the reproducible builds project. The team is using a Jenkins machine to continuously build packages to check for reproducibility, with irc notifications for changes, and also with a really useful web interface to list all the packages organized by issues that allows exploring them individually with all the available information. Also not only the infrastructure is used to build Debian amd64 packages, but also FreeBSD, NetBSD, OpenWRT, coreboot and lately Debian armhf with the help of a few new arm nodes.

Impressions about working on a free software project This was my first time working on a community driven free software project and I ve learned so many things. Something I learned during the first days, which is even more present in such wide project like the reproducible builds, is that contributing is not just writing patches for the features you want; you also need to convince the maintainer of the package to accept the patch! I was a bit surprised at the beginning because even if this is a Debian project, that aims to make changes to the whole distribution, decisions are not absolute for the whole Debian project. After taking decisions within the reproducible builds teams on how to approach things, we need to convince the rest of the Debian developers (mainly maintainers) to follow them, as they are allowed to disagree. So it is required to work together for solutions that makes everyone comfortable, usually with discussions on mailing lists or irc. There is another fact that I wasn t expecting before getting involved in this project. The kind of teamwork I have done previously involves having a leader who decides how stuff is done, who takes decisions when needed and oversees the whole project. There seems to be a different philosophy in Debian. Instead of having a leader, everyone tries their best, trying to convince the others that their solution is good, often by showing an implementation of the solution and providing proof that it works, rather than trying to get the solution accepted before coding it. Also, solutions and ideas are valued for their quality rather than by the position of the person submitting them, and there is no hierarchy within the group: all comments and advices are taken equally, valuing their usefulness regardless of who gives them. Usually there are no votes when deciding things. Members try their best on their approaches, trying to convince others as best as they can. And even if someone disagrees they may end up accepting the solution if they don t manage to convince the original proposer of doing things differently. The idea is to spend more time working and coding than arguing and deciding on the way to do things. So far I ve seen this approach to be very efficient :) I ve been told by my mentor that for difficult cases there exist a Debian committee that helps mediate on disagreements, but that is only used as a last option, probably when the discussion gets heated up.

Personal experience Overall I m very happy to finally having set my foot in the free software community, where I m able to contribute to the kind of software I have been using for years. The sense of community in Debian is really big, and everyone is invited to participate. I think that Google is doing an awesome job with the Google Summer of Code, not only because it gives a lot to free software but because it helps students to join the free software world as contributors (which is something that may be difficult to get into when you don t know how to begin, as it happened to myself for some time). I plan to continue contributing to the free software world, and I d encourage anyone to find projects to get involved and contribute as well! Happy hacking to everyone!

Norbert Preining: Upgrading my eReader Kobo GloHD

After long time of dear service, I have decided to exchange my Kobo Glo to a new device, the Kob GloHD. The reasons are simple: higher resolution, better reading quality.
kobo-glohd-1 While the photos are not showing these details, the quality of the screen is definitely much better I have now read a few hours with the new device, both Japanese and English books, and the crispness of the letters has definitely improved. Very nice. Together with the new device came a new firmware, only released in Japan, 3.16.10. Of course it can be found on MobileRead. As with previous version, I will prepare a MegaPack containing nickel patches, ssh, custom dicts, etc etc, please wait a bit longer until KSM compatibility with the new firmware is perfect again. Talking of dictionaries: The new version brings a few more dicts, especially there are now two English-Japanese dictionaries, and a much more complete Japanese dictionary.
kobo-glohd-dictlist Unfortunately, as before, there is no Japanese-English dictionary. Till now I have used Tshering s excellent Japanese dictionary which was enriched with English definitions. But the new version brought a new dictionary that is much more complete:
kobo-glohd-dicsearch I have already a script ready that enriches this new big dictionary automatically with definitions from the Edict project, so everyone can now update his own dictionary easily. I will write a blog entry about this soon. That s it for the moment, back to reading. I love the new device, and will surely post many articles about it in the future!

5 August 2015

DebConf team: DebConf15 registrations temporarily closed due to immense popularity (Posted by Martin Krafft)

The interest to participate in DebConf15, which will start in 10 days, has been overwhelming. Thank you so much! We planned the conference for 350 people and we now expect more than 570. This is fantastic and a wonderful support and encouragement for Debian and its contributors. But we are getting to the point where we have to prioritise considerations about the capacity of the venue and the safety of our attendees. We ve therefore had to close registrations until we have evaluated further options for accomodating even more attendees. For those of you who did not register for DebConf on time, but still want to participate, please bear with us and make sure to check back soon.

Antonio Terceiro: Elixir in Debian, MiniDebconf at FISL, and Debian CI updates

In June I started keeping track of my Debian activities, and this is my July update. Elixir in Debian Elixir is a functional language built on top of the Erlang virtual machine. If features imutable data structures, interesting concurrency primitives, and everything else that Erlang does, but with a syntax inspired by Ruby what makes it much more aproachable in my opinion. Those interested in Elixir for Debian are encouraged to hang around in #debian-elixir on the OFTC IRC servers. There are still a lot of things to figure out, for example how packaging Elixir libraries and applications is going to work. MiniDebconf at FISL, and beyond I helped organize a MiniDebconf at this year s FISL, in Porto Alegre on the 10th of July. The whole program was targetted at getting more people to participate in Debian, so there were talks about translation, packaging, and a few other more specific topics. I myself gave two talks: one about Debian basics, What is Debian, and how it works , and second one on packaging the free software web , which I will also give at Debconf15 later this month. The recordings are available (all talks in Portuguese) at the Debian video archive thanks to Holger Levsen. We are also organizing a new MiniDebconf in October as part of the Latinoware schedule. Ruby We are in the middle of a transition to switch to Ruby 2.2 as default in Debian unstable, and we are almost there. The Ruby transition is now on hold while GCC 5 one is going on, but will be picked up as soon as were are done with GCC 5. ruby-defaults has been uploaded to experimental for those that want to try having Ruby 2.2 as default before that change hits unstable. I myself have been using Ruby 2.2 as default for several weeks without any problem so far, including using vagrant on a daily basis and doing all my development on sid with it. I started taking notes about Ruby interpreter transitions work to make sure that knowledge is registered. I have uploaded minor security updates of both ruby2.1 and ruby2.2 to unstable. They both reached testing earlier today. I have also fixed another bug in redmine, which I hope to get into stable as well as soon as possible. gem2deb has seen several improvements through versions 0.19, 0.20, 0.20.1 and 0.20.2. I have updated a few packages: Two NEW packages, ruby-rack-contrib and ruby-grape-logging ,were ACCEPTED into the Debian archive. Kudos to the ftp-master team who are doing an awesome job reviewing new packages really fast. Debian Continuous Integration This month I have made good progress with the changes needed to make debci work as a distributed system with one master/scheduler node and as many worker nodes (running tests) as possible. While doing my tests, I have submitted a patch to lxc and updated autodep8 in unstable. At some point I plan to upload both autodep8 and autopkgtest to jessie-backports. Sponsoring I have sponsored a few packages:

4 August 2015

Matthew Garrett: Reverse this

The TODO group is an industry body that appears to be trying to define community best practices or something. I don't really know what their backstory is and whether they're trying to do meaningful work or just provide a fig leaf of respectability to organisations that dislike being criticised for doing nothing to improve the state of online communities but don't want to have to actually do anything, and their initial work on codes of conduct was, perhaps, suboptimal. But they do appear to be trying to improve things - this commit added a set of inappropriate behaviours, and also clarified that reverseisms were not actionable behaviour.

At which point Reddit lost its shit, because Reddit is garbage. And now the repository is a mess of white men attempting to explain how any policy that could allow them to be criticised is the real racism.

Fuck that shit.

Being a cis white man who's a native English speaker from a fairly well-off background, I'm pretty familiar with privilege. Spending my teenage years as an atheist of Irish Catholic upbringing in a Protestant school in a region of Northern Ireland that made parts of the bible belt look socially progressive, I'm also pretty familiar with the idea that that said privilege doesn't shield me from everything bad in life. Having privilege isn't a guarantee that my life will be better, in the same way that avoiding smoking doesn't mean I won't die of lung cancer. But there's an association in both cases, one that's strong enough to alter the statistical likelihood in meaningful ways.

And that inherently affects discussions about race or gender or sexuality. The probability that I've been subject to systematic discrimination because of these traits is vanishingly small. In the communities this policy is intended to cover, I'm the default. It's very difficult for any minority to exercise power over me. "You're white, you wouldn't understand" isn't fundamentally about my colour, it's about the fact that my colour means I haven't been subject to society trying to make my life more difficult at every opportunity. A community that considers saying that to be racist is a community that will never change the default, a community that will never be able to empower people who didn't grow up with that privilege. A code of conduct that makes it clear that "reverse racism" isn't grounds for complaint makes it clear that certain conversations are legitimate and helps ensure we have the framework we need to gradually change that default, and as such is better than one that doesn't.

(comments disabled because I don't trust any of you)

comment count unavailable comments

3 August 2015

Sven Hoexter: Failing with F5: CMP - Clustered Multiprocessing

The last Whiteboard Wednesday tackled a few of the gotchas of the F5 CMP implementation. I take that as an opportunity to finish a blog post I started to write six month ago on this topic. I've run into two scenarios where CMP got in my way: All of this happened on TMOS 11.4.1+HF run on BigIP 2K systems. Since F5 is sometimes moving very quickly you might not experience those issues on other hardware or with a different software release. applying low and strict connection limits Apparently connection limits are counted on a per tmm level and are not shared between the tmm processes running on different CPUs. So if you've two CPU cores with HT you'll see four tmm processes that will all enforce the connection limit. If you have a max connection limit set to two you might still see a maximum of eight connections. That is a very bad thing if you abuse the F5 to limit your outbound connections because your counter part is not able to enforce connection limits, but forces you via a contract to not open more then two connections. Disable CMP and it will all end on the same tmm process and suddenly your connection limits work as expected. If your connection limit is big enough you can cheat, say it's twelve connections, you can configure a limit of three. With four processes that makes a max of twelve connections as required. Though due to the hashing and distribution among the tmm processes it could happen that you reject connections on one process while others are idle. Another option would be to enter the dirty land of iRules and tables to maintain your own connection count table. There are examples given in the F5 devcentral community, but the best resolution you can get, without starting to scan the whole table on every request, is X requests per minute. active health checks Active monitoring checks were added to detect the operational state of our application servers. Now imagine you have some realtime processing with over a hundred requests per minute. So in case you've issues with one instance you'd like to remove it from the pool rather sooner then later, which requires frequent checking. So here is a simple active http check: 
defaults-from http-appmon
destination *:*
interval 9
recv "^HTTP/1\\.1 200 OK"
send "GET /mon/state HTTP/1.1\\r\\nHost: MON\\r\\nConnection: Close\\r\\n\\r\\n"
time-until-up 0
timeout 2
up-interval 1
(Configuration options are described here) With an up-interval of 1 second and a timeout of 2 seconds someone naive like me expected that a fatal issue like a crashed application would be detected within three seconds and no more requests should hit the application after second 3. That turned out to be slightly longer, at least 4s for this case, often longer. Also for a regular maintenance we had to instruct our scripts to flip the application monitor to unavailable and wait for interval X tmm process count + timeout + 1s + max allowed request duration. That is not an instant off. If you use tcpdump (on the F5 where it's patched to show you the tmm that received the request) to look at the health check traffic you'll notice that the tmm threads pass around the handle to do a health check. So my guess is the threads do only sync the health check state of the application when they're due to run the health check. Makes sense performance wise but if you operate with longer timeouts and bigger intervals you will definetly loose a notable amount of requests. Even in our low end case we already loose a notable amount of requests in 4s. While you could resent them you never know exactly if a request was already processed or not. So you have to ensure that somewhere else in your stack. One option is to disable CMP but I guess that's not an option for most people that are required to scale out to handle the load and not only for the sake of redundancy. If you're running the balancer only for the sake of redundancy the choice is easy. You find a hint on this kind of issue if you lookup the documentation for the inband monitor: Note: Systems with multiple tmm processes use a per-process number to calculate failures, depending on the specified load balancing method. That case is a bit different due to the nature of the monitor but points in the same direction. The actual check is executed by bigd according to this document, so I'm pretty sure I'm still missing some pieces of the picture. Just be warned that you've to really test your outage scenarios and you should double check it if you run a transaction based service that is sensitive to lost requests.

Next.

Previous.