Debconf is a great opportunity to meet people in real life, to express and share ideas in a different way, and to work on all sort of stuff.

I therefore spent some time to finish a couple of features in the editor for sources.debian.net. Here are some of the changes:

• Compare the source file with that of another version of the package
• And in order to present that: tabs! editor tabs!
• at the same time: generated diffs are now presented in a new editor tab, from where you can download it or email it

Get it for chromium, and iceweasel.

If your browser performs automatic updates of the extensions (the default), you should soon be upgraded to version 0.1.0 or later, bringing all those changes to your browser.

Want to see more? multi-file editing? in-browser storage of the editing session? that and more can be done, so feel free to join me and contribute to the Debian sources online editor!

Based on a discussion around breakfast, and encouraged by the people at the table, I hereby call for a new release goal (or challenge, whatever you prefer to call it):


Every package maintainer should remove one of their packages from the archive.


It's dead simple. It is acceptable to adopt a package to replace the one that has been removed, or to add a new one to the archive.
For tracking purposes please include "for RG" (release goal) in the removal request to ftp.debian.org.


And how about a debconf challenge? how about filing over 100 removal requests before the end of Debconf 15 on Saturday night? blog about it, dent/twit about it, spam IRC about it!


The idea came up after discussing about how us as package maintainers refuse to remove our obsolete or unused packages. So yes, that may also include the very first package that you got into the archive.


Sad news, good news.

I don't know most of my passwords

I don't know most of my email addresses


But I do know which site shared my email address with spammers

On confidentiality:


So that they don't know what's inside

Since the announcement of the chrome/chromium and firefox/iceweasel extensions that add in-browser editing of Debian sources there have been a few changes:

• Update to Ace 1.1.9, which brings quite some fixes
• Downloads with proper file names! this should work with recent browsers. E.g for a patch it will propose package-version_path_file.patch
• Changing the order of additions and removals in the generated patches (it was strange to see additions first)
• Improvements to the compatibility with debsources' URL parameters
• Allow the extension to be enabled in private browsing mode (firefox/iceweasel)
• Loading of the extension when browsing debsources via https
• Internal extension packaging cleanup, including the generation of a versioned copy of the web (remote) files, so that users of version 0.0.x do use the remote code version 0.0.x

If your browser performs automatic updates of the extensions (the default), you should soon be upgraded to version 0.0.10 or later, bringing all those changes to your browser.

Want to see more? multi-file editing? emacs and vim editing modes? in-browser storage of the modified files? that and more can be done, so feel free to join me and contribute to the Debian sources editor!

It appears that this year's MiniDebconf in Lyon, France is going to have a very special guest. After enjoying so much the Q&A held at Debconf14, Linus Torvalds has decided to make a quick detour from a trip to Europe to bring the opportunity for another session with the Debian community.

The event has yet to appear in the MiniDebconf agenda. Rumor has it that the hurds of fans that tend to attend his events pose a logistics and safety challenge, reason why the event might only appear in the agenda a couple of days before the event.
Lyon being such an easy place to get by train, flight, car, and even by boat, it is understandable and should be expected that a great number of people attend the minidebconf only for his very session.

If you have not yet registered, I'd recommend you to do so now and to book your travel and accommodation ASAP, before everything is overpriced due to high demand.

See you in ten days!

P.S. kudos to the organizers!

Soon after publishing the chromium/chrome extension that allows you to edit Debian online, Moez Bouhlel sent a pull request to the extension's git repository: all the changes needed to make a firefox extension!

After another session of browser extensions discovery, I merged the commits and generated the xpi. So now you can go download the Debian online editing firefox extension and hack the world, the Debian world.

Install it and start contributing to Debian from your browser. There's no excuse now.

How cool would it be to fix that one bug you just found without having to download a source package? and without leaving your browser?

Inspired by github's online code editing, during Debconf 14 I worked on integrating an online editor on debsources (the software behind sources.debian.net). Long story short: it is available today, for users of chromium (or anything supporting chrome extensions).

After installing the editor for sources.debian.net extension, go straight to sources.debian.net and enjoy!

Go from simple debsources:


To debsources on steroids:


All in all, it brings:

• Online editing of all of Debian
• In-browser patch generation, available for download
• Downloading the modified file
• Sending the patch to the BTS
• Syntax highlighting for over 120 file formats!
• More hidden gems from Ace editor that can be integrated thanks to patches from you

Clone it or fork it:

git clone https://github.com/rgeissert/ace-sourced.n.git

For example, head to apt's source code, find a typo and correct it online: open apt.cc, click on edit, make the changes, click on email patch. Yes! it can generate a mail template for sending the patch to the BTS: just add a nice message and your patch is ready to be sent.

Didn't find any typo to fix? how sad, head to codesearch and search Debian for a spelling mistake, click on any result, edit, correct, email! you will have contributed to Debian in less than 5 minutes without leaving your browser.

The editor was meant to be integrated into debsources itself, without the need of a browser extension. This is expected to be done when the requirements imposed by debsources maintainers are sorted out.

Kudos to Harlan Lieberman who helped debug some performance issues in the early implementations of the integration and for working on the packaging of the Ace editor.

Debian LTS - feedback about the feedback from my LTS talk at DebConf14 So, I'm more or less back from dc14 and today, five days later, I think I might have mostly overcome jetlag. Probably... So, at DebConf14 I gave a talk about LTS and while I'm sorry that I was that tired, I'm more or less happy how the talk went. Thankfully at least I was calm and relaxed... There are a couple of things I learned from the talk: a.) LTS has been really really perceived well b.) it fits a demand c.) people already take it for granted (eg plan for Wheezy LTS) d.) people expect the same non-intrusive changes as currently done for security updates. To explain the last point: when I explained the - so far - rather theoretical problem that ''squeeze-lts'' has no gatekeeper mechanisms whatsoever (eg no ''proposed-updates'', no NEW queue..) the reaction in the audience was basically "something like this should exist, else how can we deploy this in large scale / on important setup?!". Also currently there is no, well-documented, easily to be found policy for what kind of updates are acceptable. I said that we basically follow the same rules as there are for debian-security updates, but this should really be documented properly. This doesn't seem very hard to fix, just like many things it "just" needs someone to do the work. IOW: we explain how to use LTS, we explain how to contribute to LTS (through uploads or financially) but we lack a simple explaination what LTS is and what kind of updates to expect. It's kinda self evident, but only kinda. So since giving the talk I changed one thing in my personal usage of LTS: I don't use my personal LTS repo anymore, where I made sure only good packages got in. This is for two reasons: a.) I had too add new packages too often and b.) if it really is a problem that LTS has no gatekeeping mechanism (which I'm not sure anymore it is, after all, the updates are prepared by reasonable people with a common goal...) then I want to suffer this first hand, so I can build solutions which benefit everyone, not just me. That personal LTS repo only helped me. On the technical side I prepared five DLAs, for lzo2, libwpd, squid3, lua5.1 and bind9. Not much to see here, they all were very smooth. I still enjoyed the challenge of digging in unknown sourcecode, as described in my previous post. Then more interestingly, and with the help of Raphael Geissert and Salvatore Bonaccorso I fixed the security-tracker to also know about oldstable, after waiting for more than 8 weeks to someone else doing it. I'm very glad that this is done now, as without it was really tedious to check which issues were applying to oldstable. Oh, and another afterthought from giving the talk: currently at least parts of the security-tracker codebase assume that there won't ever be support for oldoldstable, but once jessie has been released this won't be true anymore. Then we will support stable, oldstable and oldoldstable. And oldstable will be wheezy, not squeeze. We have something like 6 months to fix this, hopefully we won't have much more time... Oh, and surely there are other places than just the security-tracker which will need to be taught about this.

Debian LTS - feedback about the feedback from my LTS talk at DebConf14 So, I'm more or less back from dc14 and today, five days later, I think I might have mostly overcome jetlag. Probably... So, at DebConf14 I gave a talk about LTS and while I'm sorry that I was that tired, I'm more or less happy how the talk went. Thankfully at least I was calm and relaxed. There are a couple of things I learned from the talk: a.) LTS has been really really perceived well b.) it fits a demand c.) people already take it for granted (eg plan for Wheezy LTS) d.) people expect the same non-intrusive changes as currently done for security updates. To explain the last point: when I explained the - so far - rather theoretical problem that ''squeeze-lts'' has no gatekeeper mechanisms whatsoever (eg no ''proposed-updates'', no NEW queue..) the reaction in the audience was basically "something like this should exist, else how can we deploy this in large scale / on important setup?!". Also currently there is no, well-documented, easily to be found policy for what kind of updates are acceptable. I said that we basically follow the same rules as there are for debian-security updates, but this should really be documented properly. This doesn't seem very hard to fix, just like many things it "just" needs someone to do the work. IOW: we explain how to use LTS, we explain how to contribute to LTS (through uploads or financially) but we lack a simple explaination what LTS is and what kind of updates to expect. It's kinda self evident, but only kinda. So since giving the talk I changed one thing in my personal usage of LTS: I don't use my personal LTS repo anymore, where I made sure only good packages got in. This is for two reasons: a.) I had too add new packages too often and b.) if it really is a problem that LTS has no gatekeeping mechanism (which I'm not sure anymore it is, after all, the updates are prepared by reasonable people with a common goal...) then I want to suffer this first hand, so I can build solutions which benefit everyone, not just me. That personal LTS repo only helped me. On the technical side I prepared five DLAs, for lzo2, libwpd, squid3, lua5.1 and bind9. Not much to see here, they all were very smooth. I still enjoyed the challenge of digging in unknown sourcecode, as described in my previous post. Then more interestingly, and with the help of Raphael Geissert and Salvatore Bonaccorso I fixed the security-tracker to also know about oldstable, after waiting for more than 8 weeks to someone else doing it. I'm very glad that this is done now, as without it was really tedious to check which issues were applying to oldstable. Oh, and another afterthought from giving the talk: currently at least parts of the security-tracker codebase assume that there won't ever be support for oldoldstable, but once jessie has been released this won't be true anymore. Then we will support stable, oldstable and oldoldstable. And oldstable will be wheezy, not squeeze. We have something like 6 months to fix this, hopefully we won't have much more time... Oh, and surely there are other places than just the security-tracker which will need to be taught about this.

Following my blog post about Long Term Support for Debian squeeze, the news was picked up by Slashdot, Reddit (again), Barrapunto, Twitter, and Phoronix (in spite of their skepticism).

Over 300 persons, some representing their companies, contacted the security team since the news about the LTS came out - it all seemed like things were finally rrolling.

However, a few days before the coordination mailing list was setup, a not-so-friendly mail was received from a legal officer of a company that produces a RPM derivative with Long Term Support and paid support contracts. The company-that-can't-be-named from here on, due to trademark abuse.

Long story short: the Debian Squeeze LTS project has been boycotted and threatened. Unfair competition (antitrust law) has been brought up against the project, among other threats.

So, great move company-that-can't-be-named, you got it - there won't be LTS, it's been decided and the interested parties have been notified. Perhaps you want to take over the actual development?

As announced in the "Bits from the Security Team" (LWN copy) email a couple of weeks ago, it is possible that Debian squeeze will have Long Term Support, primarily in the way of security updates.

The news have made it to Softpedia and reddit and even though it has not been announced how long it would be supported, what use cases are planned, etc., the answer I can give is that it depends solely on people contributing to it.

I'd like to add that no, the LTS announcement is not related to the use of Debian on the ISS laptops, as mentioned in reddit - at least not that we are aware of. And to avoid possible confusion from the Softpedia article, the plan is LTS for the squeeze release.

From the emails we have received in response to the announcement, I do wonder however where are the people who expressed interest and signs of engagement last time there was a discussion about providing LTS for Debian.

Perhaps it wasn't stressed enough in the email, but those who are interested in benefiting from (and therefore contributing to) long term support, please do contact the security team NOW.

After all, the clock keeps ticking and Debian squeeze is going to reach its End of Life in less than two months otherwise.

One great feature of POSIX shells is the so-called heredoc. They are even available in languages such as Perl, PHP, and Ruby.

So where is the bashism?

It's in the implementation. What odd thing do you see below?


$ strace -fqe open bash -c 'cat <<EOF
foo
EOF' 2>&1 grep -v /lib
open("/etc/ld.so.cache", O_RDONLY O_CLOEXEC) = 3
open("/dev/tty", O_RDWR O_NONBLOCK O_LARGEFILE) = 3
open("/proc/meminfo", O_RDONLY O_CLOEXEC) = 3
[pid 6696] open("/tmp/sh-thd-1384296303", O_WRONLY O_CREAT O_EXCL O_TRUNC O_LARGEFILE, 0600) = 3
[pid 6696] open("/tmp/sh-thd-1384296303", O_RDONLY O_LARGEFILE) = 4
[pid 6696] open("/etc/ld.so.cache", O_RDONLY O_CLOEXEC) = 3
foo
--- SIGCHLD (Child exited) @ 0 (0) ---


Yes, it uses temporary files!

So do ksh, pdksh, mksh, posh and possible other shells. Busybox's sh and dash do not use temporary files, though:


$ strace -fqe open dash -c 'cat <<EOF
foo
EOF' 2>&1 grep -v /lib
open("/etc/ld.so.cache", O_RDONLY O_CLOEXEC) = 3
[pid 6767] open("/etc/ld.so.cache", O_RDONLY O_CLOEXEC) = 3
foo
--- SIGCHLD (Child exited) @ 0 (0) ---


Next time you want data to never hit a hard disk, beware that heredocuments and herestrings are best avoided.

You've probably already done some basic maths in shell scripts, but do you know what else you can actually do?

Pick at least 4 operations that you can do in bashisms-free shell scripts:

$((n+1))
$((n>8))
$((n^4))
$((--n))
$((n*=5))
$((n++))
$((n==1?2:3))

The POSIX:2001 standard defines the arithmetic expansion requirements, which leads us to selecting all of the above operations except two:

$((--n))
$((n++))

"--" and "++" are not required to be implemented, and in some cases they may lead to unexpected results, such as the following:


$ bash -c 'n=1; echo $((++n))'
2
$ dash -c 'n=1; echo $((++n))'
1


Remember, if you rely on any non-standard behaviour or feature make sure you document it and, if feasible, check for it at run-time.

A new version of the code behind http.debian.net was deployed a few days ago. It has proved to be far more stable, faster, and scalable compared to the previous mod_perl-based deployment.

There were a couple of glitches during the earlier roll-out for IPv6 users, fixed thanks to the reports by Cyril Brulebois, Michael Stapelberg and Robert Drake.

What's behind?
The redirector is now a plack application (with no middlewere) running under the Starman server with an apache fronted. Requests are processed faster than before and there mod_perl-induced system overload is finally gone.

The redirector is now easier to test and develop. Deploying the live instance is not yet fully streamlined, but it has seen a lot of improvement. Some important changes to the way the redirector works are already on their way to see the light and I am going to be announcing them when they do. Fork the repository and hack a few changes, contributions are welcome :)

It's probably time to move it under debian.org, and finish making it the default everywhere. It's even made its way into the installation manual.

As a user of a shell you have most likely had the need to export a variable to another process; i.e. set/modify an environment variable.

Now, how do you stop exporting an environment variable? can you export anything else?

The bash shell offers the -n option of the export built-in, claiming it "remove[s] the export property". However, this feature is not part of the POSIX:2001 standard and is, therefore, a bashism.

A portable way to stop exporting an environment variable is to unset it. E.g. the effect of "export MY_VAR=foo" can be reverted by calling "unset MY_VAR" - surely enough, this will also destroy the content of the variable.

An equivalent could then be:

# to stop exporting/"unexport" the MY_VAR environment variable:
my_var="$MY_VAR" ; unset MY_VAR ;
MY_VAR="$my_var" ; unset my_var ;

The above code will make a copy of the variable before destroying it and then restoring its content.

How about exporting other things? did you know that you can export shell functions?

With the bash shell, you can export a function with the -f parameter of the export built-in. Needless to say, this is a bashism. Its secret? it's just an environment variable with the name of the function and the rest of the function definition as its value.

Try this:

$ echo="() /bin/echo 'have some bash' ; " bash -c 'echo "Hello World!"'
have some bash

Yes, this means that if you can control the content of an environment variable passed to bash you can probably execute whatever code you want. It comes handy when you want to alter a script's behaviour without modifying the script itself.

Possibilities are endless thanks to bash's support for non-standard characters in function names. Functions with slashes can also be exported, for example:

/sbin/ifconfig()
echo "some people say you should be using ip(1) instead" ;


Are you into bug hunting? export exec='() echo mount this ; '

In a response to my blogpost about bashisms in function names, reader Detlef L pointed out in a comment that aliases allow non-standard characters in their names, contrary to functions. They could then be used to, for example, set an alias of the run-parts(1) command (cf. the blog post).

Aliases indeed allow characters such as commas ( , ) to be used in the alias name. However, aliases are an extension to the POSIX:2001 specification and are therefore bashisms. Moreover, the characters set defined by POSIX does not include dashes.

Last but not least, aliases belong to the list of shell features that are usually "disabled" when the shell is run in non-interactive mode. I.e.


$ bash <<EOF
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

alias disabled
$ bash -i <<EOF # force interactive mode
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

$ alias true=false;
$ if true; then echo alias disabled; else echo alias enabled; fi
alias enabled
$ exit


To add to the fun of different behaviours, other shells always expand aliases.

If you decide to play with aliases you should note one thing: they are only enabled from the line after the definition. E.g. note the difference below

$ dash -c 'alias foo="echo foo"; foo'
dash: foo: not found
$ dash -c 'alias foo="echo foo";
foo'
foo


You are in need of support and you go ask in project x's forums (mailing list, online board, etc.) and get something worse than no support:

Responses from an incompetent with plenty of time and lots of FUD to spread.

Now that we have "umarell" thanks to Enrico Zini and other Italian folks, we need a name for the above.

Apologies to some readers who might be disappointed by this post. I don't like calling people names but I've seen too many of the above in less than two hours not to be tired.

The bashism of this week deals again with tests. How do you compare two strings to tell which one goes before or after the other in the alphabet in a shell script?
(also know as lexicographic comparison)

If you are familiar with perl and the relationship between its comparison operators and that of shell scripts, you would probably write the test as follows:


$ test bar '<' foo
$ test foo '>' bar


And yes, that would work, but not with all shells. I'm afraid to tell you that the < and > comparison operators are not required by the POSIX:2001 specification and are, therefore, bashisms.

What does this have to do with perl? well, the shell way is the inverse of the perl way.
What? In Perl you test the equality of two numbers with '==', in shell you use '-eq'; in perl the equality of strings is 'eq', in shell it is '='. In shell scripts you can compare two numbers with -gt (test 1 -gt 0) while in perl the equivalent comparison is with > (1 > 0).

So in this case the perl way is "bar lt foo" and based on the above the shell way should be "bar < foo". However, care must be taken when using such operators in shell scripts. Since < and > are used for redirections in shell scripts, one must quote them even when using bash. An alternative is to use the [[ special test function of bash which alters the shell syntax.

This time I didn't write a function to portably (or "somewhat portably") replace such comparisons. Feel free to share your solution in the comments, with bonus points if you come up with a solution without using external commands.

Did you know that you can get a user's home directory with the "~username" tilde expansion? You probably do, but how about other tilde expansions?

Thanks to a few bashisms you can make your script even more difficult to read by using "~+" to get $PWD, and "~-" to get $OLDPWD.

Are you using bash and want to access the directories stack created when using the pushd and popd bashisms? The ~i (where i is an integer) expansion can give you that. It gives you forward (~+i) and backward (~-i) access to the directories stack.

But beware, if the directories stack is smaller than the number you used, there won't be any expansion.

When using tildes in shell scripts, make sure you quote to avoid unwanted expansions. Note that the posh shell in wheezy and older do support those non-POSIX expansions.

in the first examples the tilde is quoted for easier reading; the expansion doesn't occur if the tilde is quoted.