Recently, I ate at the somewhat famous London vegetarian restaurant Indian Veg Bhelpoori House in Islington (often referred to simply as Indian Veg ).
I couldn t help but imagine that the restaurant had hired Emanuel Bronner as their interior decorator.
If all goes well, a new release of dput-ng will hit the archive this week (depending on when I can find time to close the last 2 bigger bugs) - anyone interested in getting a new feature into dput-ng should email me (or file a reportbug bug at dput-ng)
Small list of changes so far:
* Add per-host loading of command files by partially parsing command
line arguments.
* Add disallowed_distributions.
* Add codename groupping support.
* Add codename groups to vital targets, limit security uploads to the right servers (Closes: #708575).
* Implement dcut commands for Deb-o-Matic.
* Add an override and a unset option to dput which overrules any
profile key. This is an experimental option.
* Be more precise about our configuration file inheritance in dput(5).
* Fix dcut: manpage and help talk about -U and upload by replacing those parts in the text by the upload command (Closes: #699812)
* Make the login name determination more portable (Closes: #709831)
* Fix Allow to give out dm permissions without using local keyring by
* documenting the force option in the man page (Closes: #711057)
* Allow uploads to mentors to target every distribution
* Add a space in debug s help message.
* Fixed a typo (DEBNAME DEBFULLNAME) in the dcut(1) manpage.
Thanks, y all!
I've been singing for a long time. The first time I was a
member of a choir was when I was around 6-7 years old, and it was as a
member of the children's choir 'De Lambertijnen' in Ekeren, Belgium. Not
by coincidence, Ekeren is where I grew up.
However, I didn't remain a member of 'De Lambertijnen' for very long.
After a year or two, I was over the maximum age for members of that
choir, and "promoted" to Carmina, another children's choir of the same
association. I didn't like that choir as much, though, and only
remained a member for about a year or so, before I left that choir; and
for a few years, choral music was no longer part of my interests.
Then, when I turned sixteen, I was in a school where choral music was
part of the curriculum; and through class, I found out that singing in a
choir is something I actually love. By that time I was too old to join
Carmina again, so I joined the C-koren's youth choir "Cantilene" under
Luc Anthonis, instead. Luc is one of Belgium's finest choir conductors,
and so I remained a member until my 26th birthday when, again, I
was deemed too old to remain a member of that choir.
At this point, the next choir in line was "Con Amore", a choir that
does not have an age limit, but which in practice consists of mostly
people who are retired (or nearly so, at least). As a result, most
people who leave Cantilene do not join Con Amore, instead ending their
foray into choral music. I was no exception.
However, a year and a half (or so) later, the C-koren were extended
with a new choir, "Caljent ", who jumpstarted their membership by
contacting ex-Cantilene members (me amongst them), inviting them to join
this new choir. I have been a member since the first official rehearsal
of Caljent , and have always enjoyed it; enough so that when I moved out
of Ekeren some five years ago, into a flat in Mechelen (a 30 minute car
trip away), I did not quit the choir.
I can, however, not deny that since moving out, my connection to
Ekeren has become fairly strained, and that there have been times when
I felt it difficult to motivate myself into driving all the way there
for a rehearsal on friday night. I felt that to be unfair towards the
conductor and towards the other members of the choir, and so it is with
mixed feelings that I decided a month or so ago to remain a member only
until the last rehearsal before the summer break this year.
That last rehearsal was today (well, technically yesterday by now);
and so, after a "career" of almost 20 years of nearly uninterrupted
membership (including one 5-year stint as a member of Cantilene's
governing board), I am now no longer a member of the C-koren.
That's not to say I won't be singing anymore in fact, I've
already joined a different choir whose rehearsal location just happens
to be right around the corner from where I live but it does
certainly feel weird to be putting an end to my C-koren membership.
Ah well, we'll see what the future brings.
Normally Skype isn't mentioned by name on this blog, but today I'm making an exception.
When a new scandal broke out in the Australian Army today, one of the first questions asked was how does it compare to the Skype Scandal which had everybody talking (and blushing) in 2011. A concise military reply direct from the General:
"I'd say it's worse than the Skype matter"
(the original Skype Scandal cadets are going on trial in six weeks)
Jedi Council brings cultural change in the Army: from Skype to email
Since the original scandal, there have been plenty of PR-orchestrated headlines about cultural change in the Army. Sadly, it seems that several ranking officers believed that was a cue to simply start using email instead of live feeds
Operating under the operational code name Jedi Council, they've apparently been swapping annotated images of female colleagues and members of the public service.
A more detailed account summarises it:
"They were calling themselves the Jedi Council but they were doing the exact opposite of what Jedis are supposed to do"
While the Skype scandal involved cadets behaving much like college kids elsewhere, this latest scandal involves the officer corp, the most senior officer implicated is allegedly a Lieutenant Colonel. Senior ranking officers like this have given many years of dedicated service to the country. They are widely respected and the vast majority of officers in these positions are unlikely to ever be implicated in such a scandal.
Officers are in leadership roles, it should be no surprise if they have top secret security clearances.
After the revelation that Australia has a direct and high-volume data sharing arrangement with the US Government (and in turn Facebook, Google, etc), there are all kinds of questions about whether a couple of rogue officers have access to pursue their private Jedi Council project using tools like PRISM and Boundless Informant. It's already been confirmed that they were keeping their own Jedi database of their victim's names, phone numbers, home addresses and various clothing measurements and if they are already emailing these photos about on Defence computers, does it seem improbable that they would be using PRISM or other sources to cross-reference their data?
Behavior described as systematic and predatory
It can't be ruled out that some defence computer systems may have direct access to NSA copies of images from those invasive airport body scanners - like these pictures that were not supposed to exist. Just like the British undercover cops stealing the identities of dead children and fully engaging with impressionable young women from environment campaign groups, there are a long list of possible abuses when the security state is over-empowered.
While this latest Army example involves private pursuits, it is not hard to imagine many other permutations: perhaps a couple of rogue Government employees making unauthorised use of PRISM data for a commercial objective and gaining the upper hand on private sector competitors who have no such data feed.
100% trust?
In every organisation there will always be a few people bending the rules. It's human nature, if it wasn't that way, we'd all be robots (or drones). The Army and the Catholic Church tend to be hammered a lot more in the public eye when these transgressions are discovered although statistically their crime rates are no higher than average. The bottom line is that no organisation is perfect and people have to take some responisbility to object to the excessive growth of the security state and keep our data to ourselves as you just never know who you can trust.
grep -i debian-multimedia.org /etc/apt/sources.list /etc/apt/sources.list.d/*
If you can see debian-multimedia.org line in output, you should remove all the lines including it.
My hunch has just been exposed rather dramatically today: Top secret data center under construction to house all our secrets.
"There has been no discussion of the project in Senate estimates committee hearings, and the public reports of Parliament's joint committee on intelligence and security make no reference to it."
The news story also suggests that the relationship is bi-directional, with data harvested from Australian ISPs cross referenced against US databases from Facebook and Google.
From a human rights perspective, the implications of this data sharing are much more dramatic than the NSA just snooping on US citizens within the US. While Australia often tries to portray the image that it is a safe and happy country, it has a dark side. Police are kept busy investigating politicians and Government officials: Prime Minister and her associates connected to a union slush fund, the central bank's banknote printing agency's chief executive and 8 colleagues on conspiracy charges and even a senator on good old fashioned shoplifting and assault charges
Exporting PRISM data to a human rights backwater
As an Australian, my own rights and those of my family are more heavily protected and respected living in Europe than if I lived in my country of birth. Australia does not have a bill of rights like the US (where PRISM data is harvested) or the European Convention of Human Rights. Once US data is exported to Australia, it can be used to support sinister programs like the indefinite detention of coloured people in sinister desert camps. Just this week, the Government left over 50 bodies of poor refugees for the sharks to eat. For someone with such a lack of a conscience, can you imagine them thinking twice about using PRISM data for unscrupulous purposes?
Is US PRISM data at greater risk?
Despite the stench of corruption and lip service to human rights, is there any hard evidence that PRISM data will be at greater risk if allowed into the hands of Australians?
Death by SMS
One former Australian official (having dual-citizenship with Sri Lanka) is accused of facilitating war crimes in the Sri Lankan civil war. At present he is Sri Lanka's ambassador at the UN, giving him the benefit of diplomatic immunity. When he was working for Australia, how much data did he have access to? Isn't it ludicrous that in the name of stopping terrorism, data may have been collected and handed to war criminals on a silver platter?
Behind enemy lines
Like James Bond in reverse, an Australian official was exposed unofficially engaging in a relationship with a Vietnamese spy colonel. In addition to the $20 million in bribery to win a bank note printing contract, is data being used as a tool to leverage trade negotiations, officially or otherwise? If an Australian official with top-secret clearance ends up under the control of a foreign agent, what is the risk that they would be exploited to obtain PRISM data?
Defence minister resignation in Chinese spy scandal
In 2009 Australia's defence minister was identified having close links to a Chinese-Australian business woman including all the usual little luxuries: first class flights, apartment rental and joint business deals with family members. He subsequently resigned. Little details were made public. How much PRISM data was directly available to political officials like the defence minister?
Disappearing Prime Ministers
The disappearance of a serving Prime Minister, Harold Holt, during the cold war remains an unsolved mystery to this day. People continue to ask was he swept out to sea or voluntarily whisked away in a Chinese or Soviet nuclear submarine. While Australia has traditionally had strong ties to Britain and the US, there have been exceptions to the rule. Fortunately this all happened before PRISM and Harold Holt red submarine rumours are no more well-founded than the UFO theories of his abduction.
Where did all the data go?
As the music industry has found out the hard way, digital data can be copied at will with the originator having no control.
The revelations that Britain and Australia are in on the PRISM shenanigans only confirms that this data is seen as a new form of currency for exchange in geo-political posturing. While used weapons take decades to work their way down the chain, data can spread at the speed of light.
foo
or a package name with a version requirement:
foo ( > 1.24 )
or a package name with architectures restrictions:
foo [alpha amd64 hurd-arm linux-armeb]
or both:
foo ( > 1.24 ) [alpha amd64 hurd-arm linux-armeb]
or a list of alternate choices combining the possibilities above:
foo ( > 1.24 ) bar [ linux-any] baz ( << 3.14 ) [ ! hurd-armel !hurd-armeb ]
or a variable that is replaced during package build:
$ perl-depends
Writing a Parse::RecDescent grammar to parse this is relatively straightforward.
The first production handles alternate dependencies separated by and raises an error if some text was not consumed by the dependencies:
dependency_item: depend(s /\ /) eofileA dependency as explained above is expressed as:
depend: pkg_dep variableA variable like $ foo or $ bar -1.24~ is parsed with:
variable: /\$ [\w:\-]+ [\w\.\-~+]*/This rule handles a package name with optional version or arch restriction:
pkg_dep: pkg_name dep_version(?) arch_restriction(?) pkg_name: /[a-z0-9][a-z0-9\+\-\.]+/The remaining rules are quite simple:
dep_version: '(' oper version ')'
oper: '<<' '<=' '=' '>=' '>>'
version: variable /[\w\.\-~:+]+/
arch_restriction: '[' arch(s) ']'
arch: /!?[\w-]+/
eofile: /^\Z/
The grammar above works well to parse the dependency. You can test it with this small Perl script:
#!/usr/bin/perl
use strict;
use warnings;
use 5.010 ;
use Parse::RecDescent ;
my $parser = Parse::RecDescent->new(join('',));
my $dep = shift ;
say "parsing '$dep'";
my $ret = $parser->dependency_item($dep) ;
say "result is ", $ret if $ref ;
__DATA__
# insert grammar here !!!
Unfortunately, any error in the optional parts (i.e version requirements and arch restriction) leads to an error message which is not very helpful. The error message only mention that some text could not be parsed:
parsing 'foo ( != 1.24 ) bar'
ERROR (line 1): Invalid dependency item: Was expecting /\ / but found
"( != 1.24 ) bar" instead
or
parsing 'foo [ arm & armel] bar'
ERROR (line 1): Invalid dependency item: Was expecting /\ / but found
"[ arm & armel] bar" instead
The problem comes from the fact that version requirements or arch restrictions are optional. For instance if a version requirement has a syntax error, Parse::RecDescent will try to parse it as an arch restriction. This arch restriction rule will also fail and the last terminal ( eofile ) will fail. So the error message does not hint at the actual syntax problem.
To generate better error messages, I improved the suggestion made in Parse::RecDescent FAQ.
Instead of calling a plain subroutine, I use a sub reference that will store the error messages in a closure. This sub ref is declared in a start-up action. Note that the sub ref explicitly returns undef. I ll explain why later.
my @dep_errors ;
my $add_error = sub
my ($err, $txt) = @_ ;
push @dep_errors, "$err: '$txt'" ;
return ;
;
The following production always fails while ensuring that the error list is reset. This production is always run at the beginning of the dependency parsing:
dependency: @dep_errors = ();Here s the actual dependency production that is run when dependency method is called on the parser. It will return an array ref containing (1, data) if the dependency is valid or (0, errors) otherwise:
dependency: depend(s /\ /) eofile
$return = [ 1 , @ $item[1] ] ;
push( @dep_errors, "Cannot parse: '$text'" ) unless @dep_errors ;
$return = [ 0, @dep_errors ];
The following productions don t change much:
depend: pkg_dep variable
variable: /\$ [\w:\-]+ [\w\.\-~+]*/
pkg_dep: pkg_name dep_version(?) arch_restriction(?)
dep_version: '(' oper version ')'
The first rule of this production parses the package name which must be followed by a space, end of string ( or ['. A positive look-ahead assertion is used so only the package name is consumed. If the first rule fails, the second rule provides a meaningful error message. The second rule will match anything which is not a space and create an error message. Since $add_error returns undef, the second rule returns undef and the production fails. So the text stored in the error message is not consumed:
pkg_name: /[a-z0-9][a-z0-9\+\-\.]+(?=\s \Z \( \[)/
/\S+/ $add_error->("bad package name", $item[1]) ;
The same trick is used with these productions:
oper: '<<' '=' '>>'
/\S+/ $add_error->("bad dependency version operator", $item[1]) ;
version: variable /[\w\.\-~:+]+(?=\s \) \Z)/
/\S+/ $add_error->("bad dependency version", $item[1]) ;
The action of this production is a little bit more tricky. The action ensures that '!' are either added before all arch or not at all. Otherwise an error message is generated and added to the list of errors:
arch_restriction: '[' osarch(s) ']'
my $mismatch = 0;
# $ref contains ['!',os,arch] or ['',os,arch]
my $ref = $item[2] ;
for (my $i = 0; $i < $#$ref -1 ; $i++ )
$mismatch = ($ref->[$i][0] xor $ref->[$i+1][0]) ;
my @a = map ($_->[0] '') . ($_->[1] '') . $_->[2] @$ref ;
if ($mismatch)
$add_error->("some names are prepended with '!' while others aren't.", "@a") ;
else
$return = 1 ;
The check above is possible only if the "osarch" production returns an array ref containing something like ('!','linux','any') for "!linux-any' or ('','linux','any') for "linux-any":
osarch: not(?) os(?) arch
$return = [ $item[1][0], $item[2][0], $item[3] ];
/.?(?=\s \] \Z)/ $add_error->("bad arch specification: ", $item[1]) ;
not: '!'
Here's the remaining of the grammar:
os: /(any uclibc-linux linux kfreebsd knetbsd etc...)-/
/\w+/ '-' $add_error->("bad os in architecture specification", $item[1]) ;
arch: / (any alpha amd64 arm\b arm64 etc... )
(?=(\] ))
/x
/\w+/ $add_error->("bad arch in architecture specification", $item[1]) ;
eofile: /^\Z/
That's all for grammar 2.0...
Before someone yells: "Show me the message ! ", here are some example of bad dependencies and their error message generated by the parser:
parse 'foo ( != 1.24 ) bar' result is: 0 bad dependency version operator: '!=' parsing 'foo [ arm & armel] bar' result is: 0 bad arch specification: : '&' parsing 'foo [ arm armel ] bar [!moo]' result is: 0 bad arch specification: : ']' bad arch in architecture specification: 'moo'The 2 first error messages are spot on the actual error. The second one has a false positive (']' is correct) but correctly highlights the wrong arch name ('moo'). Mission accomplished. In order to keep this post (relatively) simple, I've removed the part that actually store parsed data. They don't really matter for error handling. Nevertheless, you may see the whole grammar in Config::Model::Dpkg::Dependency module. All the best
When I last lived in Seattle, nearly a decade ago, I hosted the Debian Seattle Social email list. When I left the city, the mailing list eventually fell victim to bitrot.
When Allison Randall asked me about the list a couple months ago, I decided that moving back to Seattle was a good excuse to work with Allison and some others to revive the community. Toward that end, I ve put up a little website and created a new mailing list. It s hosted on Alioth this time which will be reliable than me. Since it has been years, we have not moved over the old subscriber list so you ll have to sign up again if you were on it before.
If you re a Debian developer or user and you d like to hear about infrequent Debian social gatherings in the Seattle area, you should sign up on the list!
Migrating a Debian installation between architectures has always been difficult. The recommended way to
crossgrade an i386 Debian to amd64 Debian was to reinstall the system, move over data and configuration.
For the more brave, in-place crossgrades usually involved chroots, rescue CDs, a lot of ar p tar xf - data.tar.gz and
luck.
I have never been brave when it comes to system administration, have done a lot of architecture migrations with
reinstallation, and have always taken the opportunity to clear out the contamination that accumulates itself when a
system is running for a long time. I would even recommend doing this to most people even now. However, I have a few very
ugly systems in place that are still on i386 because I didn t dare going the reinstallation path.
Doing in-place crossgrades has become a lot easier since wheezy s release, since once now can have both i386 and
amd64 libraries installed in parallel, which allows to replace foo:i386 with foo:amd64 without influencing the other
parts of the system. The process is still full of pitfalls:
So we have an i386 system running an x86_64 kernel. Next, we add amd64 as an additional architecture which allows us to install amd64 packages. We download the amd64 packages files. Note that dpkg still considers this an i386 system with amd64 as a foreign arch.mh@swivel:~$ ssh wheezyarch.zugschlus.de $ uname -a Linux wheezyarch 3.9.4-zgsrv2008064 #2 SMP PREEMPT Wed Jun 5 12:57:51 UTC 2013 x86_64 GNU/Linux $ dpkg --print-architecture i386 $ dpkg --print-foreign-architectures $ dpkg --list grep i386 wc -l 175 $ dpkg --list grep amd64 wc -l 0 $ sudo dpkg --get-selections grep install awk print $1 sed s/:.*//;s/$/:amd64 install/; > get-selections.pre
Next, download everything that is necessary to migrate dpkg and apt from i386 and amd64. We cannot directly apt things as the remove-install cycle used by apt will leave us without working dpkg. One could work around this by un-aring the dpkg.deb and directly forcing the amd64 files upon the system, but I think this approach is much cleaner. apt-get download will not resolve dependices while apt-get --download install does. After the download, dpkg --install the packages. dpkg is not particularly smart about installation order, so Pre-Depends may fail or not. If they fail, dpkg will give a list of deb files that were affected, which is an easy help to finish their installation. One could also choose the easy way (which I did here) and repeat the dpkg --install *.deb command. Since the Pre-Depends will already be installed, the second install run will succeed.$ sudo dpkg --add-architecture amd64 $ sudo apt-get update Get:1 http://security.debian.org wheezy/updates Release.gpg [836 B] ( ) Fetched 6.155 kB in 4s (1.400 kB/s) Reading package lists... Done $ dpkg --print-architecture i386 $ dpkg --print-foreign-architectures amd64 $ dpkg --list grep i386 wc -l 175 $ dpkg --list grep amd64 wc -l 0
From dpkg s point of view, the system is now already amd64, with i386 being a foreign arch. However, the majory of packages is still i386.$ sudo apt-get --download-only install dpkg:amd64 apt:amd64 Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: gcc-4.7-base:amd64 libapt-pkg4.12:amd64 libbz2-1.0:amd64 libc6:amd64 libgcc1:amd64 liblzma5:amd64 libselinux1:amd64 libstdc++6:amd64 zlib1g:amd64 Suggested packages: aptitude:amd64 synaptic:amd64 wajig:amd64 dpkg-dev:amd64 apt-doc:amd64 python-apt:amd64 glibc-doc:amd64 locales:amd64 The following packages will be REMOVED: apt cron-apt dpkg The following NEW packages will be installed: apt:amd64 dpkg:amd64 gcc-4.7-base:amd64 libapt-pkg4.12:amd64 libbz2-1.0:amd64 libc6:amd64 libgcc1:amd64 liblzma5:amd64 libselinux1:amd64 libstdc++6:amd64 zlib1g:amd64 0 upgraded, 11 newly installed, 3 to remove and 0 not upgraded. Need to get 10,0 MB of archives. After this operation, 14,9 MB of additional disk space will be used. Do you want to continue [Y/n]? Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main gcc-4.7-base amd64 4.7.2-5 [144 kB] ( ) Fetched 10,0 MB in 2s (3.535 kB/s) Download complete and in download only mode $ sudo dpkg --install /var/cache/apt/archives/*.deb (Reading database ... 16586 files and directories currently installed.) Preparing to replace apt 0.9.7.8 (using .../archives/apt_0.9.7.8_amd64.deb) ... Unpacking replacement apt ... dpkg: regarding .../dpkg_1.16.10_amd64.deb containing dpkg, pre-dependency problem: dpkg pre-depends on libbz2-1.0 dpkg: error processing /var/cache/apt/archives/dpkg_1.16.10_amd64.deb (--install): pre-dependency problem - not installing dpkg Selecting previously unselected package gcc-4.7-base:amd64. ( ) Errors were encountered while processing: /var/cache/apt/archives/dpkg_1.16.10_amd64.deb $ sudo dpkg --install /var/cache/apt/archives/*.deb (Reading database ... 16905 files and directories currently installed.) Preparing to replace apt 0.9.7.8 (using .../archives/apt_0.9.7.8_amd64.deb) ... Unpacking replacement apt ... ( ) $
The apt resolver is currently in a very bad broken state since the system is missing most essential packages in its native architecture. This is not easily solved by apt-get -f install, as this would zap most of the existing system . We let apt-get --download-only -f install download the packages that it wants to replace with its amd64 counterparts and their dependencies, and, again, use dpkg --install *.deb to install them.$ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 173 $ dpkg --list grep amd64 wc -l 11 $
The concluding apt-get clean prevents packages from being reinstalled again in the next dpkg --install *.deb run. Now, the apt resolver is kind of fixed, but apt will still want to remove most of the system. I haven t found a way to get out of this mess short of allowing apt to de-install those packages and to manually install their amd64 counterparts later. First, we see what apt wants to do, and paste the The following packages will be removed part of apt-get s output into a sed expression which fixes the architecture strings for us so that we can use the resulting file as input for the following installation procedure. After that, we let apt-get do what itself considers a bad thing and requires us to type a consent sentence. The key is that this deinstallation of essential packages breaks the system, but not apt and dpkg, which allows us to un-break the system afterwards.$ sudo apt-get --download-only -f install Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: cpio gettext-base:i386 klibc-utils libapt-inst1.5 libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libklibc libreadline5:i386 Use apt-get autoremove to remove them. The following extra packages will be installed: acpid aide bsd-mailx cpio cron curl daemon dctrl-tools exim4-base exim4-daemon-light initscripts insserv kbd klibc-utils less libbsd0 libcomerr2 libcurl3 libdb5.1 libexpat1 libgcrypt11 libgdbm3 libgnutls26 libgpg-error0 libgssapi-krb5-2 libidn11 libk5crypto3 libkeyutils1 libklibc libkmod2 libkrb5-3 libkrb5support0 libldap-2.4-2 liblockfile1 libncursesw5 libnewt0.52 libp11-kit0 libpam-modules libpam0g libpcre3 libpopt0 libreadline6 librtmp0 libsasl2-2 libslang2 libsqlite3-0 libssh2-1 libssl1.0.0 libtasn1-3 libterm-readkey-perl libtinfo5 openssl perl perl-base python2.7 python2.7-minimal sysvinit-utils whiptail Suggested packages: libarchive1 anacron logrotate checksecurity debtags mail-reader eximon4 exim4-doc-html exim4-doc-info spf-tools-perl swaks bootchart2 rng-tools krb5-doc krb5-user libpam-doc perl-doc make python2.7-doc binutils binfmt-support bootlogd sash Recommended packages: exim4 postfix mail-transport-agent psmisc mailx ca-certificates krb5-locales libgpm2 libfribidi0 libsasl2-modules libpng12-0 The following packages will be REMOVED: acpid:i386 aide:i386 aptitude:i386 bsd-mailx:i386 cpio:i386 cron:i386 curl:i386 daemon:i386 dctrl-tools:i386 debsecan dmsetup:i386 e2fsprogs:i386 exim4-base:i386 exim4-daemon-light:i386 git:i386 grub-common:i386 grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386 initramfs-tools initscripts:i386 insserv:i386 ippl:i386 jed:i386 kbd:i386 klibc-utils:i386 less:i386 libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 libklibc:i386 libnewt0.52:i386 libparted0debian1:i386 libterm-readkey-perl:i386 lsof:i386 lvm2:i386 molly-guard ntp:i386 openssh-server:i386 openssl:i386 parted:i386 perl:i386 perl-base:i386 procps:i386 python-apt:i386 python2.7:i386 python2.7-minimal:i386 rsyslog:i386 sysvinit:i386 sysvinit-utils:i386 udev:i386 util-linux:i386 whiptail:i386 The following NEW packages will be installed: acpid aide bsd-mailx cpio cron curl daemon dctrl-tools exim4-base exim4-daemon-light initscripts insserv kbd klibc-utils less libbsd0 libcomerr2 libcurl3 libdb5.1 libexpat1 libgcrypt11 libgdbm3 libgnutls26 libgpg-error0 libgssapi-krb5-2 libidn11 libk5crypto3 libkeyutils1 libklibc libkmod2 libkrb5-3 libkrb5support0 libldap-2.4-2 liblockfile1 libncursesw5 libnewt0.52 libp11-kit0 libpam-modules libpam0g libpcre3 libpopt0 libreadline6 librtmp0 libsasl2-2 libslang2 libsqlite3-0 libssh2-1 libssl1.0.0 libtasn1-3 libterm-readkey-perl libtinfo5 openssl perl perl-base python2.7 python2.7-minimal sysvinit-utils whiptail 0 upgraded, 58 newly installed, 53 to remove and 0 not upgraded. Need to get 23,4 MB of archives. After this operation, 17,5 MB disk space will be freed. Do you want to continue [Y/n]? Y Get:1 http://security.debian.org/debian-security/ wheezy/updates/main libgnutls26 amd64 2.12.20-7 [619 kB] ( ) Fetched 23,4 MB in 11s (2.038 kB/s) Download complete and in download only mode $ sudo dpkg --install /var/cache/apt/archives/*.deb (Reading database ... 16905 files and directories currently installed.) Preparing to replace acpid 1:2.0.16-1+deb7u1 (using .../acpid_1%3a2.0.16-1+deb7u1_amd64.deb) ... ( ) Selecting previously unselected package libpam-modules:amd64. dpkg: regarding .../libpam-modules_1.1.3-7.1_amd64.deb containing libpam-modules:amd64, pre-dependency problem: libpam-modules pre-depends on libdb5.1 libdb5.1:amd64 is unpacked, but has never been configured. dpkg: error processing /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb (--install): pre-dependency problem - not installing libpam-modules:amd64 Selecting previously unselected package libpcre3:amd64. Unpacking libpcre3:amd64 (from .../libpcre3_1%3a8.30-5_amd64.deb) ... ( ) Errors were encountered while processing: /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb $ sudo dpkg --install /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb (Reading database ... 17007 files and directories currently installed.) Unpacking libpam-modules:amd64 (from .../libpam-modules_1.1.3-7.1_amd64.deb) ... Setting up libpam-modules:amd64 (1.1.3-7.1) ... $ sudo apt-get clean $ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 151 $ dpkg --list grep amd64 wc -l 66 $
This process may re-ask some debconf questions that you have already answered during initial system setup. I find it comforting to actually see that my answers were preserved, but you can always export DEBIAN_FRONTEND=noninteractive if you want installation to be silent. If you know how this step can be done more elegantly, please comment. We now, finally, have the resolver in a consistent state.$ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: gettext-base:i386 libapt-inst1.5 libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. The following extra packages will be installed: sysvinit-utils:i386 Suggested packages: bootlogd:i386 sash:i386 The following packages will be REMOVED: adduser aide-common aptitude:i386 bsd-mailx console-common console-data console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386 exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386 grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386 initramfs-tools initscripts ippl:i386 jed:i386 libcurl3 libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386 libswitch-perl libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist openssl-blacklist-extra parted:i386 perl perl-modules procps:i386 python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils tzdata ucf udev:i386 util-linux:i386 The following NEW packages will be installed: sysvinit-utils:i386 WARNING: The following essential packages will be removed. This should NOT be done unless you know exactly what you are doing! e2fsprogs:i386 util-linux:i386 (due to e2fsprogs:i386) sysvinit:i386 sysvinit-utils tzdata (due to util-linux:i386) 0 upgraded, 1 newly installed, 62 to remove and 0 not upgraded. Need to get 97,1 kB of archives. After this operation, 131 MB disk space will be freed. You are about to do something potentially harmful. To continue type in the phrase Yes, do as I say! ? (abort with Ctrl-C) $ sed s/:i386//g > removedpkg (paste The following packages will be REMOVED: stanza here and type Ctrl-D afterwards) adduser aide-common aptitude:i386 bsd-mailx console-common console-data console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386 exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386 grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386 initramfs-tools initscripts ippl:i386 jed:i386 libcurl3 libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386 libswitch-perl libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist openssl-blacklist-extra parted:i386 perl perl-modules procps:i386 python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils tzdata ucf udev:i386 util-linux:i386 $ cat removedpkg adduser aide-common aptitude bsd-mailx console-common console-data console-log cron curl debian-goodies debsecan dmsetup e2fsprogs exim4-base exim4-config exim4-daemon-light git grub-common grub-pc grub-pc-bin grub2-common ifupdown initramfs-tools initscripts ippl jed libcurl3 libdevmapper-event1.02.1 libdevmapper1.02.1 liberror-perl libfile-find-rule-perl libnumber-compare-perl libparted0debian1 libswitch-perl libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl libtimedate-perl locales lsof lvm2 molly-guard ntp openssh-client openssh-server openssl openssl-blacklist openssl-blacklist-extra parted perl perl-modules procps python-apt rsyslog stow sysv-rc sysvinit sysvinit-utils tzdata ucf udev util-linux $ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: gettext-base:i386 libapt-inst1.5 libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. The following extra packages will be installed: sysvinit-utils:i386 Suggested packages: bootlogd:i386 sash:i386 The following packages will be REMOVED: adduser aide-common aptitude:i386 bsd-mailx console-common console-data console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386 exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386 grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386 initramfs-tools initscripts ippl:i386 jed:i386 libcurl3 libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386 libswitch-perl libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist openssl-blacklist-extra parted:i386 perl perl-modules procps:i386 python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils tzdata ucf udev:i386 util-linux:i386 The following NEW packages will be installed: sysvinit-utils:i386 WARNING: The following essential packages will be removed. This should NOT be done unless you know exactly what you are doing! e2fsprogs:i386 util-linux:i386 (due to e2fsprogs:i386) sysvinit:i386 sysvinit-utils tzdata (due to util-linux:i386) 0 upgraded, 1 newly installed, 62 to remove and 0 not upgraded. Need to get 97,1 kB of archives. After this operation, 131 MB disk space will be freed. You are about to do something potentially harmful. To continue type in the phrase Yes, do as I say! ?] Yes, do as I say! (actually type this!) Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main sysvinit-utils i386 2.88dsf-41 [97,1 kB] Fetched 97,1 kB in 0s (687 kB/s) (Reading database ... 17050 files and directories currently installed.) Removing aide-common ... ( ) dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing e2fsprogs ... Removing sysv-rc ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing sysvinit ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing sysvinit-utils ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing ucf ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing libdevmapper1.02.1:i386 ... Removing libswitch-perl ... Removing perl ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing dmsetup ... Removing perl-modules ... dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. Removing util-linux ... Removing tzdata ... Processing triggers for mime-support ... Selecting previously unselected package sysvinit-utils. (Reading database ... 9802 files and directories currently installed.) Unpacking sysvinit-utils (from .../sysvinit-utils_2.88dsf-41_i386.deb) ... Setting up sysvinit-utils (2.88dsf-41) ... $ sudo apt-get install $(cat removedpkg) Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. The following extra packages will be installed: e2fslibs gettext-base libapt-inst1.5 libasprintf0c2 libattr1 libblkid1 libboost-iostreams1.49.0 libcap2 libcurl3-gnutls libcwidget3 libedit2 libept1.4.12 libfreetype6 libfuse2 libgpm2 libncurses5 libopts25 libprocps0 libreadline5 libsepol1 libsigc++-2.0-0c2a libss2 libudev0 libuuid1 libwrap0 libxapian22 logrotate Suggested packages: liblocale-gettext-perl tasksel debtags unicode-data anacron checksecurity popularity-contest xdg-utils zenity gpart e2fsck-static mail-reader eximon4 exim4-doc-html exim4-doc-info spf-tools-perl swaks git-daemon-run git-daemon-sysvinit git-doc git-el git-arch git-cvs git-svn git-email git-gui gitk gitweb multiboot-doc grub-emu xorriso desktop-base isc-dhcp-client dhcp-client ppp rdnssd net-tools gpm libcwidget-dev fuse libparted0-dev libparted0-i18n xapian-tools ntp-doc ssh-askpass libpam-ssh keychain monkeysphere rssh ufw parted-doc perl-doc make libpod-plainer-perl python-apt-dbg python-gtk2 python-vte python-apt-doc rsyslog-mysql rsyslog-pgsql rsyslog-doc rsyslog-gnutls rsyslog-gssapi rsyslog-relp doc-base sysv-rc-conf bum bootlogd sash util-linux-locales dosfstools Recommended packages: aptitude-doc-en aptitude-doc apt-xapian-index libparse-debianchangelog-perl exim4 postfix mail-transport-agent psmisc mailx patch rsync ssh-client os-prober busybox busybox-initramfs busybox-static ca-certificates uuid-runtime xauth ncurses-term iso-codes usbutils The following packages will be REMOVED: gettext-base:i386 libboost-iostreams1.49.0:i386 libcwidget3:i386 libept1.4.12:i386 libopts25:i386 libxapian22:i386 logrotate:i386 sysvinit-utils:i386 The following NEW packages will be installed: adduser aide-common aptitude bsd-mailx console-common console-data console-log cron curl debian-goodies debsecan dmsetup e2fslibs e2fsprogs exim4-base exim4-config exim4-daemon-light gettext-base git grub-common grub-pc grub-pc-bin grub2-common ifupdown initramfs-tools initscripts ippl jed libapt-inst1.5 libasprintf0c2 libattr1 libblkid1 libboost-iostreams1.49.0 libcap2 libcurl3 libcurl3-gnutls libcwidget3 libdevmapper-event1.02.1 libdevmapper1.02.1 libedit2 libept1.4.12 liberror-perl libfile-find-rule-perl libfreetype6 libfuse2 libgpm2 libncurses5 libnumber-compare-perl libopts25 libparted0debian1 libprocps0 libreadline5 libsepol1 libsigc++-2.0-0c2a libss2 libswitch-perl libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl libtimedate-perl libudev0 libuuid1 libwrap0 libxapian22 locales logrotate lsof lvm2 molly-guard ntp openssh-client openssh-server openssl openssl-blacklist openssl-blacklist-extra parted perl perl-modules procps python-apt rsyslog stow sysv-rc sysvinit sysvinit-utils tzdata ucf udev util-linux WARNING: The following essential packages will be removed. This should NOT be done unless you know exactly what you are doing! sysvinit-utils:i386 0 upgraded, 89 newly installed, 8 to remove and 0 not upgraded. Need to get 55.3 MB of archives. After this operation, 134 MB of additional disk space will be used. You are about to do something potentially harmful. To continue type in the phrase Yes, do as I say! ?] Yes, do as I say! (actually type this!) Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main e2fslibs amd64 1.42.5-1.1 [197 kB] ( ) Fetched 55.3 MB in 20s (2648 kB/s) perl: warning: Setting locale failed. ( )
Now we can force-crossgrade all i386 packages that are still left on the system. Again, we use the apt-get --download install, dpkg --install stunt as coreutils gets crossgraded at this step and dpkg is less than happy if it finds itself without /bin/rm. Remember, with --download, nothing is actually removed during this step even if apt-get claims to remove things. Apt will also complain that many of these packages are already installed. Those are libs that were pulled in previously. The first dpkg --install run in this step will complain about gazillions of dependency problems, which will all be resolved in concluding apt-get -f install after the second dpkg --install run, this time a manual one.$ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. $ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 121 $ dpkg --list grep amd64 wc -l 119 $
We have completed the actual crossgrade. All non-multiarch packages are now amd64. The resolver is now fine as well. At this current state, all i386 packages that are still installed should be unneeded libs. We can safely nuke them. Careful minds may inspect the list of packages to be removed for important or non-lib items. After completion of the command, a second call of the same command verifies that we were successful.$ sudo apt-get --download-only install $(dpkg --list awk if( $1 == ii && $4 == i386 ) print $2 sed s/:i386// ) Reading package lists... Done Building dependency tree Reading state information... Done ( ) The following packages were automatically installed and are no longer required: libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. Suggested packages: powermgmt-base bash-doc bzip2-doc diffutils-doc wdiff mlocate locate gnupg-doc xloadimage imagemagick eog libpcsclite1 iproute-doc resolvconf avahi-autoipd nfs-common ncompress indent Recommended packages: bsdmainutils gnupg-curl libatm1 The following packages will be REMOVED: anacron:i386 apt-utils:i386 base-files:i386 base-passwd:i386 bash:i386 ( ) The following NEW packages will be installed: anacron apt-utils base-files base-passwd bash bsdutils busybox bzip2 ( ) 0 upgraded, 56 newly installed, 49 to remove and 0 not upgraded. Need to get 24,1 MB of archives. After this operation, 2.030 kB of additional disk space will be used. Do you want to continue [Y/n]? y Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main kmod amd64 9-3 [60,5 kB] ( ) Fetched 24,1 MB in 11s (2.120 kB/s) Download complete and in download only mode $ sudo dpkg --install /var/cache/apt/archives/*.deb (Reading database ... 17098 files and directories currently installed.) Preparing to replace adduser 3.113+nmu3 (using .../adduser_3.113+nmu3_all.deb) ... Unpacking replacement adduser ... ( ) Errors were encountered while processing: /var/cache/apt/archives/bash_4.2+dfsg-0.1_amd64.deb /var/cache/apt/archives/coreutils_8.13-3.5_amd64.deb initscripts lvm2 procps rsyslog sysvinit sysv-rc udev util-linux aide-common dmsetup e2fsprogs ifupdown initramfs-tools libdevmapper1.02.1:amd64 libdevmapper-event1.02.1:amd64 libparted0debian1:amd64 molly-guard openssh-server parted grub-common grub-pc grub-pc-bin grub2-common $ sudo dpkg --install /var/cache/apt/archives/bash_4.2+dfsg-0.1_amd64.deb /var/cache/apt/archives/coreutils_8.13-3.5_amd64.deb (Reading database ... 17129 files and directories currently installed.) Preparing to replace bash 4.2+dfsg-0.1 (using .../bash_4.2+dfsg-0.1_amd64.deb) ... Unpacking replacement bash ... Preparing to replace coreutils 8.13-3.5 (using .../coreutils_8.13-3.5_amd64.deb) ... Unpacking replacement coreutils ... Setting up bash (4.2+dfsg-0.1) ... update-alternatives: using /usr/share/man/man7/bash-builtins.7.gz to provide /usr/share/man/man7/builtins.7.gz (builtins.7.gz) in auto mode Setting up coreutils (8.13-3.5) ... $ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. The following extra packages will be installed: sysvinit-utils Suggested packages: bootlogd sash The following packages will be REMOVED: libslang2-modules:i386 sysvinit-utils:i386 The following NEW packages will be installed: sysvinit-utils WARNING: The following essential packages will be removed. This should NOT be done unless you know exactly what you are doing! sysvinit-utils:i386 0 upgraded, 1 newly installed, 2 to remove and 0 not upgraded. 23 not fully installed or removed. Need to get 0 B/99,5 kB of archives. After this operation, 328 kB disk space will be freed. You are about to do something potentially harmful. To continue type in the phrase Yes, do as I say! ?] Yes, do as I say! (actually type this!) dpkg: warning: overriding problem because --force enabled: This is an essential package - it should not be removed. (Reading database ... 17129 files and directories currently installed.) Removing sysvinit-utils ... Selecting previously unselected package sysvinit-utils. (Reading database ... 17105 files and directories currently installed.) Unpacking sysvinit-utils (from .../sysvinit-utils_2.88dsf-41_amd64.deb) ... Setting up sysvinit-utils (2.88dsf-41) ... Setting up sysv-rc (2.88dsf-41) ... Setting up initscripts (2.88dsf-41) ... Setting up util-linux (2.20.1-5.3) ... Setting up e2fsprogs (1.42.5-1.1) ... Setting up sysvinit (2.88dsf-41) ... sysvinit: restarting... done. (Reading database ... 17129 files and directories currently installed.) Removing libslang2-modules:i386 ... Setting up ifupdown (0.7.8) ... Setting up procps (1:3.3.3-3) ... [ ok ] Setting kernel variables ... /etc/sysctl.conf...done. Setting up udev (175-7.2) ... [ ok ] Stopping the hotplug events dispatcher: udevd. [ ok ] Starting the hotplug events dispatcher: udevd. update-initramfs: deferring update (trigger activated) Setting up rsyslog (5.8.11-3) ... [ ok ] Stopping enhanced syslogd: rsyslogd. [ ok ] Starting enhanced syslogd: rsyslogd. Setting up aide-common (0.15.1-8) ... Setting up initramfs-tools (0.109.1) ... update-initramfs: deferring update (trigger activated) Setting up openssh-server (1:6.0p1-4) ... [ ok ] Restarting OpenBSD Secure Shell server: sshd. Setting up molly-guard (0.4.5-1) ... Setting up libdevmapper1.02.1:amd64 (2:1.02.74-7) ... Setting up libdevmapper-event1.02.1:amd64 (2:1.02.74-7) ... Setting up libparted0debian1:amd64 (2.3-12) ... Setting up grub-common (1.99-27+deb7u1) ... Setting up grub2-common (1.99-27+deb7u1) ... Setting up grub-pc-bin (1.99-27+deb7u1) ... Setting up grub-pc (1.99-27+deb7u1) ... Installation finished. No error reported. Generating grub.cfg ... Found linux image: /boot/vmlinuz-3.9.4-zgsrv2008064 Found initrd image: /boot/initrd.img-3.9.4-zgsrv2008064 done Setting up parted (2.3-12) ... Setting up dmsetup (2:1.02.74-7) ... update-initramfs: deferring update (trigger activated) Setting up lvm2 (2.02.95-7) ... [ ok ] Setting up LVM Volume Groups...done. update-initramfs: deferring update (trigger activated) Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-3.9.4-zgsrv2008064 Found 12 processes using old versions of upgraded files ( ) $ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libreadline5:i386 Use apt-get autoremove to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. $ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 72 $ dpkg --list grep amd64 wc -l 175 $
To finally make the packages really vanish from the system, we purge all packages that dpkg --list still reports in state rc , which means removed, configuration still present . Thankfully, dpkg seems to handle the case where a dpkg-conffile belongs to a transitioned package gracefully.$ sudo apt-get remove $(dpkg --list awk if( $1 == ii && $4 == i386 ) print $2 ) Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: e2fslibs:i386 gcc-4.7-base:i386 libacl1:i386 libapt-inst1.5:i386 libapt-pkg4.12:i386 libasprintf0c2:i386 libattr1:i386 libblkid1:i386 libbsd0:i386 libbz2-1.0:i386 libc6:i386 libcap2:i386 libcomerr2:i386 libcurl3:i386 libcurl3-gnutls:i386 libdb5.1:i386 libedit2:i386 libexpat1:i386 libfreetype6:i386 libfuse2:i386 libgcc1:i386 libgcrypt11:i386 libgdbm3:i386 libgnutls26:i386 libgpg-error0:i386 libgpm2:i386 libgssapi-krb5-2:i386 libidn11:i386 libk5crypto3:i386 libkeyutils1:i386 libkmod2:i386 libkrb5-3:i386 libkrb5support0:i386 libldap-2.4-2:i386 liblockfile1:i386 liblzma5:i386 libmagic1:i386 libncurses5:i386 libncursesw5:i386 libp11-kit0:i386 libpam-modules:i386 libpam0g:i386 libpci3:i386 libpcre3:i386 libpng12-0:i386 libpopt0:i386 libprocps0:i386 libreadline5:i386 libreadline6:i386 librtmp0:i386 libsasl2-2:i386 libselinux1:i386 libsemanage1:i386 libsepol1:i386 libsigc++-2.0-0c2a:i386 libslang2:i386 libsqlite3-0:i386 libss2:i386 libssh2-1:i386 libssl1.0.0:i386 libstdc++6:i386 libtasn1-3:i386 libtinfo5:i386 libudev0:i386 libusb-0.1-4:i386 libustr-1.0-1:i386 libuuid1:i386 libwrap0:i386 linux-image-3.9.4-zgsrv2008064:i386 zlib1g:i386 0 upgraded, 0 newly installed, 70 to remove and 0 not upgraded. After this operation, 67,5 MB disk space will be freed. Do you want to continue [Y/n]? y (Reading database ... 17111 files and directories currently installed.) Removing e2fslibs:i386 ... ( ) $ sudo apt-get remove $(dpkg --list awk if( $1 == ii && $4 == i386 ) print $2 ) Reading package lists... Done Building dependency tree Reading state information... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. $ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 70 $ dpkg --list grep amd64 wc -l 175 $
We now verify that all packages we have are either arch all or arch amd64.$ sudo dpkg --purge $(dpkg --list awk if ($1 == rc ) print $2 ) (Reading database ... 15868 files and directories currently installed.) Removing e2fslibs:i386 ... Purging configuration files for e2fslibs:i386 ... ( ) $ dpkg --print-architecture amd64 $ dpkg --print-foreign-architectures i386 $ dpkg --list grep i386 wc -l 0 $ dpkg --list grep amd64 wc -l 175 $
As I found myself suddenly without e2fsprogs during one of my experiments, I check for e2fsprogs being present. If this package is missing, you ll only find out after rebooting.$ dpkg --list grep -v \(amd64 all\) Desired=Unknown/Install/Remove/Purge/Hold Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) / Name Version Architecture Description +++-===============================-=========================-============-========================================================================
My locally built amd64 kernel with 32 bit support was an arch i386 package as well, so it got zapped in the migration process. Replace it with a real amd64 kernel. This step will vary if you use Debian kernels. Do not forget the firmwares.$ dpkg --list e2fsprogs Desired=Unknown/Install/Remove/Purge/Hold Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) / Name Version Architecture Description +++-==============-============-============-================================= ii e2fsprogs 1.42.5-1.1 amd64 ext2/ext3/ext4 file system utilit
I now make sure that the initrd I have built contains all file system modules and is of the right architecture:$ sudo apt-get -t wheezy-zg-experimental install kernel-image-zgserver linux-firmware-image Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: cpio initramfs-tools kernel-image-zgsrv20080 klibc-utils libklibc linux-image-3.9.4-zgsrv20080 Suggested packages: libarchive1 The following NEW packages will be installed: cpio initramfs-tools kernel-image-zgserver kernel-image-zgsrv20080 klibc-utils libklibc linux-firmware-image linux-image-3.9.4-zgsrv20080 0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded. Need to get 11,2 MB of archives. After this operation, 30,5 MB of additional disk space will be used. Do you want to continue [Y/n]? Get:1 http://zg20110.debian.zugschlus.de/ wheezy-zg-experimental/main linux-image-3.9.4-zgsrv20080 amd64 3.9.4.20130605.0 [10,4 MB] ( )
As the last step, remove i386 as an architecture and reboot into the newly migrated system. We verify that the system considers itself amd64 with no foreign architectures.$ cd /tmp $ < /boot/initrd.img-3.9.4-zgsrv20080 gunzip cpio -i 32053 blocks $ find . -name *ext*.ko ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext3/ext3.ko ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext2/ext2.ko ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko $ file ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), BuildID[sha1]=0x259a642448805c9597932ee25c62c8e6a00a32ad, not stripped $ cd
We now check whether we have lost any packages in this process:$ sudo dpkg --remove-architecture i386 $ dpkg --print-architecture amd64 $ $ dpkg --print-foreign-architectures $ dpkg --list grep i386 wc -l 0 $ dpkg --list grep amd64 wc -l 181 $ sudo shutdown -r now Broadcast message from root@wheezyarch (pts/0) (Wed Jun 12 16:09:24 2013): The system is going down for reboot NOW! $ Connection to wheezyarch.zugschlus.de closed by remote host. Connection to wheezyarch.zugschlus.de closed. $ ssh wheezyarch.zugschlus.de Linux wheezyarch 3.9.4-zgsrv20080 #2 SMP PREEMPT Wed Jun 5 13:12:04 UTC 2013 x86_64 $ dpkg --print-architecture amd64 $
So we have not actually lost things, but we have gained the pure amd64 kernel and the packages that are needed to build a proper initrd. These were missing from the initial install of the i386 test VM. The downside of the process is that we have lost most of the automatically installed markers in aptitude which make sure to remove a library once nothing depends on it any more. If you make a point in having those markers correctly set, as I do, this is tedious work to set it on every package in the aptitude curses interface, making aptitude suggest removing the entire system, and then again remove it from the packages you actually intend to keep. This should not result in anything being removed, otherwise you have goofed previously. All Done! I am now off perfecting my upgrade process so that the i386 lenny and squeeze machines can first be moved to wheezy and then to amd64.$ sudo dpkg --get-selections grep install awk print $1 sed s/:.*//;s/$/:amd64 install/; > get-selections.post $ wc -l get-selections.* 234 get-selections.post 226 get-selections.pre $ diff --unified=0 get-selections.pre get-selections.post --- get-selections.pre 2013-06-12 19:47:40.278176000 +0200 +++ get-selections.post 2013-06-12 20:28:51.389874000 +0200 @@ -22,0 +23 @@ +cpio:amd64 install @@ -58,0 +60 @@ +initramfs-tools:amd64 install @@ -68,0 +71,3 @@ +kernel-image-zgserver:amd64 install +kernel-image-zgsrv20080:amd64 install +klibc-utils:amd64 install @@ -108,0 +114 @@ +libklibc:amd64 install @@ -163,0 +170,2 @@ +linux-firmware-image:amd64 install +linux-image-3.9.4-zgsrv20080:amd64 install
So I'd previously talked about migrating machines. From having one virtual machine running "mail" + "web" + "stuff" I've now got three hosts:
The left column, Urgent items, typically must be executed by a certain date (e.g. buying a gift before a birthday or installing a new SSL certificate before the old one expires). A security update or Acrobat reader update does not have this same characteristic. Under this model:
| Urgent | Non-urgent | |
|---|---|---|
| Important | Replace SSL certificate Buy birthday gift |
Security update Run backup job |
| Not important | Register for conference before deadline for free gift | Non-security update for Acrobat reader |
echo options i915 invert_brightness=1 tee /etc/modprobe.d/i915.conf update-initramfs -u -k allSince March 2012 there is a mechanism in the Linux kernel to tell the i915 driver which hardware have this problem, and get the driver to invert the brightness setting automatically. To use it, one need to add a row in the intel_quirks array in the driver source drivers/gpu/drm/i915/intel_display.c (look for "static struct intel_quirk intel_quirks"), specifying the PCI device number (vendor number 8086 is assumed) and subdevice vendor and device number. My Packard Bell EasyNote LV got this output from lspci -vvnn for the video card in question:
00:02.0 VGA compatible controller [0300]: Intel Corporation \
3rd Gen Core processor Graphics Controller [8086:0156] \
(rev 09) (prog-if 00 [VGA controller])
Subsystem: Acer Incorporated [ALI] Device [1025:0688]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- \
ParErr- Stepping- SE RR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- \
<tabort-><mabort->SERR- <perr- intx-="INTx-" latency:="Latency:"><unassigned> [disabled]
Capabilities: <access denied="denied">
Kernel driver in use: i915
The resulting intel_quirks entry would then look like this:
struct intel_quirk intel_quirks[] =
...
/* Packard Bell EasyNote LV11HC needs invert brightness quirk */
0x0156, 0x1025, 0x0688, quirk_invert_brightness ,
...
According to the kernel module instructions (as seen using
modinfo i915), information about hardware needing the
invert_brightness flag should be sent to the
dri-devel
(at) lists.freedesktop.org mailing list to reach the kernel
developers. But my email about the laptop sent 2013-06-03 have not
yet shown up in
the
web archive for the mailing list, so I suspect they do not accept
emails from non-subscribers. Because of this, I sent my patch also to
the Debian bug tracking system instead as
BTS report #710938, to make
sure the patch is not lost.
Unfortunately, it is not enough to fix the kernel to get Laptops
with this problem working properly with Linux. If you use Gnome, your
worries should be over at this point. But if you use KDE, there is
something in KDE ignoring the invert_brightness setting and turning on
the screen during login. I've reported it to Debian as
BTS report #711237, and
have no idea yet how to figure out exactly what subsystem is doing
this. Perhaps you can help? Perhaps you know what the Gnome
developers did to handle this, and this can give a clue to the KDE
developers? Or you know where in KDE the screen brightness is changed
during login? If so, please update the BTS report (or get in touch if
you do not know how to update BTS).
Over my time working with Debian packages, I've always been concerned that
I have been missing catchable mistakes by not running all the static checking
tools I could run. As a result, I've been interested in writing some code that
automates this process, a place where I can push a package and come back a few
hours later to check on the results. This is great, since it provides a slightly
less scary interface to new packagers, and helps avoid thinking they've
just been told off by a Developer.
I've spent the time to actually write this code, and I've called it
debuild.me. The code itself is in its fourth
iteration, and is built up from a few core components. The client / server code
(lucy and
ethel) are quite interconnected, but
firehose works great
on it's own, and is a single, unified (and sane!) spec that is easy
to hack with (or even on!). Hopefully, this means that our wrappers will be
usable outside of debuild.me, which is a win for everyone.
Backend Design
The backend (lucy) was the first part
I wanted to design. I made the decision (very early on) that everything was
going to be 100% Python 3.3+. This lets me use some of the (frankly, sweet)
tools in the stdlib. Since I've written this type of thing before
(i've tried to write this tool many,
many,
many,
many times before), so I had a rough
sense of how I wanted to design the backend. Past iterations had suffered from
and overly complex server half, so I decided to go ultra minimal with the
design of debuild.me.
<aside class="left">
You can find the code for the server (lucy) on
my GitHub
</aside>
The backend watches a directory (using a simple inotify script) and processes
.changes files as they come in. If the package is a source package, a set of
jobs are triggered (such as lintian, build and desktop-file-validate),
as well as a different set for binary packages (such as lintian, piuparts
and adequite). Only people may upload source packages (without any debs) and
only builders can upload binary packages (without source).
The client and server talk using
XML-RPC with BASIC HTTP
auth. I'm going to (eventually) SSL secure the transport layer, but for now,
this will work as a proof of concept.
Since I tend to like to keep my codebase simple and straightforward, I've used
MongoDB as Lucy's DB. This lets me move between
documents in Mongo to Python objects without any trouble. In addition, I
evaluated some of the queue code out there (ZMQ, etc), and they all seemed
like overkill for my problem, and had a hard time keeping track of jobs that
(must never!) get lost. As a result, I wrote my own (very simple) job queue
in Mongo, which has no sense of scheduling (at all), but can do it's job (and
do it well).
Jobs describe what's to be built with a link to the package document
that the job relates to, and it's arch and suite (don't worry about the
rest just yet). Jobs get assigned via natural sort on it's UUID based _id,
and assigned to the first builder that can process it's arch / suite.
Source packages are considered arch:all / suite:unstable (so they always
get the most up-to-date linters on any arch that comes along).
Lucy also allows for uploads to be given an X-Lucy-Group tag to manage which
set of packages they're a part of. This comes in handy for doing partial
archive rebuilds, or eventually using it to manage what jobs should be run
on which uploads. This will allow me to run much more time-consuming tools
for packages I want to review versus rebuilding to ensure packages don't
FTBFS or aren't adequite.
Client Design
The buildd client (ethel) talks with lucy
via XML-RPC to get assigned new jobs, release old jobs, close finished jobs,
and upload package report data. When the etheld requests a new job, it also
passes along what suites it knows of, which arches it can build, as well
as what types it can run (stuff like lintian, build or cppcheck.) Lucy
then assigns the builder to that job (so that we don't allocate the same job
twice), and what time it was assigned at.
<aside class="right">
You can find the code for the client (ethel) on
my GitHub
</aside>
Ethel then takes the result of the job (in the form of a firehose.model tree)
and transmits it over the line back to the Lucy server as a report (which also
contains information on if the build failed or not), at which point
lucy hands back a location (on the lucy host) that the daemon can write the log
to.
If the job was a binary build, the etheld process will dput the package to
the server, with a special X-Lucy-Job tag to signal which job that build
relates to, so that future lint runs can fetch the deb files that the build
produced.
Tooling
Ethel runs a set of static checkers on the source code, which are basically
fancy wrappers around the tools we all know and love (like
lintian,
desktop-file-validate,
or piuparts) which output Firehose in place of
home-grown stdout. This allows us to programmatically deal with the output
of these tools in a normal and consistent way.
<aside class="left">
You can read more about Firehose over in the Firehose
README.rst
</aside>
Some of the more complex runners are made of 3 parts - a runner, wrapper
and command. The server invokes the command routine, which invokes the
runner (the command just provides a unified interface to all the runners),
who's output gets parsed by the wrapper to turn it into a Firehose model
tree.
The goal here is that tons of very quick-running tools get run over a
distributed network, and machine-readable reports get filed in a central
location to aid in reviewing packages.
Ricky
In addition to the actual code to run builds, I've worked on a few tools to
aid with using debuild.me for my DD related life. I have some uncommon
use-cases that are nice to support. One such use-case is the ability to rebuild
packages from the archive (unmodified) to check that they rebuild OK against
the target. This is handy for things like arch:all packages that get
uploaded (since they never get rebuilt on the buildd machines, and FTBFSs are
sadly common) or packages that have had a Build-Dependency change on them.
Ricky is able to create a .dsc url to your friendly local mirror, and fetch
that exact version of the package. Ricky can then also use the .dsc (in a
monumental hack) to forge a package_version_source.changes file, and sign
it with an autobuild key and upload it to the debuild.me instance. Since it
can also modify the .changes's target distribution, you can also use this to
test if a package will build on stable or testing, unmodified.
Fred
Fred is a wrapper around Ricky, to help with fetching packages that may not
exist yet. Fred also contains an email scraper to read off such lists as
debian-devel-changes, and
add an entry to fetch that upload when it becomes available on the local
mirror, pass it to ricky, and allow debuild.me to rebuild new packages
that match a set of criteria.
I'm currently playing around with the idea of rebuilding all incoming
Python packages to ensure they don't FTBFS in a clean chroot.
Loofah
Loofah is also another wrapper around Ricky, but for use manually. Loofah
is able to sync down the apt Sources list, and place it in Mongo for fast
queries. This than allows me to manually run rebuilds on any Source package
that fits a set of critera (written in the form of a Mongo query), which get
pulled and uploaded by Ricky.
An example script to rebuild any packages that Build-Depend on
python3-all-dev in Debian unstable / main would look like:
<aside class="right">
You can find more queries in the Loofah
examples
</aside>
[
"version": "unstable", "suite": "main" ,
"Build-Depends": "python3-all-dev"
]
[
,
"$or": [ "Build-Depends": "cdbs" ,
"Build-Depends-Indep": "cdbs" ]
]
Sources.gz file to query off of (
including Maintainer!)
Future Work
The future work on debuild.me will be centered around making it easier for
buildd nodes to be added to the network, with more and more automation in that
process (likely in the form of debs). I also want to add better control over
the jobs, so that packages I upload only go to my personal servers.
I'd also very much like to get better EC2 / Virtualization support integrated
into the network, so that the buildd count grows with the queue size. This
is a slightly hard problem that I'm keen to fix.
I'm also considering moving the log parsing code out of the workers, so that
the parsing code can be fixed without upgrading all the workers. This would also
drop the Firehose dep on the client code, which would be nice.
Migration from a debuild.me build into a local reprepro repo is something
that would be fairly easy to do as well, likely to be done remotely via
the XML-RPC interface, which calls a couple of reprepro commands (such as
includedsc and includedeb) and publishes it to the user's repo. This is
a nice use of the debs that get built, and could also allow debuild.me to be
used like a PPA system, but this allows the user to not migrate packages
that may contain piuparts issues.
systemd has too many dependencies, or it is bloated, or it does too many things, or is too complexNow this concern actually has a lot of different facets, and I am trying to share my opinion on each of them. systemd has too many dependencies First, let s start with too many dependencies , because that is easy to check and reason about. I have created a document which lists all dependencies of the systemd binary itself (pid 1) and all the binaries which are currently shipped by the systemd Debian package. If you don t want to take my word for granted, please read that document. Have you read the document? Very nice! As you can see, the systemd binary itself has 10 dependencies (excluding libc). Now, the question is, what is bad about dependencies? Why do people list dependencies as a top concern?
enforce_user
which uses libcap. You ll notice that the code is not complex and usage of
the libraries is clear.
Software bloat is a process whereby successive versions of a computer program become perceptibly slower, use more memory or processing power, or have higher hardware requirements than the previous version whilst making only dubious user-perceptible improvements.The first part of the definition certainly does not match systemd it is measurably faster than sysvinit. As for memory usage: systemd s RSS is 1.8 MiB, whereas sysvinit uses 0.8 MiB. As I argued on the More memory use point in the dependencies section, I think the additional resource cost is well worth the benefits. Also note that systemd s features are NOT all implemented in the binary which is PID 1. As explained in the dependency list, systemd consists of many cleanly separated binaries. So if a new version of systemd gathers an additional feature, this does not mean that your PID 1 will be bigger. While systemd runs on any hardware, it has an indirect hardware requirement: it requires some Linux kernel features (which are all enabled in Debian kernels). That might rule out usage of systemd on really old embedded hardware where you don t have a chance to update the kernel. While it is sad that those machines cannot profit from systemd, switching to systemd as a default has no downside either: Debian continues to support sysvinit for quite some time, so these machines will continue to work even with upcoming Debian versions. systemd does too many things The Wikipedia definition continues:
[ ] perceived bloat can occur from the software servicing a large, diverse marketplace with many differing requirements. Most end users will feel they only need some limited subset of the available functions and will regard the others as unnecessary bloat, even if people with different requirements do use them.I think the last part of the Wikipedia definition applies to systemd: it does service a large and diverse marketplace . That marketplace is the entirety of existing software which is started by an init system. Also, systemd can be used on a wide range of hardware (embedded devices, tablets, phones, notebooks, desktops, servers) which requires different features. As an example: on a desktop system you typically don t care strongly about a watchdog feature, but on embedded or servers that feature is very handy. Similarly, on a tablet, forward secure sealing of logfiles is not as important as on a server. Therefore, I can understand if you feel that you don t need many of the features systemd provides. But please think of other users and maintainers who are very happy with systemd s benefits. Also note that while systemd supports many things (in separate binaries!), you don t have to use them all. It still makes sense to ship them all in the same package. Take coreutils as another example in that area. The binaries belong together, even though you most likely haven t used all of them (e.g. od, pr, ptx, :-)). systemd is too complex The remaining concern is that systemd is too complex. In my experience, complexity is often inherent to a specific area and one cannot simply make it go away. Instead, there are different models of how that complexity is represented. Think of the monolithic Linux kernel versus the MINIX microkernel. The latter has a very small amount of lines of source in the kernel, but puts the complexity into userspace. The former uses a different approach with more source in the kernel. The arguments between both camps show that neither is clearly right or clearly wrong. In a way, sysvinit represents the MINIX model: it has a small core (the init binary itself), but a lot of complexity in shell scripts and external programs. The fact that solutions are copied from one init script to another leads to lots of subtle errors and makes code reuse really hard. systemd however has more source code in the binaries, but requires only very simple, descriptive, textual configuration instead of complex init scripts. To me, it seems preferable to have the complexity in a single place instead of distributed across lots of people and projects. Conclusion In a way, you are right. systemd centralizes complexity from tons of init scripts into a single place. However, it therefore makes it very easy for maintainers to write service files (equivalent of an init script) and provides a consistent and reliable interface for service management. Furthermore, it is different than sysvinit, and different solutions often seem complex at first. While systemd consumes more resources than sysvinit, it uses them to make more information available about services; its finer-grained service management requires more state-keeping, but in turn offers you more control over your services.
Australia's close ties with the US
Australia has had particularly close ties with the US for many years and just as the British GCHQ has been linked to PRISM, it seems Australia's ASIO is also in on the act. For those not familiar with the depth of this alliance, it is worth checking up on Pine Gap, the ECHELON program and the recent exposure (by Wikileaks) of one of our senators with unauthorised US spy connections - he quietly resigned for family reasons'.
When I was preparing my blog entry about the Gold Standard in Free communications, I had absolutely no idea that The Guardian (another Ganglia user) would be hot on my heels with dramatic revelations about US Government surveillance of dangerous terrorists and maybe sucking up a little bit of data about a few hundred million of their own citizens and another 90% of the world's population for good measure.
Some people even thought I've been a bit paranoid with my concerns about excessive surveillance. However, it is just remarkable to see that in the same week that the trial of Bradley Manning is getting under way for inappropriate use of his employer's computer, the US has been exposed plotting cyber attacks and setting a very bad example for all those little script kiddies out there.
Practical questions for every one of us
Is it time to start blocking email to and from sites like gmail and hotmail?
What about the reports that the US Government was engineering back doors in the OpenBSD operating system? Have any open source projects actually been comprised in this way?
Will spammers and other criminals take this as a cue that there is nothing morally wrong with hacking?
Have certificate authorities been infiltrated too? They may well be the elephant in the room - while everybody was joking about the NSA key hidden in the depths of Microsoft Windows, maybe one or more of the well known trusted root certificates, right under our noses, is also a back door?
The danger is real
Anybody wondering about the practical implications of all this data gathering doesn't have to look very far to find out what can go wrong. In the same week as all these things were exposed, there have been more dramatic revelations about law enforcement officers selling private data for their own commercial gain. While the vast majority of police are surely good citizens, every organisation has it's bad apples and as Bradley Manning demonstrated so well, it only takes one person to breach security and enormous volumes of data can end up escaping.
For email extensions such as SPF, DKIM and DMARC, I think the most flexible and portable system is the milter protocol. Originally developed for Sendmail, it is now also supported by Postfix, and it allows to plug specific filters in the mail server without the hassle of the previous systems like SMTP proxies.
I ll be presenting Authoritative linked data descriptions of debian source packages using ADMS.SW at OSS 2013.
Here s the abstract :
The Debian Package Tracking System is a Web dashboard for Debian contributors and advanced users. This central tool publishes the status of subsequent releases of source packages in the Debian distribution. It has been improved to generate RDF meta-data documenting the source packages, their releases and links to other packaging artifacts, using the ADMS.SW 1.0 model. This constitutes an authoritative source of machine-readable Debian facts and proposes a reference URI naming scheme for Linked Data resources about Debian packages. This should enable the interlinking of these Debian package descriptions with other ADMS.SW or DOAP descriptions of FLOSS projects available on the Semantic Web also using Linked Data principles. This will be particularly interesting for traceability with upstream projects whose releases are packaged in Debian, derivative distributions reusing Debian source packages, or with other FLOSS distributions.If you are interested, a previous paper which describes the same experiment is available online. See also previous installments on ADMS.SW in this blog.
As part of writing a new mail client I'm wondering about how to change my email-life, and how other people process/handle their incoming email.
I sort my incoming email into folders at delivery-time using procmail. Mail is generally filtered into mailboxes on the basis of the company that sent it, the person that sent it, or the machine which generated it.
Because I manage a lot of machines personally I've split things up so that I have a folder per host. So on a morning I might have unread mail in the following folders:
machines.steve.org.uk/ machines.da-db1/ machines.da-db2/ machines.da-web1/ machines.da-web2/ machines.da-web3/ machines.da-web4/The per-machine mailboxes usually contain a single mail every day from LogWatch, along with output from any cron-jobs. For example today I received the mail:
From: Cron Daemon To: steve@steve.org.uk Subject: Cron steve@steve.org.uk /home/steve/bin/download-check URL http://nodejs.org/ - no longer matches v0.10.9Generally speaking I don't need to read the per-machine messages. I'll keep the most recent 100 for reference, but only need to look if something seems "off" on a machine. But if I don't look I'd not see the node upgrade notice, so find that I do read them after all. This suggest to me that email isn't the right way to handle this kind of thing. Instead I should use a notification system - at work we have a central service called MauveAlert (yes, Red Dwarf reference). Mauve receives "alerts" of various kinds, via UDP. The alerts are then fanned out to appropriate people via XMPP, Email, or SMSs. I have a similarly-inspired system I use on my Debian Administration cluster. A (node) service runs non-stop collecting UDP messages and showing them on a dashboard. I look at it throughout the day to see when slaughter runs, etc. Anyway in conslusion I get a lot of mail. Some of it is related to random projects, and all ends up in the steve.org.uk/ mailbox, some of it relates to machines, and gets filed away, and I have regular conversions with folk so I have a .people.kirsi/ folder which receives a lot of attention, for example. ObRandom:daily() - Mark ~/Maildir/.machines.*, etc, read.
Next.