Search Results: "xam"

16 August 2013

Justus Winter: All the important bits are there - please test and review :)

Finally, more bootlog-pr0n:
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
/dev/hd0s1: clean, 44680/181056 files, 292234/723200 blocks
done.
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Cleaning up temporary files... /tmp.
Mounting local filesystems...done.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 8
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 34744 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
unexpected ACK from keyboard
GNU 0.3 (debian) (console)
login: root
root@debian:~# shutdown -h now
Broadcast message from root@debian (console) (Fri Aug 16 20:02:47 2013):
The system is going down for system halt NOW!
INIT: Switching to runlevel: 0root@debian:~#
INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
Using makefile-style concurrent boot in runlevel 0.
Stopping deferred execution scheduler: atd.
task c10f72a8 deallocating an invalid port 2098928, most probably a bug.
Asking all remaining processes to terminate...done.
All processes ended within 1 seconds...done.
Stopping enhanced syslogd: rsyslogd.
Deconfiguring network interfaces...ifdown: failed to open pid file /run/network/ifdown-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
/bin/sh: 1: ifconfig: not found
done.
Deactivating swap...swapoff: /dev/hd0s5: 177152k swap space
done.
Unmounting weak filesystems...umount: /etc/mtab: Warning: duplicate entry for device /dev/hd0s1 (/dev/cons)
umount: could not find entry for: /dev/cons
done.
Unmounting local filesystems...done.
mount: cannot remount /: Device or resource busy
Will now halt.
init: notifying pfinet of shutdown...init: notifying tmpfs swap of shutdown...init: notifying tmpfs swap of shutdown...init: notifying tmpfs swap of shutdown...init: notifying ext2fs device:hd0s1 of shutdown...init: halting Mach (flags 0x8)...
In tight loop: hit ctl-alt-del to reboot
(You might note that df complains about not being able to read the mtab file. That is because it has been built with _PATH_MOUNTED being /var/run/mtab. This will correct itself when the coreutils package is being rebuilt against a patched libc.) I spent my last two weeks with polishing my patch series, that meant a lot of package rebuilds and that means a lot of waiting (even more so on the Hurd, short version: fakeroot-hurd could be fast but is not yet working properly, fakeroot-tcp is slow) and thus some frustration ;) Also I had to pay special attention so that the upgraded packages could be installed without accidentally breaking anything in the process. Making sysvinit pid 1 is surprisingly tricky in this regard since it breaks the ABI and requires a libc fix that also worked with the current Hurd servers. Here are the patches: So I had some spare time on my hand while waiting for numerous package rebuilds and I took this as an opportunity to read papers about Mach and to familiarize myself with mig, the Mach Interface Generator. While I have used it in the past, I had not yet looked at its implementation. And I had to patch the exec server, and there was both code implementing a questionable feature (on-demand unzipping of binaries) and code that was not even compiled (courtesy of the preprocessor) and had probably bit-rot by now. So I figured I could spend my time doing some cleanups: I have rebuild all the necessary packages and uploaded them into an apt repository:
deb http://teythoon.cryptobitch.de/gsoc/heap/debian unstable main
Please use unstable for now. Also make sure that you have a recovery plan for your Debian/Hurd installation if anything goes wrong. For your convenience there's a seed tarball containing packages with the appropriate sources.list.d snippets and the repository key: https://teythoon.cryptobitch.de/gsoc/heap/debian/seed.tar If you want to switch to the new runsystem.sysv, do:
# update-alternatives --config runsystem
Whenever you switch runsystems, please use reboot-hurd to reboot the system. This is the most robust way. Known issues: Next week I will address the network related issues. By now they are the source of most of the error messages in the bootlog.

Richard Hartmann: Release Critical Bug report for Week 33

One more for DebConf. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9 39 (32+7) +39 (+32/+7)
16 release+10 20 (12+8) +20 (+12/+8)
17 release+11 24 (19+5) +24 (+19/+5)
18 release+12 2 (2+0) +2 (+2/+0)
Graphical overview of bug stats thanks to azhag:

15 August 2013

Bits from Debian: Happy 20th birthday Debian!

Today is Debian's 20 anniversary. This anniversary would have been impossible without a strong community of users and developers. That's why for its commemoration, we asked the Debian community what Debian meant to them. Below you have a selection of the answers we got.
Since I run Debian on my computers, I do not play anymore to 3D shooting games, not because of the lack of Free 3D drivers, but because developing Debian is more fun and addictive. Debian is a truly community based distro which is devoted to FOSS ideas and standards. It perfectly works on high variety of hardware. Users from all over the world have been using Debian and contributing to it during 20 years. And I am proud to be one of them. Happy Birthday, Debian! When I considered switching to Linux I asked friends which distribution to choose. I was told skip other distributions that were considered more newbie-friendly at the time and go straight to Debian instead. It might be more work initially, they said, but it would save me the hassle of switching to Debian later on, which I would inevitably do. Turns out they were right. I started my Linux experience more than ten years ago with Debian and have yet to see a better Linux distribution. [...] You are a worldwide community of volunteers working together for 20 years now. To me this is an encouraging example and, given the sorry state of the world we live in, more important than the technical quality of the operating system. I wish you another successful twenty years and to stay as independent as you are. I initially started in Debian as it was an interesting technical challenge. Over the years the community and having good standards around what Free Software is has become more important. As a user of Debian for 14 years, and a former Debian Developer for 10, I would like to wish a happy birthday to the best project on the internet, and the best Linux distribution ever. Thanks for all of your support over the years! "Rock Solid Stability and Absolute Freedom" Thats what Debian means to me. Debian gives me a feeling that I am using the best linux has to offer. You know your machine is in safe hands. As a long time Debian user and sometimes supporter [...] I wish Debian all the best for next 20 years and beyond! Debian is the universal operating system. And it's free (and not just as in beer ;-) Debian is an awesome combination of obsessive high quality software and software freedom. It's a pleasure to be able to use and contribute to this project. Thanks to all for their excellent work! Cheers to 2**20 years more! [...] I am so very thankful to all of the people who have contributed to and continue to contribute to such a wonderful ecosystem of tools. I love the commitment to the long term goals of security, freedom and transparency with respect to the computer systems that we all use and rely upon and the information that we store on them. Debian is the operating system that makes me free. Debian is a family gathered around great idea. Its pure love.

Daniel Pocock: Free software for free markets

Co-inciding with the first DebConf in Switzerland, one of the world's leading financial centers, the first official packaging of open messaging and market data distribution framework OpenMAMA for a Linux distribution has just been uploaded. The packages, along with the Avis low-latency event-router middleware/transport were uploaded to the Debian unstable catalog this week and will soon be available conveniently to install with apt-get. OpenMAMA was developed by NYSE Technologies and includes code originally developed under their previous brand Wombat Financial Software. The project was recently placed under the strategic umbrella of the Linux Foundation. Packages for Fedora and EPEL The initial packaging effort has identified a number of build system bugs, patches have been contributed upstream. The resolution of these issues and the experience gained with the Debian package will make it relatively easy to package for RPM-based platforms in the near future. Not just for finance While it is clear that OpenMAMA's designers had financial services in mind (and the low-latency, high throughput transport of market price data in particular), the solution is generalised and likely to be useful in other domains where asynchronous messaging and inter-process communication is required. In fact, this distinction is made particularly clear by the division of OpenMAMA into two subprojects:
  • OpenMAMA itself is a general-purpose programming API for asynchronous messaging. It hides the details of various low-level transports, including the open source Avis event router and popular proprietary solutions like Tibco Rendezvous. Java programmers have had such a generalised API for many years (JMS), whereas OpenMAMA brings a similar paradigm to the C/C++ developer.
  • OpenMAMDA is a further layer on top of OpenMAMA designed to encapsulate market data. It competes directly with proprietary solutions such as those from Reuters and Bloomberg. OpenMAMDA comes with specific APIs for news, order books and options and could be extended for other markets.
Commercial background Reuters and Bloomberg are well known as two of the main vendors in the market data industry. Both of them (and other competitors) provide a range of data services and related software. Buying a data feed doesn't automatically give access to the software from the same franchise: the software and APIs are promoted as a valuable product in their own right. It is actually quite possible (and not unheard of) for an institution to run a Reuters Market Data System (RMDS) software stack fed by a data feed from Bloomberg or vice-versa. When an IT department or third-party vendor of trading systems wants to transport data between applications (or between multiple users of a single application) they will not typically do it themselves and one of these APIs is usually chosen. OpenMAMA appears to be a strong contender for such use cases. Bloomberg has recently made their API public, although OpenMAMA goes much further by making their source code public too. Development samples with the OpenMAMA API Programming the OpenMAMA API appears to be relatively consistent with the event-loop model that many developers are already comfortable with. For a very fast start, two compelling examples are the basic publisher and basic subscriber. Here is a sample of the code from the basic subscriber, demonstrating how to start the event loop and receive event notifications in a callback class:
static void createSubscriber (void)
 
    try
     
        gSubscription = new MamaBasicSubscription;
        gSubscription->createBasic (gTransport,
                                    gDefaultQueue,
                                    new SubscriptionCallback (),
                                    gTopic);
        gSubscription->setDebugLevel (gSubscLogLevel);
     
    catch (MamaStatus &status)
     
        cerr << "Error creating subscription: " 
             << status.toString () << endl;
        exit (1);
     
 
and here is some basic code that processes messages asynchronously:
class SubscriptionCallback : public MamaBasicSubscriptionCallback
 
public:
    virtual ~SubscriptionCallback ()  
    virtual void onCreate (MamaBasicSubscription*  subscription)
     
        if (gQuietLevel < 2)
         
            printf ("Created inbound subscription.\n");
         
     
    virtual void onError (MamaBasicSubscription*  subscription,
                          const MamaStatus&       status,
                          const char*             subject) 
     
        printf ("Error creating subscription: %s\n", 
                status.toString ());
        exit (1);
     
    virtual void onMsg (MamaBasicSubscription* subscription, 
                        MamaMsg&               msg)
     
        if (gQuietLevel < 2)
         
            printf ("Recieved msg.\n");
         
        DisplayIterator it;
        msg.iterateFields (it, NULL, NULL);
     
    //JG: need to add in onDestroy as this is abstract and must be implemented
    virtual void onDestroy (MamaBasicSubscription*  subscription, void * closure)
     
        if (gQuietLevel < 2)
         
            printf ("Destroyed subscription.\n");
         
     
 ;
Practical uses outside financial services Some people may feel that having a low-latency market-data system is overkill for anybody not running a trading desk. Not so. OpenMAMA may well prove useful and convenient as a one-stop-shop API for multi-currency accounting software (Postbooks, GnuCash, etc), for multi-currency web sites (e.g. UberCart) and many other applications.

Ingo Juergensmann: Exim4 and TLS with GMX/Web.de

Due to the unveiling of the NSA surveillance by Edward Snowden, some German mail providers decided last week to use TLS when sending mails out. For example GMX and Web.de. Usually there shouldn't be a problem with that, but it seems as if the Debian package of Exim4 (exim4-daemon-heavy) doesn't support any TLS ciphers that those providers will accept. The Debian package uses GnuTLS for TLS and there is Bug #446036 that asks for compilation against OpenSSL instead. Anyway, maybe it's something in my config as I don't use the Debian config but my own /etc/exim4/exim4.conf. Here are the TLS related parts:
tls_advertise_hosts = *
tls_certificate = /etc/exim4/ssl.crt/webmail-ssl.crt
tls_privatekey = /etc/exim4/ssl.key/webmail-server.key
That's my basic setup. After discovering that GMX and Web.de cannot send mails anymore, I added some more, following the Exim docs (it's commented out, because I don't use GnuTLS anymore):
#tls_dhparam = /etc/exim4/gnutls-params-2236
#tls_require_ciphers = $ if == $received_port 25 \
# NORMAL:%COMPAT \
# SECURE128
But still I got this kind of errors:
2013-08-14 22:49:27 TLS error on connection from mout.gmx.net [212.227.17.21] (gnutls_handshake): Could not negotiate a supported cipher suite.
As this didn't help either, I recompiled exim4-daemon-heavy against OpenSSL and et voila, it worked again. So, the question is if there's any way to get it working with GnuTLS ? Does the default Debian config work and if so, why? And if not, can a decision be made to use OpenSSL instead of GnuTLS? Reading the bug report it seems as if there are exemptions for linking against OpenSSL , so GPL wouldn't be violated. UPDATE 16.08.2013:
I reinstalled the GnuTLS version of exim4-daemon-heavy to test the recommendation in the comments with explicit tls_require_chiphers settings, but with no luck:
#tls_require_ciphers = SECURE256
#tls_require_ciphers = SECURE128
#tls_require_ciphers = NORMAL
These all resulted in the usual "(gnutls_handshake): Could not negotiate a supported cipher suite." error when trying one by one cipher setting. UPDATE 2 16.08.2013:
There was a different report about recent GnuTLS problem on the debian-user-german mailing list. It's not the same cause, but might be related.
Kategorie:

13 August 2013

Russell Coker: Nexus 4 Ringke Fusion Case

LG Nexus 4 phone with Ringke Fusion case I ve been using Android phones for 2.5 years and for all my phones (Sony-Ericsson Xperia X10, Samsung Galaxy S, Samsung Galaxy S3, and LG Nexus 4) I ve bought gel cases. The gel cases are made of soft rubber which allows phones to bounce rather than break if dropped and also provide better grip (both for holding the phone and for resting it on a vibrating surface such as a tray in a car or plane). The cases have cost me between $12.50 and $15, I haven t felt inclined to pay any more because a case has always seemed like a needless expense, something that merely compensates for deficiencies in phone design. There are some problems with the gel cases, the first one is that they are rather ugly, while I m not really interested in decorating my phones it seems a waste to have a nice phone like the Nexus 4 which has a neat sparkling pattern on the back and then use a case that entirely covers it. The next problem is the fact that the gel cases tend to restrict access to the buttons on the outside of the case. I have been given a Rearth Ringke Fusion case for my Nexus 4 by MobileZap (this is a link for the Nexus 4 cases they sell) [1]. The case features a transparent layer of plastic at the back which protects the phone against having the rear glass of the case break (a known risk with the Nexus 4) while still allowing the user to admire it. With good lighting the sparkling pattern on the back of the phone can still be enjoyed, unfortunately this doesn t show up at all in my photo. It s impossible for a photo to capture a shimmering effect although other photographers have done a better job at showing off a Nexus 4. Also the transparent back to the case gives the option of putting any form of decoration between the phone and the case, for example one could print the logo of their Ingress team and put it between the phone and the case. One thing that is visible in the photo is the feet on the back of the case. With small feet at the corners the transparent back usually won t touch a surface that it rests on, this should prevent unsightly scratches to the back of the case. Also the feet can get some grip on the surface that it rests on, my car has a tray suitable for resting a phone which has a pattern of dimples that catch the feet of the Ringke case and prevent it from sliding. Finally the external buttons for power and volume control are not obstructed in any way. After using 7 phones of 4 different models with gel cases I ve become accustomed to buttons either being difficult to press or easy to press accidentally. The Ringke case allows the buttons to be pressed easily when needed but doesn t seem prone to accidental presses. I m not about to buy a new case for my Galaxy Note 2 (my main phone at the moment) because I don t like paying to replace something that works. But next time I buy a case I ll look for something better than a plain gel case, it s worth paying an extra $9 to get a good case for a phone that costs $350 (the current discount rate for a Nexus 4) or more. I definitely recommend the Ringke case for a Nexus 4.

12 August 2013

Sylvain Le Gall: Migrating a puppet maintained computer from Squeeze to Wheezy

This blog post is a little recipe to do a Debian migration for a node using Puppet and some other good practices. We do all the following commands as root, which on of the exceptional situation where you should have a real root session (login through console, or su -). I tend to avoid using the X server while doing an upgrade, so my 'best' setup is to have a laptop to take note and check things on the internet and a session to the computer to upgrade (ssh + su - or login as root into console). In both case, I use screen during the upgrade so that I can handle disconnection. Create or update your admin-syscheck script First of all, a good practice is to have a script that runs various test on the system that checks everything is ok. This is not only for upgrade but in general. But in the case of upgrade, it can be particularly useful. I call this script admin-syscheck. This is a simple bash script. This script check various aspect of the system and serve me as my external worker to check the most advanced knowledge I have gathered about setting up a service. For example, I know that having *.bak or *.dpkg-dist in /etc/ means that something needs to be merged and a file should be deleted. Another example is about setting up the right alias for 127.0.0.1 and ::1 (which you can differentiate using getent ahostsv4 and getent ahostsv6). I have packaged this script and distributed it using a specific apt-get repository. You can just distribute it using puppet. I recommend to run it daily to track changes (e.g. after an apt-get dist-upgrade) and to check that my setup is aligned with my most up-to-date knowledge about setting up a service (i.e. this is my external worker). In our case we are interested in checking presence of old and new configuration files, before and after upgrading. Here is this specific section of my script:
if ! $IS_ROOT; then
  warning "Not trying to detect dpkg leftover file in /etc/."
else
  LEFTOVER_FILES=( $(find /etc/ \
      -name "*.dpkg-dist" -o \
      -name "*.dpkg-old" -o \
      -name "*.ucf-old" -o \
      -name "*.ucf-dist" -o \
      -name "*.bak") )
  for i in "$ LEFTOVER_FILES[@] "; do
    if [ "$i" = "/etc/hosts.deny.purge.bak" ]; then
      continue
    fi
    if $fix; then
      BASE=$ i%.* 
      cond_exec vim -d $BASE $i
      read -p "Delete $i (y/N)? " ans
      if [ "$ans" = "y" ]; then
        cond_exec rm $i
      fi
    else
       report_error "dpkg leftover file: '$i'."
    fi
  done
fi
(cond_exec allows to do a dry run, you can just remove it). Setting $fix to true will spawn a vim -d old new command where you can edit and then delete the leftover file. This is extremly handy. Upgrading to Wheezy I strongly recommend to read first the upgrade chapter of the release notes. This gives you a more complete overview of the upgrade procedure. I just go through the basic step here. 1. Update everything on the system:
$> apt-get update 
$> apt-get dist-upgrade
2. Check that the current configuration apply cleanly:
$> puppet agent --test
3. Run admin-syscheck:
$> admin-syscheck
And fix all the problems. 4. Disable puppet: I use a cronjob to run puppet, so I just comment the line for the job (/etc/cron.d/puppet-custom). You should disable puppet by stopping the daemon and preventing it to run by editing /etc/default/puppet and set START=no. 4. Fix your sources and pinning: Change squeeze to wheezy in /etc/apt/sources.list and remove useless files in /etc/apt/sources.lists.d/. (You may keep certain sources that refers to stable, like google-chrome.list).
$> rm /etc/apt/sources.list.d/* # Check if this ok to do this with your system.
Although, I tend to fully purge /etc/apt/sources.list expect for the main line (removing backports and security is fine for a short time). The first run of puppet after upgrade will anyway reset this file.
$> rm /etc/apt/preferences.d/* # (at least the ones that do pin some version)
You can although remove all pinning from /etc/apt/preferences. 5. Now you start the real upgrade:
$> apt-get update 
$> apt-get dist-upgrade
6. During the upgrade you will be asked if you want to keep old configuration files or install the newer one from the maintainer. I have always wondered what to answer to this. But here is the answer after a few major upgrade: always install configuration files from maintainer if the service has no ultra-specific settings that could break during the upgrade. The only file, that I should not upgrade on my system, is /etc/sudoers. In this very specific case, you need to make sure before the upgrade that the old and new configuration can coexist. In the squeeze to wheezy case, I have just setup a few extra augeas rules to set the secure_path before the upgrade and it was fine. This is typically the kind of situation where you are thankful to have a real root session. 7. The upgrade can be long and require various fixing to remove/re-add packages to circumvent problems. At the end you will have a set of file *.dpkg-old and *.ucf-old (and some *.dpkg-dist and *.ucf-dist). The *-old files are your old version of the file, while the corresponding files match the maintainer version of it. The *-dist files are the maintainer version of the file and the corresponding files match your old version of it. Starting from here you have 2 options: First upgrade This is the tricky part, you'll have to spend a little time on it: 1. Go over all *. ucf,dpkg - old,dist and merge them with the corresponding configuration file. Use admin-syscheck with fix=true 2. Make a copy of your /etc directory into /etc.new:
$> cp -rp /etc /etc.new
3. Run puppet again:
$> puppet agent --test
4. Disable again the automatic run of puppet, if the previous command has re-enabled it. 5. Make a diff between /etc and /etc.new. Since you have a run of puppet, you know what has changed and should not have changed.
$> diff -Nurd /etc.new /etc
Everytime you find some files that doesn't match your expectation for the upgrade with puppet, change the corresponding puppet manifest to have what you expect. For example, if this is a file:
if ($lsbdistcodename == 'wheezy')  
  file  
    "/etc/foo":
        source => "puppet:///files/foo.wheezy"
    
  else  
  file  
    "/etc/foo":
        source => "puppet:///files/foo.squeeze"
    
 
People working with augeas and puppet, will appreciate the fact that they probably have 0 changes to make for this to work (since it only does a few replacement in configuration files). 6. Once you are happy with the changes, copy back /etc.new to /etc and go to step 3 until the difference is almost 0. 7. Re-enable automatic run of puppet. Do this procedure for a least each computer category you have (e.g. Desktop and Server nodes). Once you are fully confident your new puppet setup works, you will be able to use 'further upgrade' for the other nodes. Further upgrade. This one is super easy compare to a first upgrade: 1. Re-enable puppet and have it run at least once:
$> puppet agent --test
2. Merge *. dpkg,ucf - dist,old with corresponding files (you can run admin-syscheck with fix=true). This is mostly a sanity check since you should have already solved most problem with the 'first upgrade' procedure. That's it. Enjoy your upgrade to Wheezy with puppet.

Chris Lamb: Cotswold Classic Middle Distance triathlon

https://chris-lamb.co.uk/wp-content/2013/cotswold_course.jpg Since becoming interested in triathlon in October of last year, my goal in 2013 was not to just finish a "70.3" (or "Half-Ironman"), but to complete one in under 5 hours. At the time, each discipline was woefully inadequate; I had not swum in over a decade, my cycling was average and I had only begun running again after years away from exercise. The somewhat arbitrary target time would dictate my training. For example, the high proportion of time spent on the bike combined with the injury-sensitive nature of running meant that cycling would have a highest return on investment of the three disciplines. Despite that, I would need to take the run seriously but carefully - I decided to enter a series of monthly 10K races where I could steadily reduce my times over the year. Races themselves formed a crucial part of my preparation and a carefully balanced race calendar was surprisingly important, balancing upcoming milestones that would provide an achievable goal as well as uncovering weaknesses in enough time so they could be worked on. Whilst I did post about my first triathlon, I felt no compulsion with the interleaving five or six races as I was evaluating them in the context of the 70.3 rather than as races in their own right.
Swim
Distance
1900m
Time
37:42
Whilst this wasn't my worst open water swim, it was certainly not my best. The first 500m was straightforward, but I lost my form over the next 500m and I overcompensated with excessive kick. My calves responded by cramping badly two or three times after that, losing both momentum and time. Curiously, whenever I could identify my speed relative to my effort (for example, by some underwater reeds) I could use that as feedback and my rotation (etc.) would temporarily return. Pool swimming always has markers of this kind, whilst a murky lake does not. In conclusion, my lack of open water training outside of races held me back more than I expected.
Bike https://chris-lamb.co.uk/wp-content/2013/cotswold_bike.jpg
Distance
52 miles
Time
2:22 (21.9 mph average)
I had thoroughly absorbed the advice that you should prepare for your primary race so that it feels like another day at the office. On this criterion, the bike leg was almost perfect. My nutrition strategy energy drink every 5 minutes, gel every 30 minutes worked fine until mile 35 where I felt like I had taken on too much fluid (and no time to take a bathroom break). I had a temporary hiccup with my rear derailleur but my "tame" 52/34 12x28 gearing was vindicated on the climbs, although in retrospect that might mostly be schadenfreude. Tens of hours acclimatising to a time trial bike was rewarded by being able to spend the entire leg in "aero" position (ignoring 90-degree turns or serious gradients), improving efficiency.
Run https://chris-lamb.co.uk/wp-content/2013/cotswold_run.jpg
Distance
13.1 miles (21km)
Time
1:54:18 (5:25/km)
I had prepared a chart which provided the pace I required based on the race time at the start of the run. I would then only need to monitor my run pace instead of repeatedly extrapolating a finish time as even basic arithmetic can become impossible when fatigued. This strategy was chosen as I was too risk-adverse to aim for a faster time than strictly required; the downside of missing the target from aiming too high and "blowing up" outweighed going a few minutes faster overall. Despite that, I ran half-marathon personal best. Unfortunately, due to pressing the wrong watch button after the swim, the chart was worthless unless I could assume the race started exactly at 6:30AM. With no other option I decided to trust that with some padding, but the uncertainly it added was unsettling. Adding to that feeling, my left hip started to twinge at 7K, something it has never done before. My plan was to get to 16K, take a caffeine gel and then keep ratcheting up the pace until the finish. In my long run training I could speed up to a 4:10/km pace but after 5 hours of racing and being reasonably sure I was going to reach my goal, the best I could summon was 4:45/km. My form completely shot, I crossed the finish line and got into a fetal position in the shade behind a tent, scaring the race organisers for a few minutes.
Overall
Total time
4:56:50
Writing this the day after the race, I am still a little unsure of how I feel. I am obviously extremely content that I reached my goal but given the volume and detail of preparation it was not a huge surprise to me on race day that I did so. This combined with the "real" work and progress being achieved throughout the training seems to have robbed the event of some element of triumph it could have had, but that in itself was somewhat expected. Where next? I had always dismissed the consensus view of needing to take a decent break after the season has ended but now I am looking forward to putting triathlon-specific training on hold for a while and explore other things. I highly doubt this will be my last triathlon though. https://chris-lamb.co.uk/wp-content/2013/cotswold_result.jpg (Full results)

Joey Hess: please build a haskell to perl compiler

The Fay compiler is a simple way to build fairly comprehensible javascript code from Haskell source. It occurs to me that it should be rather easy to modify Fay to emit perl code rather than javascript. This would allow contributing things like plugins to various perl programs, without writing perl. Of course, the same idea could probably be used to compile Haskell to other languages like python, but perl seems particularly well suited as a second Fay target, since javascript and it have quite similar syntax and similar support for features like closures which Fay relies on. I do not have time to work on this idea myself. It would be a good project for a beginning Haskell programmer. You probably don't even need to fully understand monads to do it! Essentially, look at Fay output examples, translate them from javascript to perl, and then much of the code changes in Fay would probably be in simple string generation code.
I will forward any bitcoins sent to the address 149eBtWS6i8cwQdPQJJ8hAGpDuEqNidyTj to whoever makes this. If it doesn't happen in 1 year, any donations will be forwarded to the EFF instead.

11 August 2013

Lisandro Dami&aacute;n Nicanor P&eacute;rez Meyer: Qt in Debian: using Qt4 and/or Qt5 in your packages

Hi everyone! We now have both Qt4 and Qt5 in the archive. Those using Qt4 should not need to make any changes in their packages, although you can be extra-safe with a few steps. Don't rush, just read below.
Some backgroundSune took the time some months ago to consult upstream for a sane way to allow both SDKs to coexist without us distros having to reinvent the wheel choosing which tools have to be in use in each case.

After a long discussion, upstream decided to write qtchooser (already in the archive) to be able to select between Qt4, Qt5 and even special user's cases like cross-platform builds.

So instead of going trough Debian's alternatives as we did with Qt3/Qt4, we will make use of this new tool.
My package uses Qt, how should I proceed?There are many ways of choosing either of the versions of Qt:

- Using any qtchooser method (preferred):

* Exporting QT_SELECT with 4, qt4, 5 or qt5 as a value in debian/rules.
* Call the tool using the '-qtx' parameter, where x can be replaced with any of the options above.

- Build-depending on qt4-default or qt5-default. You can't B-D on both of them, as they can't coexist.

It is good to notice that:

- any qtchooser method will take precedence over build depending on qtX-default.
- If you export XDG_CONFIG_DIRS it will ignore the default paths to qtchooser's configs we setted up in the packages.

We have also provided qt4-[arch-triplet] and qt5-[arch-triplet] options for special cases.

Once again, if you are already using Qt4, there is no need to rush. See below.

Can is use both Qt4 and Qt5 in my package?You can't mix Qt4 and Qt5 in the same binary, but you may provide libraries compiled against one or the other. For example, your source package foo could provide both libqt4foo1 and libqt5foo1. You need to mangle your debian/rules and/or build system accordingly to achieve this. At the time of this writing I don't know of any examples yet.

So are you going to break the archive with a big transition?No, we have done our best to avoid having to make any changes to existing Qt4 packages. Qt tools should default to Qt4 except if overriden by any method described above.

My package uses Qt4, can I leave it as it is?While there is no need to apply the changes in this case, explicitly setting the Qt version will surely not hurt at all. But don't rush ;-)

9 August 2013

Richard Hartmann: Release Critical Bug report for Week 32

As a more or less random data point for DebConf13, here's the current bug stats for Debian. Hope to see you there, and if you want to attend the BoF Gitify ALL the things or the talk Gitify your life you can do so on site, via IRC in #vcs-home on OFTC, or simply follow the live streams. If you see my in person, I have two crates of Scheider Aventinus with me and don't plan on bringing home any full bottles. The UDD bugs interface currently knows about the following release critical bugs: Graphical overview of bug stats thanks to azhag:

Daniel Pocock: GSoC students progress

I've been lucky enough to be involved in a team mentoring two students contributing to Debian under the Google Summer of Code program this year. Both students have demonstrated a lot of enthusiasm and talent and have just passed their mid-term evaluations. Students often provide a fresh insight to projects. Their potential is not to be underestimated: one diligent student, Thomas Herndon from UMass Amherst recently came to fame exposing fundamental spreadsheet errors in calculations used to back the IMF case for austerity. Herndon was lucky to receive a copy of the spreadsheet (the source was not published openly as in free software): that is one problem that GSoC students don't have to wrestle with. Both of our GSoC students have exposed previously unknown issues in the code they are working on: Fabian discovered API differences in GnuTLS caused problems for dynalogin, while Catalin helped discover that some headers from reSIProcate are susceptible to ABI-breaking behavior dependending on the combination of CPPFLAGS. DebConf this month Both students are going to discuss some of their work on 17 August at DebConf - they are particularly keen to meet with other members of the Debian community throughout the week and find out how their work can best be leveraged by the community and Debian's wider userbase (and derivatives). Fabian Gr nbichler : one time passwords Fabian's project has been working on oath-toolkit and dynalogin. These are excellent projects, helping us realise new opportunities to use tokens for various types of authentication exchanges, such as login. Part of his project involves implementing the recently formalized IETF RFC 6287 for OCRA in a practical way that developers can use. Catalin Usurelu : real-time communications Catalin's project involves working with a variety of open source communications applications to make them work well together on a Debian system. One piece of work he is just completing provides a turn-key SIP conferencing solution that could replace Mumble (which attracted quite some controversy in the previous Debian release cycle) Community development These are all community projects and any feedback from users and other developers is always very welcome. For example, if you have time to test or provide suggestions, please feel free to jump in on the mailing lists we use for these projects.

8 August 2013

Craig Small: postfix transport with smarthost

Dear lazyyweb, Has anyone got a way for postfix to use a transport map such that it sends email to the given MX host for some specified domains then default to a smarthost for the remainder? The logic would be: IF domain in ( example.net , foo.bar ,etc etc) THEN use the relevant MX host ELSE send to smarthost.isp.net i can use a transport map to send from specific domains to specific hosts, eg @domain.com goes to otherisp.net mailserver but not to just use domain.com s MX hosts

Craig Small: postfix transport with smarthost

Dear lazyyweb, Has anyone got a way for postfix to use a transport map such that it sends email to the given MX host for some specified domains then default to a smarthost for the remainder? The logic would be: IF domain in ( example.net , foo.bar ,etc etc) THEN use the relevant MX host ELSE send to smarthost.isp.net i can use a transport map to send from specific domains to specific hosts, eg @domain.com goes to otherisp.net mailserver but not to just use domain.com s MX hosts

Enrico Zini: Getting new Front Desk members (part of things to do at Debconf)

Getting new Front Desk members (part of things to do at Debconf) We need more Front Desk members. Jonathan Wiltshire has kindly put together a job description for the current status quo. Currently we'd like a person to have been an Application Manager before joining Front Desk, but FD tasks are now more diverse than they used to be, and I'd like to spend some time thinking about it, for example here. Job Description for Front Desk Assistant Front Desk assistants are often the first point of contact for new applicants to Debian's New Member process, and are responsible for guiding them through all stages. Front Desk collates applications, assigns them to an Application Manager, checks over the completed application and hands it off to the Account Managers. Front Desk also handles membership of the collab-maint Alioth project. It is not currently involved with applications for Debian Maintainer status. You will 'meet' a wide variety of applicants, some of whom are excited to be starting their journey to Debian membership, whilst others are already established and taking the next step to full developer status. This is a mostly procedural role, but some diplomatic skills are required, particularly when applications are held up for reasons beyond your control. You must have excellent written communication, an eye for detail, and a thorough understanding of what it is to be Debian Members. Expect to spend 2-3 hours per week on this role, divided by as many days as you can manage (to ensure mail is dealt with as soon as possible). The most time-consuming part is processing the completed application. Responsibilities:
  1. To be the first point of contact at Front Desk
  2. To perform initial triage of applications
  3. To match each application with an appropriate Application Manager
  4. To ensure applicants are processed in a timely manner, which may require intervening with an application when it is not progressing satisfactorily
  5. To check completed applications for any missing detail and pass them on to the Account Managers
  6. To handle collab-maint membership business
  7. To answer any queries from applicants, Application Managers, and other stakeholders
  8. To monitor output from the NM routine jobs and fix or escalate any problems
Front Desk staff are not expected to make a judgement call on applicants, though some discretion can be used when appropriate.

6 August 2013

Craig Small: Careful with apache upgrades

You might (or not if you don t visit) notice all my websites were down. A rushed apt-get dist-upgrade and I found two problems:
  1. PHP5 got removed, which is bad if you run a wordpress site that uses PHP to run
  2. The apache configuration has changed.
Yes, the NEWS entries did warn me, if I read them fully. Yes, I didn t read them enough. Apache now ignores configuration files that don t end in .conf To give a completely non-theoretical example, if you have your virtual hosts in files such as /etc/apache2/sites-enabled/enc.com.au then this will not be recognised and your sites will show the default It works page. Stuff that doesn t fall in the usual places where website stuff should go, which for my setup is a lot of things, will also be denied as the developers have tightened up the rules around what is permitted. Pretty simple to fix with a few <Directory blah> clauses. This isn t a criticism of the Debian apache developers. They do an awesome job of keeping the package workable, flexible but secure which isn t easy. Now it s all back working, I actually agree with the changes they have made. It is just that the latest changes are, well, tricky so be forewarned.

5 August 2013

Thomas Goirand: Why using Mailman when MLMMJ is available?

Daniel Pocock just wrote a blog post about how to setup mailman for virtual hosting. Well, it strikes me that mailman is a bad solution, for many reasons. First, it imposes you to use @lists.example.com lists instead of @example.com. I m not sure if that is mandatory, but at least I ve seen only mailman setups done this way. At least, it is how mostly everyone does the setup. I think that s really ugly. Any mailbox should be fine, IMO. What I found particularly lame about Mailman, is that these issues (plus the ones which Daniel listed) have been known for YEARS, though nobody came up with a patch to fix it. And it s really not hard. Why do I know it? Well, because I ve been using MLMMJ for years without such troubles. The current situation where everyone is with mailman is really LAME. Not only MLMMJ is better because it is easier to install and supports virtual hosting out of the box, but also it is written in C and is much faster than Mailman. MLMMJ has been used in high traffic lists like for SUSE and Gentoo. The fact that some major sites decided to do the switch isn t a proof that MLMMJ is perfect, but is a good indication that it at least works well without too much trouble. Also, with mailman, you have to use the subject lines to control your list subscriptions and send command to it. No need to do that with MLMMJ, because everything is controled with the mailbox extension. For example, mylist+subscribe@example.com can be used to subscribe (instead of mylist-requests@lists.example.com, then fill-in the subject line with mailman). So, if you don t like some of the (bad) limitations of Mailman, would like to test something faster, and easier to setup, have a try with MLMMJ (see mlmmj.org for more details, and see the README.Debian inside my package).

Ian Wienand: Skipping bash history for command-lines starting with space

I was recently reading about a case where part of the evidence appears to be a deleted bash history-file. From what I gather, the accused says that the removal was a clean-up job to remove inadvertently stored passwords rather than an attempt to hide nefarious activity. What had managed to pass me by in 15 or so years of using bash but not reading man bash properly is the HISTCONTROL variable. If it is set to ignorespace then commands entered with a leading-space will not be stored in the history. Like all the best discoveries I found this only by accident on a machine where it was turned on. It's been quite useful in keeping my history pruned. However, not usually for security reasons around hiding passwords passwords on the command-line have other problems and I'm sure any security person would tell you not to use them just from a defense-in-depth perspective. That said, from a "keeping ctrl-r reversi-search useful" point-of-view it's often helpful; for example I've mostly trained my fingers to <space>rm -rf because I'm sure I'm not the only one who has deleted the wrong thing via a history-recall-combined-with-typing-to-fast scenario. So, HISTCONTROL=ignoreboth (which also ignores duplicates) is certainly a useful one to slip into your .bashrc. At least it would be one-less thing to explain to the FBI!

Daniel Pocock: Effectively supporting virtual domains with Mailman

When I first decided to run my own list server, I was well aware of the popularity of Mailman but I was deterred from using it by the complexity of setting up virtual hosts. In particular, 99% of the software I use for servers is packaged. Whenever I choose a new piece of software, using something that is packaged and supported is high on my list of priorities. Mailman's distribution tarball and the packages built from it only have limited support for virtual hosting. My own objective is to host lists for open source projects. As some of these projects don't have dedicated budgets, it is particularly important to find lightweight, low-maintenance solutions. Furthermore, it is important that those solutions don't detract from the time I want to invest in the projects themselves. I've developed some scripts to solve the problem: to start using them quickly, please see the README file. Here I provide a more detailed overview of the logic behind this approach. Mailman's limitations and workarounds For me, the biggest limitation of Mailman is that the site list is always linked to a single domain, which has to be chosen arbitrarily from the list of domains being hosted. Although it is not emphasized in the documentation, there is also the inconvenience of having all the lists from potentially unrelated projects appearing together on Mailman's main index page. Another limitation is that each list name must be globally unique across all domains. Some existing workarounds exist but they have limitations too. There is a virtual domain patch, but it is not clear how this handles the site list or the site index page. There is also the postfix-to-mailman script distributed by Debian and Ubuntu: using a transport-based solution looks elegant at first, but when you realise that you need to whitelist the addresses (to ensure messages are rejected at ingress rather than bounced), it becomes just as much work to maintain as the alternatives it aims to avoid: Mailman's authors do not endorse this approach. Experimenting with Sympa Deterred by Mailman's limitations, I initially went with Sympa and that seemed to run quite well for a number of years. In particular, it supports full virtual hosting out-of-the-box and all this is standard in the convenient Debian packages. From time-to-time, users have complained about the use of JavaScript in the web UI, but that was not something I could immediately resolve. However, when I upgraded to Debian 7 (wheezy), more serious problems appeared. Initially, the web UI (used for subscribing or browsing the archives) was completely broken. Anybody accessing the site would be given a download of the wws binary rather than the page that was supposed to be generated by the binary. I discovered errors about this in Apache's error.log and tried installing libapache2-mod-fcgid. The problem was resolved, but from then on, my whole web server became unstable and at least once every day I would have to restart the Apache process or it would become inaccessible. I put the details into a bug report and after not getting any feedback, decided to go and have another look at Mailman. A closer look at Mailman Looking at Mailman again, it appeared that the only way to achieve a relatively "pure" virtual hosting solution would be to run multiple installations of Mailman in different directory trees. Initially, I hoped this could be done by sharing copies of the binaries and just creating replicas of the structure under /var/lib/mailman. Looking under the hood, I discovered that the paths were compiled into the binaries and could not be easily overriden. It appears that this limitation could be overcome with some trivial patches to read directory names from the environment. The main aims of using shared binaries are disk space efficiency and also being able to benefit from security updates for the binary packages. However, if I had to patch the binaries, then I wouldn't be able to benefit from the security update packages anyway. So, it became inevitable that I would have to build binaries from source one way or another: either with a patch or by building multiple copies of each binary, each hardcoded with a different path. Putting it into practice Looking over the Mailman architecture, there are four things to be replicated for each installation:
  • init script to start a standalone qrunner per domain
  • cron jobs
  • CGI configuration from Apache (easily isolated by using a different ScriptAlias inside each Apache <VirtualHost>
  • Invocation of the Mailman binary from the mailer's aliases file
The init scripts and crontabs can be easily generated from a template (I used Debian's copies of these files as a base for that). Some of the Apache configuration can be handled by putting wildcards in the <Directory> specifications and the rest can be easily cut-and-pasted into <VirtualHost> definitions. Based on all of the above, all I had to do was run Mailman's configure script and make once for each virtual domain. I've scripted the whole process - my script is available here Managing the aliases file One particular limitation of my mailer (Postfix) is that the Mailman message processor can't be invoked directly from a rule in the virtual alias map. Scripts like this can only be invoked from the normal /etc/aliases file. This problem is not unique to Mailman: for Sympa, I also had to deal with this. The most common solution is to use the virtual file to translate all the virtual list addresses into very verbose usernames within the local domain:
Address Mapped to
widget-users@lists.example.org lists.example.org-widget-users@my-local-domain
The local part, lists.example.org-widget-users can then be used in the regular /etc/aliases file. This approach is described in the Sympa documentation and also in one of the virtual domain patches for Mailman. I've included a script for generating these alias mappings in the repository with my other scripts for building virtual Mailman installations. It would also be possible to patch Mailman itself to generate such files dynamically when new lists are created. Conclusion and next steps Using the techniques described and these small scripts, I successfully built independent virtual hosts for two of my domains, lists.lumicall.org and lists.dynalogin.org on a single mail host. Notice how each of them has the appearance of a clean, standalone Mailman installation? Going forward, I believe it would not be hard for the 2.x branch of Mailman to support this approach to virtual hosting with three changes:
  • Allowing the binaries to take their directory tree name from an environment variable or command line option or by inspecting argv[0] to discover their own location,
  • Generating the aliases file in the more verbose format and
  • Providing some convenient init script and crontab that iterates over all the domains

4 August 2013

Dirk Eddelbuettel: RcppArmadillo 0.3.900.7

A bug-fix release 3.900.7 of Armadillo was provided by Conrad a few days ago. It has been rolled into a new RcppArmadillo release 0.3.900.7 which, after a brief delay, is now on CRAN and in Debian. The main bug fix is a change in the in-place version of reshape(). This in turn caused the CRAN package Amelia to fail, but some diligent debugging by Conrad (which was much appreciated) revealed that the error was in fact in RcppArmadillo where we did not set the matrix flag properly in the constructor. The corresponding one-character fix had been made in this release. Another package exhibiting an issue is unmarked. It needs to make the target variable of reshape a matrix object in a handful of places. I sent the authors a small patch addressing this but have not yet heard back. The complete list of changes is below. In the meantime, I have also done some work on a new as() and wrap() converters for sparse matrices; a first implementation is in SVN.
Changes in RcppArmadillo version 0.3.900.7 (2013-08-02)
  • Upgraded to Armadillo release Version 3.900.7 (Bavarian Sunflower)
    • minor fix for inplace reshape()
    • minor corrections for compilation issues under GCC 4.8+ and MSVC
  • Corrected setting of vec_stats in intialization of row, vector and matrix objects
  • The inline package is no longer used in the examples and unit tests which have all been converted to using Rcpp attributes
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

Next.

Previous.