Search Results: "ana"

5 January 2012

Aurelien Jarno: Performances of open-source Radeon driver

I am the happy owner of a new netbook with an AMD Fusion E-450 APU, which includes a Radeon graphics card. I am using the open-source driver on it, that is a 3.2-rc7 kernel for KMS, and xserver-xorg-video-radeon package from sid. I have to say I am not really happy about the performances. No I don t speak about the graphical performances that are pretty good (especially compared to my Intel Atom N450 based previous netbook) but about the power consumption. With this setup and with the original battery I get 2h30 of autonomy. Switching to UMS and adding some power management options in xorg.conf improves it to 2h40, but breaks suspend to ram/disk (a pity for a netbook) and switch between VT. I then tried the non-free fglrx driver, it also suffers from the suspend to ram/disk issue, in addition to crashing xorg when playing videos On the other hand I get an impressive 3h30 of autonomy, and additionally a silent netbook (contrary to the open-source driver, the fan doesn t spin at idle). I have tried plenty of options, ranging from adding some power management options to xorg.conf, to passing dynclks=1 to the radeon module, including setting /sys/class/drm/card0/device/power_method to dynpm. Right now I have worked around the issue by buying a bigger battery which brings me 5h30 of autonomy, but I would really appreciate any software way to improve it with the open-source driver.

Aurelien Jarno: Performances of open-source Radeon driver

I am the happy owner of a new netbook with an AMD Fusion E-450 APU, which includes a Radeon graphics card. I am using the open-source driver on it, that is a 3.2-rc7 kernel for KMS, and xserver-xorg-video-radeon package from sid. I have to say I am not really happy about the performances. No I don t speak about the graphical performances that are pretty good (especially compared to my Intel Atom N450 based previous netbook) but about the power consumption. With this setup and with the original battery I get 2h30 of autonomy. Switching to UMS and adding some power management options in xorg.conf improves it to 2h40, but breaks suspend to ram/disk (a pity for a netbook) and switch between VT. I then tried the non-free fglrx driver, it also suffers from the suspend to ram/disk issue, in addition to crashing xorg when playing videos On the other hand I get an impressive 3h30 of autonomy, and additionally a silent netbook (contrary to the open-source driver, the fan doesn t spin at idle). I have tried plenty of options, ranging from adding some power management options to xorg.conf, to passing dynclks=1 to the radeon module, including setting /sys/class/drm/card0/device/power_method to dynpm. Right now I have worked around the issue by buying a bigger battery which brings me 5h30 of autonomy, but I would really appreciate any software way to improve it with the open-source driver.

Aurelien Jarno: Performances of open-source Radeon driver

I am the happy owner of a new netbook with an AMD Fusion E-450 APU, which includes a Radeon graphics card. I am using the open-source driver on it, that is a 3.2-rc7 kernel for KMS, and xserver-xorg-video-radeon package from sid. I have to say I am not really happy about the performances. No I don t speak about the graphical performances that are pretty good (especially compared to my Intel Atom N450 based previous netbook) but about the power consumption. With this setup and with the original battery I get 2h30 of autonomy. Switching to UMS and adding some power management options in xorg.conf improves it to 2h40, but breaks suspend to ram/disk (a pity for a netbook) and switch between VT. I then tried the non-free fglrx driver, it also suffers from the suspend to ram/disk issue, in addition to crashing xorg when playing videos On the other hand I get an impressive 3h30 of autonomy, and additionally a silent netbook (contrary to the open-source driver, the fan doesn t spin at idle). I have tried plenty of options, ranging from adding some power management options to xorg.conf, to passing dynclks=1 to the radeon module, including setting /sys/class/drm/card0/device/power_method to dynpm. Right now I have worked around the issue by buying a bigger battery which brings me 5h30 of autonomy, but I would really appreciate any software way to improve it with the open-source driver.

Ryan Kavanagh: mutt-fetchbug: fetch BTS bug reports from mutt

I've been a longtime mutt user, but have gotten somewhat annoyed of having to open a new terminal when I want to read or reply to a Debian bug with mutt (using 'bts show 123456'). How nice it would be to be able to fetch a bug report from within mutt! And so, ladies and gentlemen, I present to you mutt-fetchbug. It's extensively based on Zack's mutt-notmuch script (a nice interface between mutt and notmuch for searching mail, I highly recommend it). By adding a line like the following to your ~/.muttrc, you can simply press "F7" and enter a bug number from your mutt index in order to fetch and display a bug report's mbox file:
    macro index <f7> \
        "<enter-command>unset wait_key<enter><shell-escape>~/bin/mutt-fetchbug --prompt search<enter><change-folder-readonly>~/.cache/mutt_btsresults<enter><enter-command>set wait_key<enter>" \
        "fetch bug(s) (using bts show)"
The above presupposes you've downloaded mutt-fetchbug and placed it with executable permissions in ~/bin. Those interested in making improvements or extending mutt-fetchbug are more than welcome to do so, it's licensed under the GPLv3+. It can be fetched from my git branch. Update: Fixed the broken link to mutt-fetchbug, thanks Christian for the heads up.

Michal &#268;iha&#345;: ColorHug client in unstable

It took me much more uploads than I expected, but ColorHug client is finally in unstable. I was fighting with various bugs, which I could not reproduce locally and of course with my own stupidity in configuring pbuilder. Anyway the buildds overview now shows mostly green, so this is good. The remaining ones mostly wait for libgusb, which needs build on armhf and needs to be ported to non Linux (or rather libusb needs to exist there). One of things I've learned on the way is that docbook-utils is not enough to convert docbook to man pages, you need docbook package as well. Still I don't understand why this did locally work for me... The one outstanding issue is that client code assumes it runs on little endian host (or rather same endianity as the ColorHug device) and thus will break on PPC and other big endian machines. The upstream is aware of that an I even managed to produce patch which should fix it, though I have no means to test it (lack of big endian hardware and lack of the ColorHug device itself).

Filed under: Debian English 4 comments Flattr this!

Russell Coker: DRBD Benchmarking

I ve got some performance problems with a mail server that s using DRBD so I ve done some benchmark tests to try and improve things. I used Postal for testing delivery to an LMTP server [1]. The version of Postal I released a few days ago had a bug that made LMTP not work, I ll release a new version to fix that next time I work on Postal or when someone sends me a request for LMTP support (so far no-one has asked for LMTP support so I presume that most users don t mind that it s not yet working). The local spool on my test server is managed by Dovecot, the Dovecot delivery agent stores the mail and the Dovecot POP and IMAP servers provide user access. For delivery I m using the LMTP server I wrote which has been almost ready for GPL release for a couple of years. All I need to write is a command-line parser to support delivery options for different local delivery agents. Currently my LMTP server is hard-coded to run /usr/lib/dovecot/deliver and has it s parameters hard-coded too. As an aside if someone would like to contribute some GPL C/C++ code to convert a string like /usr/lib/dovecot/deliver -e -f %from% -d %to% -n into something that will populate an argv array for execvp() then that would be really appreciated. Authentication is to a MySQL server running on a fast P4 system. The MySQL server was never at any fraction of it s CPU or disk IO capacity so using a different authentication system probably wouldn t have given different results. I used MySQL because it s what I m using in production. Apart from my LMTP server and the new version of Postal all software involved in the testing is from Debian/Squeeze. The Tests All tests were done on a 20G IDE disk. I started testing with a Pentium-4 1.5GHz system with 768M of RAM but then moved to a Pentium-4 2.8GHz system with 1G of RAM when I found CPU time to be a bottleneck with barrier=0. All test results are for the average number of messages delivered per minute for a 19 minute test run where the first minute s results are discarded. The delivery process used 12 threads to deliver mail.
P4-1.5 p4-2.8
Default Ext4 1468 1663
Ext4 max_batch_time=30000 1385 1656
Ext4 barrier=0 1997 2875
Ext4 on DRBD no secondary 1810 2409
When doing the above tests the 1.5GHz system was using 100% CPU time when the filesystem was mounted with barrier=0, about half of that was for system (although I didn t make notes at the time). So the testing on the 1.5GHz system showed that increasing the Ext4 max_batch_time number doesn t give a benefit for a single disk, that mounting with barrier=0 gives a significant performance benefit, and that using DRBD in disconnected mode gives a good performance benefit through forcing barrier=0. As an aside I wonder why they didn t support barriers on DRBD given all the other features that they have for preserving data integrity. The tests with the 2.8GHz system demonstrate the performance benefits of having adequate CPU power, as an aside I hope that Ext4 is optimised for multi-core CPUs because if a 20G IDE disk needs a 2.8GHz P4 then modern RAID arrays probably require more CPU power than a single core can provide. It s also interesting to note that a degraded DRBD device (where the secondary has never been enabled) only gives 84% of the performance of /dev/sda4 when mounted with barrier=0.
p4-2.8
Default Ext4 1663
Ext4 max_batch_time=30000 1656
Ext4 min_batch_time=15000,max_batch_time=30000 1626
Ext4 max_batch_time=0 1625
Ext4 barrier=0 2875
Ext4 on DRBD no secondary 2409
Ext4 on DRBD connected C 1575
Ext4 on DRBD connected B 1428
Ext4 on DRBD connected A 1284
Of all the options for batch times that I tried it seemed that every change decreased the performance slightly but as the greatest decrease in performance was only slightly more than 2% it doesn t matter much. One thing that really surprised me was the test results from different replication protocols. The DRBD replication protocols are documented here [2]. Protocol C is fully synchronous a write request doesn t complete until the remote node has it on disk. Protocol B is memory synchronous, the write is complete when it s on a local disk and in RAM on the other node. Protocol A is fully asynchronous, a write is complete when it s on a local disk. I had expected protocol A to give the best performance as it has lower latency for critical write operations and for protocol C to be the worst. My theory is that DRBD has a performance bug for the protocols that the developers don t recommend. One other thing I can t explain is that according to iostat the data partition on the secondary DRBD node had almost 1% more sectors written than the primary and the number of writes was more than 1% greater on the secondary. I had hoped that with protocol A the writes would be combined on the secondary node to give a lower disk IO load. I filed Debian bug report #654206 about the kernel not exposing the correct value for max_batch_time. The fact that no-one else has reported that bug (which is in kernels from at least 2.6.32 to 3.1.0) is an indication that not many people have found it useful. Conclusions When using DRBD use protocol C as it gives better integrity and better performance. Significant CPU power is apparently required for modern filesystems. The fact that a Maxtor 20G 7200rpm IDE disk [3] can t be driven at full speed by a 1.5GHz P4 was a surprise to me. DRBD significantly reduces performance when compared to a plain disk mounted with barrier=0 (for a fair comparison). The best that DRBD could do in my tests was 55% of native performance when connected and 84% of native performance when disconnected. When comparing a cluster of cheap machines running DRBD on RAID-1 arrays to a single system running RAID-6 with redundant PSUs etc the performance loss from DRBD is a serious problem that can push the economic benefit back towards the single system. Next I will benchmark DRBD on RAID-1 and test the performance hit of using bitmaps with Linux software RAID-1. If anyone knows how to make a HTML table look good then please let me know. It seems that the new blog theme that I m using prevents borders. Update: I mentioned my Debian bug report about the mount option and the fact that it s all on Debian/Squeeze. Related posts:
  1. I need an LMTP server I am working on a system where a front-end mail...
  2. Some Notes on DRBD DRBD is a system for replicating a block device across...
  3. paper about ZCAV This paper by Rodney Van Meter about ZCAV (Zoned Constant...

Jaldhar Vyas: Jeopardy!: The Post-Mortem

By now hopefully you got a chance to see my Jeopardy! episode so it won't be much of a spoiler to mention that I didn't win. It went like this. I flew into LA on the night of October 31st. The taping was to take place on the next two days. Jeopardy! tapes two weeks of shows, two days a week for a total of 10 episodes. Contestants are randomly chosen for one of those 10 episodes but you don't find out which one you will be on until the last minute. Randomness and secrecy have become become a big part of the production process in games since the quiz show scandals of the 1950s. (Interesting fact: it is a Federal felony to tamper with a TV game show.) There were a group of just under 20 0f us altogether waiting to go on. All very nice people. Here in the New York area one tends to equate very intelligent people with "Type A" personalities but we got on well together. Interspersed between the tapings, we had short practice sessions to familiarize ourselves with the studio setup and this is where I first started getting a little apprehensive. Jeopardy! is not just about general knowledge, there is a physical element too in that you have to buzz in by pressing a button on a stick which is wired up to some producers console. You can't buzz in before Alex has finished reading the question and obviously you can't buzz in if someone is already answering the question. If you buzz at the wrong time, your buzzer locks for 0.75 seconds so just jamming your finger on the button. I didn't even get that far as on the entire first day I failed to buzz in at all despite changing hands, changing grip different ways and other strategies. I was pretty despondent by the second day but then miraculously I managed to get the hang of it. So when I was finally selected (for the 3rd game of that day.) I was back to my usual cool self. I was up against Dave Leach and Nicholas Campiz who were returning after having tied in the previous game. And the game did start well enough. Oh one thing you may notice is that Alex mispronounces my name Jaldhal. I knew he did that once or twice and even mentioned it to a producer but it was only viewing the episode right now that I see he does it consistently. I assumed such things would be fixed up in post-editting but I guess not. Oh well, after living in this country for so many years I've learned to be happy whenever two consonants are more or less in the right place. This particular mispronunciation is actually fairly common, even Indian people have been known to make it. The alternative theory is that Alex couldn't read my horrible handwriting which is also plausible. Anyway, I digress. I started off fairly well. My first mistake was in a category about alcoholic drinks. I answered vermouth when the correct response was gin. In my defense I'm a strict teetoler so none of these words mean anything to me. I also earned the disapproval of my superhero fan son by mistaking Dr. Strange for Dr. Doom. I did get the one about Spiderman villain Dr. Octopus so that mollified him somewhat. The highlight of my evening was strangely enough in a category about books of the Bible. I got a Daily Double. It was a risky move but I have always wanted to make a "true" daily double so I threw caution to the winds and wagered everything I had. To my surprise as much as anyones, got the answer right doubling my score. I was in a pretty good position for the Double Jeopardy but thats when Dave started dominating. If you haven't been watching previous games, Dave Leach is arguably the best player this season. Not only is he smart but he is almost supernaturally fast on the buzzer. You can't see because I kept my hands behind the podium but I again and again I would desperately try to ring in on a question I knew the answer to only to be beaten to the punch by Dave. This caused me to start getting a bit reckless and answering questions where I was less than certain which cost me dearly. One particularly dumb mistake was in a category where all the answers were supposed to start with the letter g. I said compass when I should said gyrocompass but actually I should have left that category alone entirely. By the time Final Jeopardy came around winning was no longer an option as Dave was too far in the lead. I was slightly ahead of Nicholas in 2nd place. This is where I again made a strategic error. My original plan to bet nothing in Final Jeopardy but the category was "1930s Novels" and I liked my chances. The clue was read out and it involved an anti-war novel whose author was blacklisted. I was elated. It had to be "All Quiet on the Western Front" right? The blacklist bit gave me pause as the author Erich Maria Remarque was German and this was obviously a reference to the anti-Communist blacklist in the 1950s USA. But somehow I remembered that he had spent the end of his life in Hollywood or maybe I just convinced myself that it was plausible. Evidently my thought process wasn't too crazy because all three of us came up with the same answer. All three of us were wrong. The correct answer was "Johnny Got His Gun" by Daltan Trumbo. In hindsight I knew this but it illustrates the strange ways the mind works. I knew because the video for Metallicas song "One" from "...And Justice for All" is based on the film adaptation of this book. However in a previous orientation session the lady who is the contestant coordinator for Jeopardy! had mentioned AQOTHWF as anexample of something and this got stuck in my brain crowding out the previously known fact. So when all was said and done I ended up in 3rd place. (I bet big whereas Nicholas was more conservative.) Oh well. Despite not winning I had a blast. I fulfilled my dream of appearing on Jeopardy! I met Alex Trebek. (And I should be receiving an autographed picture of us together soon.) My witty banter was sufficiently witty. I made enough money to cover my travel expenses and still have a nice amount left. And I got to meet lots of interesting and wonderful people. So I'm satisfied.

4 January 2012

Clint Adams: The rise of TH

We are about to lose hledger on armel, armhf, mips, mipsel, powerpc, s390, and s390x. If this affects you, we need working Template Haskell and GHCi support on your architecture.

Pau Garcia i Quiles: FOSDEM 2012 CrossDesktop DevRoom: deadline extension

FOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium). One of the developer rooms will be the CrossDesktop DevRoom, which will host Desktop-related talks. Are you interested in giving a talk about open source and Qt, KDE, Enlightenment, Gnome, XFCE, Windows, Mac OS X, general desktop matters, mobile development, applications that enhance desktops and/or web? We have extended the deadline for a few more days, until January 8th. If you want to submit a talk proposal, hurry up! I have to say I am very surprised to see very few Qt/KDE talk proposals. Is there nothing interesting the Qt and KDE world have to say to 5,000+ people?
There is more information in the Call for Talks we published a couple of months. If you are interested in Qt/KDE, come visit us at the KDE booth. If you add yourself to the KDE FOSDEM 2012 wiki page, we will be able to better organize the usual dinner on Sunday and/or smaller meetings for special interest groups .

3 January 2012

Matthew Garrett: TVs are all awful

A discussion a couple of days ago about DPI detection (which is best summarised by this and this and I am not having this discussion again) made me remember a chain of other awful things about consumer displays and EDID and there not being enough gin in the world, and reading various bits of the internet and wikipedia seemed to indicate that almost everybody who's written about this has issues with either (a) technology or (b) English, so I might as well write something.

The first problem is unique (I hope) to 720p LCD TVs. 720p is an HD broadcast standard that's defined as having a resolution of 1280x720. A 720p TV is able to display that image without any downscaling. So, naively, you'd expect them to have 1280x720 displays. Now obviously I wouldn't bother mentioning this unless there was some kind of hilarious insanity involved, so you'll be entirely unsurprised when I tell you that most actually have 1366x768 displays. So your 720p content has to be upscaled to fill the screen anyway, but given that you'd have to do the same for displaying 720p content on a 1920x1080 device this isn't the worst thing ever in the world. No, it's more subtle than that.

EDID is a standard for a blob of data that allows a display device to express its capabilities to a video source in order to ensure that an appropriate mode is negotiated. It allows resolutions to be expressed in a bunch of ways - you can set a bunch of bits to indicate which standard modes you support (1366x768 is not one of these standard modes), you can express the standard timing resolution (the horizontal resolution divided by 8, followed by an aspect ratio) and you can express a detailed timing block (a full description of a supported resolution).

1366/8 = 170.75. Hm.

Ok, so 1366x768 can't be expressed in the standard timing resolution block. The closest you can provide for the horizontal resolution is either 1360 or 1368. You also can't supply a vertical resolution - all you can do is say that it's a 16:9 mode. For 1360, that ends up being 765. For 1368, that ends up being 769.

It's ok, though, because you can just put this in the detailed timing block, except it turns out that basically no TVs do, probably because the people making them are the ones who've taken all the gin.

So what we end up with is a bunch of hardware that people assume is 1280x720, but is actually 1366x768, except they're telling your computer that they're either 1360x765 or 1368x769. And you're probably running an OS that's doing sub-pixel anti-aliasing, which requires that the hardware be able to address the pixels directly which is obviously difficult if you think the screen is one size and actually it's another. Thankfully Linux takes care of you here, and this code makes everything ok. Phew, eh?

But ha ha, no, it's worse than that. And the rest applies to 1080p ones as well.

Back in the old days when TV signals were analogue and got turned into a picture by a bunch of magnets waving a beam of electrons about all over the place, it was impossible to guarantee that all TV sets were adjusted correctly and so you couldn't assume that the edges of a picture would actually be visible to the viewer. In order to put text on screen without risking bits of it being lost, you had to steer clear of the edges. Over time this became roughly standardised and the areas of the signal that weren't expected to be displayed were called overscan. Now, of course, we're in a mostly digital world and such things can be ignored, except that when digital TVs first appeared they were mostly used to watch analogue signals so still needed to overscan because otherwise you'd have the titles floating weirdly in the middle of the screen rather than towards the edges, and so because it's never possible to kill technology that's escaped into the wild we're stuck with it.

tl;dr - Your 1920x1080 TV takes a 1920x1080 signal, chops the edges off it and then stretches the rest to fit the screen because of decisions made in the 1930s.

So you plug your computer into a TV and even though you know what the resolution really is you still don't get to address the individual pixels. Even worse, the edges of your screen are missing.

The best thing about overscan is that it's not rigorously standardised - different broadcast bodies have different recommendations, but you're then still at the mercy of what your TV vendor decided to implement. So what usually happens is that graphics vendors have some way in their drivers to compensate for overscan, which involves you manually setting the degree of overscan that your TV provides. This works very simply - you take your 1920x1080 framebuffer and draw different sized black borders until the edge of your desktop lines up with the edge of your TV. The best bit about this is that while you're still scanning out a 1920x1080 mode, your desktop has now shrunk to something more like 1728x972 and your TV is then scaling it back up to 1920x1080. Once again, you lose.

The HDMI spec actually defines an extension block for EDID that indicates whether the display will overscan or not, but doesn't provide any way to work out how much it'll overscan. We haven't seen many of those in the wild. It's also possible to send an HDMI information frame that indicates whether or not the video source is expecting to be overscanned or not, but (a) we don't do that and (b) it'll probably be ignored even if we did, because who ever tests this stuff. The HDMI spec also says that the default behaviour for 1920x1080 (but not 1366x768) should be to assume overscan. Charming.

The best thing about all of this is that the same TV will often have different behaviour depending on whether you connect via DVI or HDMI, but some TVs will still overscan DVI. Some TVs have options in the menu to disable overscan and others don't. Some monitors will overscan if you feed them an HD resolution over HDMI, so if you have HD content and don't want to lose the edges then your hardware needs to scale it down and let the display scale it back up again. It's all awful. I recommend you drink until everything's already blurry and then none of this will matter.

comment count unavailable comments

Julien Danjou: Google Calendar notifications using pynotify

I use Google Calendar to manage my calendars, and I really missed something to warn me whenever I have an appointment with an alert set. So here is an example of a Python program to do such a thing. It is written using the Google Data APIs Python client library and pynotify. I'll detail the code here, so you can build your own and adapt it to your needs. First, we need to import GTK+ and pynotify, and initialize it.
import gtk
import pynotify
pynotify.init(sys.argv[0])
Then, we need to import gdata Calendar API and connect to the calendar. I'll use the simple email/password way to login, which is clearly not the best, but it's also the simplest. Feel free to use OAuth 2.0. :-)
calendar_service = gdata.calendar.service.CalendarService()
calendar_service.email = 'mygooglelogin'
calendar_service.password = 'mygooglepassword'
calendar_service.ProgrammaticLogin()
Now we're ready to request stuff and notify! First, request the events from the default calendar.
feed = calendar_service.GetCalendarEventFeed()
Now we can iterate over feed and do various checks.
for event in feed.entry:
    # If the event status is not confirmed, go to the next event.
    if event.event_status.value != "CONFIRMED":
        continue
    # Now iterate over all the event dates (usually it has one)
    for when in event.when:
        # Parse start and end time
        try:
            start_time = datetime.datetime.strptime(when.start_time.split(".")[0], "%Y-%m-%dT%H:%M:%S")
            end_time = datetime.datetime.strptime(when.end_time.split(".")[0], "%Y-%m-%dT%H:%M:%S")
        except ValueError:
            # ValueError happens on parsing error. Parsing errors
            # usually happen for "all day" events since they have
            # not time, but we do not care about this events.
            continue
        now = datetime.datetime.now()
        # Check that the event hasn't already ended
        if end_time > now:
            # Check each alert
            for reminder in when.reminder:
                # We handle only reminders with method "alert"
                # and whose start time minus the reminder delay has passed
                if reminder.method == "alert" \
                        and start_time - datetime.timedelta(0, 60 * int(reminder.minutes)) < now:
                    # Build the notification
                    notification = pynotify.Notification(summary=event.title.text,
                                                         message=event.content.text)
                    # Set an icon from the GTK+ stock icons
                    notification.set_icon_from_pixbuf(gtk.Label().render_icon(gtk.STOCK_DIALOG_INFO,
                                                                              gtk.ICON_SIZE_LARGE_TOOLBAR))
                    notification.set_timeout(0)
                    # Show the notification
                    notification.show()
Running this program, you should see a notification if an appointment has an alert to be raised at that time. This should be enough to start to build something. If you don't want to program this into Python, you might want to take a look at gcalcli.

Rapha&#235;l Hertzog: My Debian Activities in December 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (364.18 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg and Multiarch I had some hope to have a multiarch-enabled dpkg in sid for Christmas as Guillem told me that it was realistic. Alas Guillem got sick. We re in January and we re still not there. While some of Guillem s commits in December were related to multi-arch, the size of his pu/multiarch/master branch did not really shrink. We still have 36 commits to merge most of the work he did was refactoring some parts of the code that were already merged. And he initiated some discussion on interface changes. I participated to those discussions hoping to bring them to a quick resolution. I m still maintaining my own pu/multiarch/full branch, it is based on Guillem s branch but with further fixes that I developed and that he has not yet merged and with a change reverted (Guillem s branch allows crossgrading packages between different architectures while dpkg does not manage this correctly yet). I can only hope that January will be the last month of this never-ending saga. It s been one year since I started working on this project. :- Misc dpkg work I reviewed (and later merged) a patch of Kees Cook to enhance dpkg-buildflags so that it can report which hardening features are enabled. This feature might then be used by tools like lintian to detect missing hardening features. I mentored Gianluca Ciccarelli who is trying to enhance dpkg-maintscript-helper to take care of replacing a directory by a symlink and vice-versa. I took care of #651993 so that dpkg-mergechangelogs doesn t fail when it encounters an invalid version in the changelog, and of #652414 so that dpkg-source --commit accepts a relative filename when a patch file is explicitly given. Guillem also merged a fix I developed for LP#369898. Packaging work WordPress 3.3 came out so I immediately packaged it. Despite my upstream bug report, they did not update their GPL compliance page which offers the corresponding sources for what s bundled in the tarball. So I hunted for the required sources myself, and bundled them in the debian.tar.xz of the Debian source package. It s a rather crude solution but this allowed me to close the release critical bug #646729 and to reintroduce the Flash files that were dropped in the past which is great since the Flash-based file uploader is nicer than the one using the browser s file field. Quilt 0.50 came out after 2 years of (slow) development. The Debian package has many patches and several of them had to be updated to cope with the new upstream release. Fortunately some of them were also merged upstream. It still took an entire morning to complete this update. I also converted the packaging from CDBS to dh with a short rules file. Zim 0.54 came out and I immediately updated the package since it fixed a bug that was annoying me. Review of the ledgersmb packaging As the sql-ledger maintainer (and a user of this software for my accounting), I have been hoping to get ledgersmb packaged as a possible replacement for it. I have been following the various efforts initiated over time but none of them resulted in a real package in Debian. This is a real pity so I tried to fix this by offering to sponsor package uploads. That s why I did a first review of the packaging. It took several hours because you have to explain everything that s not good enough. I also filed a wishlist bug against lintian (#652963) to suggest that lintian should detect improper usage of dpkg-statoverride (this is a mistake that was present in the package that I reviewed). nautilus-dropbox work I wanted to polish the package in time for the Ubuntu LTS release and since Debian Import Freeze is in January, I implemented some of the important fixes that I wanted. The Debian package diverges from upstream in that the non-free binaries are installed in /var/lib/dropbox/ instead of $HOME.
Due to a bug, the files were not properly root-owned so I first fixed this (unpacking the tarball as root lead to reuse of the embedded user & group information, and those information changed recently on the Dropbox side apparently). Then we recently identified other problems related to proxy handling (see #651065). I fixed this too because it s relatively frequent that the initial download triggered during the package configuration fails and in that case it s the user that will re-trigger a package download after having given the appropriate credentials through PackageKit. Without my fix, usage of pkexec would imply the loss of the http_proxy environment variable and thus it would not be possible for a user to download through a proxy. Last but not least I reorganized the Debian specific patches to better separate what can and should be merged upstream, from the changes that upstream doesn t want. Unfortunately Dropbox insists on being able to auto-update their non-free binaries, they are, thus, against the installation under /var/lib/dropbox and the corresponding changes. Book update We re making decent progress in the translation of the Debian Administrator s Handbook, about 6 chapters are already translated (not yet reviewed though). The liberation campaign is also (slowly) going forward. We re at 67% now (thanks to 90 new supporters!) while we were only at 60% at the start of December. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Jaldhar Vyas: Watch Me on Jeopardy! This Wednesday

<video controls="controls" poster="http://www.braincells.com/debian/images/jaldhar_jeopardy_howdy.jpg">
<source src="http://www.braincells.com/debian/images/jaldhar_jeopardy_howdy.mpg" type="video/mpeg">
<source src="http://www.braincells.com/debian/images/jaldhar_jeopardy_howdy.ogv" type="video/ogg">
HTML 5 Video does not seem to be supported by your browser. Download the video of my Jeopardy! "Hometown Howdy" here (MPG format) or here (OGG format). </video> Some time ago, I mentioned auditioning for Jeopardy!, Americas top TV quiz show. I didn't hear back from the producers for quite a while so I assumed I hadn't been selected but then suddenly one day in October I got a call inviting me to come out to Sony Studios in Culver City California for a taping in November. I went and it is finally airing now. So if you are in the US or Canada (the show is possibly syndicated elsewhere but I'm not sure) please be sure to watch me on Wednesday, January the 4th. I won't say how well I did until afterwards but I was quite pleased with my performance.

1 January 2012

Joey Hess: a resolution that stuck

Last year, my new year's resolution was to write in my journal every day. That actually stuck, I wrote 262 journal entries in 2011. While I've been keeping a journal intermittently since 1998, last year I doubled the number of entries in it. And wrote a novel's worth of entries -- 53 thousand words! Most of it is of course banal and mundane stuff. Not good compared with Lars, who does something with his journal where he goes into some detail about code he's working on, and other work. The excerpts I've seen are quite nice. But after I've written code, written a commit message, documentation, perhaps bug reports etc, I often can't find much to say about it in my journal, beyond the bare bones that I worked on $foo today or faced a particularly hard bug. I also worry that the journal, and my reluctance to repeat myself, often tips the balance away from me blogging, if I write down something in the journal first.
Here's my journal for today:
Compare what jokes are funny now with those in 1982. The 1982 ones from net.jokes on olduse.net seem juvenile. Now compare what Unix joke man pages are funny now with those I'm reading from 1982. They seem basically the same. What would Biella make of this? Liw noticed ikiwiki OOM on pell. Tracked down to a perl markdown bug with long lines. Had quite enough of perl markdown; ikiwiki will be moving to a different engine. Added discount support to it today, still needs Debian package tho. [censored] Really gorgeous sunset, with a high wind, moon, puffy low, fast moving clouds. Enjoyed it ecstaticly. It's going to get cold soon. Very rainy early, but then got intermittently sunny; power is holding out ok. Was going to roast a chicken today, but got distracted and had a large lunch besides. Need to find some quick food for supper. I need to start a new book, should it be the River Cottage book about meat that I stole from Anna, or some SF? Blogged about journaling, and put this journal entry in it, so also journaled about blogging. Wrote it somewhat self-conciously.

The benefits for me have ranged from being able to go back and work out dates of events, to forwarding the odd excerpts to others. The best thing though is certianly having a regular time of introspection, to look back over my the day. If you've not got a new year's resolution yet, I recommend this one. (Learning Haskell would be another good one, if you haven't yet.) Just write something, anything, down in your journal every day.

Gregor Herrmann: RC bugs 2011/52

a rather lazy bugsquashing week but with a little cheating a managed to get at at least 7 RC bugs:

Russ Allbery: 2011 Book Reading in Review

For the year of 2011, I finished and reviewed 60 books. This is a huge milestone for me; it's the first time since the second year I started doing this that the number of books I read actually increased. This gives me more confidence that I've stabilized the year-by-year decline in my reading. I did that while substantially increasing the amount of time I spent enjoying video games, which was another major goal of the year. Only two books received a 10 out of 10 this year, one fiction and one non-fiction. The novel was Jo Walton's Among Others: the best book I read this year. It's a delightful look at the process of finding a place for oneself in the world and features one of the best protagonists that I've seen. The non-fiction book was Rory Stewart's The Prince of the Marshes, which means that both of Stewart's books that I've read have received 10 ratings. The Prince of the Marshes is his look at his time spent in the provisional government of Iraq after the overthrow of Saddam Hussein. I think it should be required reading for anyone expressing an opinion on what the US and other western powers should or should not have done in Iraq. It lays bare the difficulties, confusion, and frequent stupidity of going into someone else's country and trying to fix it, and I think seriously calls into question whether this sort of international intervention can ever work. Other fiction highlights of the year were Kelley Eskridge's Solitaire, a startling and deep look at identity and social connection, and Mira Grant's Feed, a zombie apocalypse story that completely overcame my deep dislike of zombie apocalypse stories. Feed should have won the Hugo in 2011, despite some unbelievable politics and a bit too much cheering for bloggers. This was the year for excellent protagonists, with all three of my top-rated fiction books featuring unique and memorable characters who left a deep and lasting emotional impact. The two other non-fiction standouts were both a bit dry, but if you have the patience and attention, they reward persistance. Richard Hofstadter's Anti-Intellectualism in American Life is a deserved classic that gave me an eye-opening perspective on the long history of interactions between intellectualism and populism in US politics and culture. David Levering Lewis's God's Crucible is a wonderful history of Islam as it related to Europe and filled in some large gaps in my knowledge of world and religious history. 60 books a year, or five books a month, feels like a comfortable and sustainable level, although I'm going to keep my formal goal at a book a week (52 in the year) to give myself some leeway to either get distracted by video games or by other projects. My reading did concentrate more than usual in science fiction and fantasy this year, and I'd like to add more mainstream fiction and more non-fiction. The full analysis includes some additional personal reading statistics, probably only of interest to me.

Tanguy Ortolo: Debian Developper

Debian red swirl For this new year I got a quite specific present : a Debian Developer account. Of course this is not an end by itself but rather a beginning, or perhaps a continuation of my modest effort to contribute to the greater good of our users and free software. As an introduction, I would like to quickly explain how I came to this point, and I also have some people to thank.

Debianist journey
1997-2003
First family computer, a PC running Microsoft Windows 98. I became a power-user, and I progressively replaced all the pieces of that computer, except it operating system (though I reinstalled it several times).
2003
First contact with GNU/Linux: at my preparatory school, we use XEmacs and Kile under Mandrake Linux 9.2 to do some works in Caml Light and LaTeX. I start enjoying free software at home, with Mozilla Firefox, Thunderbird and Libre^WOpenOffice.org.
2005, preparatory school
I try Knoppix at home and eventually install Mandriva Linux 10.2 for my own use. Since then I have never used Microsoft Windows for personal work any more.
2005, engineer school
I integrate Centrale Paris and I join the team of VIA Centrale R seaux, which I believe is a known den of debianists. So I start using Debian on my personal laptop.
2006
Debian power-user. I start reporting bugs.
2007
First small contributions: some of my bug reports now include patches. I also start maintaining a French training document about Debian: formation Debian GNU/Linux.
2008
First packaging work with the help of adn, a modification and split of frozen-bubble to have a server-only package (contribution finally refused, actually).
2010
First package: I adopt dokuwiki with the help of adn. I become a Debian Maintainer, with the intent of applying as a Debian Member later.
2012
I am now a Debian Developer, responsible for a handful of packages and able to do NMUs and to sponsor new contributors.
As you may guess, I have always been a tinkerer, about computers but also about bikes (the ones that work with manpower, not the ones that make noise and smoke ;-)). And I also always wanted to publish my works somehow, because I hate to think that someone could have to do again some reusable work I had to do before instead of taking advantage of it. Well, it happened that free software was a way to achieve both goals: to hack and to make my work available for anyone. Thanks I would like to thank all those who helped me to discover and contribute to free software, especially: adn, who helped me with my first steps to directly contribute to Debian, dererk and all the Debian Developers that patiently reread and sponsored my packages, and my Application Manager gwolf, who gave quite a lot of time asking me questions, reading my replies, and replying to some questions that arose during the process. Outside of the Debian Project I have to thank Alexis de Lattre, who wrote the original Formation Linux that I am now maintaining under the name of Formation Debian GNU/Linux , a document which helped me to discover GNU/Linux and Debian specifically. More generally, I would like to thank all the Free Software community that made that possible: Debian Developers, hackers and users from everywhere. We have built and we are still building quite an impressive pile of very useful work, and we can be proud of it. Notes
  1. Here in the northern hemisphere our school years begin in the middle of the civil year, which makes it tricky to tell them apart
  2. This is almost a synonym for tinkering, in the context of free software, is it not?

31 December 2011

Steve Kemp: The final updates of 2011

I've been informed by a couple of people that the Debian Administration site is down. Sadly it is; at the moment the host isn't showing anything on the serial console and remotely power-cycling it isn't showing any signs of life. At this time of year I don't want to drag anybody in to take care of it, so ETA on recovery/replacement hardware is Monday/Tuesday. In other news I've made it to year five of the KVM hosting sub-project/thing. Originally started as a Xen host its been running happily for quite some time. I suspect next year, or the year after that the price/specification ratio will end up losing out and we'll cancel the whole thing - but there are no immediate reasons to make any change. Finally I knocked up a simple tool to validate my TinyDNS records prior to uploading them. It is simplistic, but adequate to catch the kind of mistakes I make: Honestly it probably wants to be rationalised a little more - and check records more carefully. e.g. Ensure that the host a CNAME refers to itself exists, and making sure that the nameservers specified are valid. I just wanted to make something quick after accidentally uploading a zonefile where I'd managed to fat-finger several important records. le sigh. Oddly enough asking on serverfault.com showed no real suggestions - other than actually running tinydns locally and doing a zone-xfer to validate records. Overkill and harder than I'd like. Happy New year if you care about such things.. "I finished growing up, L on. I just get older. " - Leon

Russell Coker: My Blog Server was Cracked

On the 1st of August I noticed that the server which runs my blog among other things was having an occasional SEGV from a sshd process. Unfortunately I was busy and didn t pay much attention to this, which turned out to be a big mistake. On the 12th of September I started investigating this properly and noticed that when someone tried to connect to ssh with password authentication sshd would SEGV after it was denied access to a shared memory region or a semaphore which had a SE Linux type of unconfined_t. I added some SE Linux auditallow rules and discovered that the memory region in question was created by the ssh client. Shortly after that I came to the conclusion that this wasn t some strange feature of ssh (or one of the many shared objects it uses) but hostile activity. The ssh client appeared to be storing passwords that it used in a shared memory region and sshd was also collecting passwords in the same region and presumably offering them to a ssh client which uses some extension to the ssh protocol. The sshd process was crashing because it couldn t handle EPERM on access to shared memory or semaphores. Presumably if the system in question wasn t running SE Linux then the exploit would have remained undetected for a lot longer. At this stage we don t know how the attacker got in. Presumably one of the people with root access ran a ssh client on a compromised system and had their password sniffed. One such client system was mysteriously reinstalled at about that time, the sysadmin of the system in question claimed to have no backups which made it impossible to determine if that system had been compromised. I believe that the sysadmin of the client system knew that their system was compromised, kept that information secret, and allowed other systems to become and remain compromised. The attacker made no good effort to conceal their presence, they replaced ssh, sshd, and ssh-add and didn t bother changing the Debian checksums so the debsums program flagged the files as modified. Note that I have kept copies of the files in question and am willing to share them with anyone who wants to analyse them. Steinar H. Gunderson has named this trojan Ebury [1]. Recovery By the evening of the 13th of September I had the system mostly working again. Jabber still isn t working because ejabberd is difficult to get working at the best of times, I am now investigating whether there is a better Jabber server to use, but as I don t use Jabber often this hasn t been a priority for me. Some of the WordPress plugins I use and all of the WordPress themes that are installed were outside the Debian packaging system, as I couldn t be sure that they hadn t been altered (because the people who wrote WordPress plugins don t keep old versions online) I had to upgrade to the newer versions. Of course the newer versions weren t entirely compatible so I had to use a different theme and I couldn t get all plugins working. Link Within no longer works, not that it ever worked properly [2], I wanted to try Outbrain again but their web site won t let me login (and they haven t responded to my support request). Does anyone know of a good WordPress plugin to provide links to related content? Either related content on my blog or on the Internet in general will be OK. Some people have asked me about the change in appearance of my blog. It was simply impossible (for someone with my PHP skills) to get my blog looking the same way as it did before the server was cracked. I think that the new look is OK and don t mind if people think it looks likw a VW advert VW make great cars, I was very satisfied with the VW Passat I used to drive. Future Plans I had bought some Yubikeys (USB devices that generate one-time passwords) [3] to control access to that server, if I had configured the software to use them then this might not have happened. The use of one-time password devices can prevent passive password-sniffing attacks. It would still allow active attacks (such as using ControlPath/ControlMaster options on the ssh client to allow a hostile party to connect later (EG the -M, -S, and -o ControlPersist options for the ssh client). It s a pity that there doesn t seem to be a way to configure the ssh server to disable ControlMaster. Conclusion It would be good to have some changes to sshd to allow more restrictions on what a client can request, as ControlMaster functionality isn t needed by most users it should be possible to disable it. SE Linux doesn t protect against a compromised client system or any other way of stealing passwords. It did do a good job of stopping Ebury from doing all the things it wanted to do and thus making me aware of the problem. So I count this as a win for SE Linux. Yubikeys are the cheapest and easiest way of managine one-time passwords. I had already bought some for use on the system in question but hadn t got around to configuring them. I have to make that a priority.

30 December 2011

Alexander Reichle-Schmehl: Release Critical Bug report for Week 52

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:
In Total:1586
Affecting Wheezy:960
Wheezy only:163
Remaining to be fixed in Wheezy:797
Of these 797 bugs, the following tags are set:
Pending in Wheezy:49
Patched in Wheezy:117
Duplicates in Wheezy:50
Can be fixed in a security Update:26
Contrib or non-free in Wheezy:9
Claimed in Wheezy:0
Delayed in Wheezy:8
Otherwise fixed in Wheezy:49
Ignoring all the above (multiple tags possible) 552 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 862 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

Next.

Previous.