Search Results: "ag"

13 August 2010

Jonathan McDowell: Not my usual sort of keys

In a week I'll be in California. I'm not packed, though I've been spending a lot of time trying to figure how exactly what I need to take, what I want to take, and what I really shouldn't take.

One item I've been uncertain about are keyboards. When Simon went to France for a year he took a UK keyboard with him, but then they have AZERTY which is substantially different. There are a number of US keyboards lying around the office so a few weeks ago I switched to one for work to see how it went. I think I'm bringing a stash of UK keyboards with me. Annoyances with the US keyboard:

Joey Hess: DebConf10 report: Eben Moglen

Long before I took the train up to New York for DebConf10, I was sure that Eben Moglen's talk, the followup to his earlier, excellent "Freedom in the Cloud", would be a highlight of the conference. Even though it was scheduled for an ugly 9 am slot the morning after that other certain highlight, the wine and cheese party. I was not disappointed. The amazing DebConf video team has already put up the video of "How We Can Be the Silver Lining of the Cloud". Embedded below for those with html5. If you watch one video from DebConf, this should be it. <video controls="controls" src="http://meetings-archive.debian.net/pub/debian-meetings/2010/debconf10/high/1252_How_We_Can_Be_the_Silver_Lining_of_the_Cloud.ogv"></video> (Note that the audio stops cutting in and out after he switches mics a few minutes in.) In this talk, Eben goes far past the earlier "Freedom in the Cloud" talk, changing the focus from scraping Facebook, to laying down a concrete, detailed vision of the "FreedomBox", and how it could be a game changer. After the first standing ovation I can remember at a DebConf, given to the first speaker in a suit I can remember at a DebConf, many of us ended up in this huddle in the hall. Which turned into an impromptu BoF in a nearby room, which led to this wiki page, which was followed by seemingly everyone buzzing about it for the rest of the conference, and a second BoF that probably exceeded its room's occupancy limit. Which led to ... ??
I will spend the next several months trying to find other things we can do to help make this suddenly happen. And believe me, to the rest of the world, this will happen very suddenly. We will disrupt a lot of games we would like to have disrupted.
-- Eben Moglen


My thoughts:

12 August 2010

Kartik Mistry: Report from MiniDebConf India 2010

* So, here is late report! Sometime back Praveen started discussion at DebianIndia mailing list about having MiniDebConf kind of conference here in India. MiniDebConfs are generally focused on development rather than at introduction to Debian . But, since this was our first attempt, out focus was more on to introduce Debian to student community and after writing this after 5 days I realize that we ve done good job! Many thanks to Joerg Jaspert for setting up Mini website: http://in2010.mini.debconf.org/ Praveen got bunch of sponsors, many thanks to CDAC and other sponsors for making event happen with testy food and tea that kept me awake. Now what really happened at Pune? My flight was nice. I went to guest house of COEP (the host college, 2nd old engineering college in India) and took some rest and called Praveen. He was already at computer lab and setting up Virtualbox and other things. We had two places hall and lab. Unfortunately, both were at different buildings and subway between them :) I discussed with couple of students and we went to dinner at late night. Pro. Abhijit who was kindly give permission to use lab and college hall was another source of energy. He is handling many FOSS events in college and probably only faculty who encourage students to use and learn FOSS. The Day 1 started with long lines of registration process. As we had around 200 registration, turn out of students were 100% on first day. Hall was almost full. Dr. Shashikumar from CDAC gave nice lecture about contribution which was well received. Praveen and Onkar then took the stage and gave nice introduction about Debian. I met Onkar first time. We had really nice discussion ranging from becoming Ubuntu MOTU to his Powerbook s adapter. After that Vikram took lively session on i18n and l10n in Debian and I took boring session on Contributing to Debian . Yet another boring session from me was Developer Toolbox . Packaging workshop by me at hall made many faces to yawn heavily while Onkar and Praveen handled it nicely at Lab. Main reason for yawning was Wifi and my Macbook s capacity to deal with ndiswrapper and crappy broadcom s drivers. (Note: I should get nice machine.) I met Sham first time. I met Kushal once again and felt energetic again! Day 2 As we expected number was very low compared to Day 1. So, we closed hall and moved entire stuffs to Lab only. Vikram took again nice session on Debian Edu . It was very interactive and I should learn something from his act of presentation. Another nice thing from Day 2 was discussion with BOSS Linux team. Hold on. Lets keep some old woes aside. They are basically doing what their bosses are telling. Developers were clueless about Sending patches to upstream. Their bugzilla had Zorro bug reported. I made promise to follow-up on this and set-up patches from boss at somewhere depending upon their response. That mail is my draft let me finish it by weekend. Pavi and others took video from session. It will be nice if someone will post it somewhere :) We then continued on discussing bugs and packaging area. It was amazing to see Praveen become DM during conference and also to see: upstream, maintainer and sponsor in one room. I really had nice food at dinners. Mango Mastani was nice. Beer was nice. How can I forget discussion with Ramki, Ninad, Pro.Abhijit, Praveen, Pavi and others after dinner? So lets work for MiniDebConf 2011 now!

Olivier Berger: Report on Debbugs New developments in the ongoing struggle against bugs by Don Armstrong at Debconf

Here are some notes I took about the lecture Debbugs New developments in the ongoing struggle against bugs by Don Armstrong at Debconf 10 (that I watched from the stream). Note that the slides are at : http://rzlab.ucr.edu/debian/debbugs_presentation_debconf10.pdf, and here, I ll only mention most noteworthy elements I ve noticed. There s a linear growth of number of bugs reported for several years, from the statistics made by Don. In average, there s same number of bugs filed / archived per day : Debian s not losing ground against bugs ;) A new affect command has been added, for bugs present in a package affecting another package. The summary command which can be used to quickly spot the summary of lengthy discussions deserves more love. Included summary -1 to send a new summary by email. The new local-debbugs package : can mirror your bugs locally and allow to search, show, etc. even offline. Lots of ideas for new features still in development (but lacking manpower). Interesting idea about action required sorting which exhibits different interests for triager, maintainer, user, to maximize productivity. I guess this should be implemented in clients of the BTS also. Interesting ideas about distributed bug tracking : sharing status with other bugtrackers (see other report), sideways syncing with other distributions, and making it available to upstream also. Note this was discussed in a subsequent BoF, which I ve already blogged about. Some elements taken from the Q&A : Overall, a very interesting presentation. Well worth watching the video recording. See also my report on the presentation by Christine Spang on Distributing bugtracking for Debian with SD which also discussed some debbugs things.

Rapha&#235;l Hertzog: How to make 110.28 EUR in one month with free software and Flattr

Flattr FOSS Logo
  1. Create an account on Flattr.com (mine is here);
  2. Submit the URL of your free software project to the Flattr directory with a good description and the proper tags (here s my entry for dpkg);
  3. Add a Flattr button on your website (I put one on wiki.debian.org/Teams/Dpkg);
  4. Inform your users and reach out to free software users using Flattr by registering your project in Flattr FOSS.
And I m not alone, Joey Hess made 25 EUR in one week only. And right now only 30K people are using Flattr. I expect this number to increase significantly now that invites are no longer required to open a Flattr account (see the announce on Flattr s blog). It s time to spread the word about Flattr FOSS! Flattr this Share/Bookmark 8 comments Support my work

Olivier Berger: My report after the BOF on distributed bugtrackers at DebConf

Here are my notes about the very insteresting BOF on distributed bugtrackers held at DebConf This is just some elements of report (some taken from the gobby edits, and written down by me after watching the video (unfortunately missing 2 mins of sound in the beginning of the video stream/recording)). This is not a full report of all the discussions, as some discussed elements weren t of interest IMHO. You ll find here many ideas of mine, links and pointers (resulting of our work in the context of the Helios project on similar topics), so this is in no way a report of what was said exactly during the BOF. Aspects of interest: Note that these topics may in principle be discussed on the Distributed bug trackers mailing list: http://kitenet.net/cgi-bin/mailman/listinfo/dist-bugs Tools / implementation ideas : Storage of repositories / databases of remote bugs : About such storage, and the RDF store option, here s a prototype of ours that exposes UDD s bug facts through a SPARQL interface, allowing to publish bug facts as RDF (setup in the frame of the Helios project). It includes both Debian and Ubuntu bugs, and allows interlinking them in a standard way (with Semantically described links in RDF). See http://testforge.int-evry.fr/d2r-server/resource/spackage/evolution for an example. The problem is not how to distribute (distributed database systems exist), but the semantics of bug properties, and the different implementations of interfaces of different tools : which standard to implement, or how many converters are necessary if no standard ? We believe that navigation/referencing/tracking of external bugs would be much easier if bugs were LinkedData citizens, i.e. they would be exposed with (standard) formats like Semantic Web ontologies served with Semantic Web standards (like RDF). The worst case is currently to have to perform web scraping on bugtracker interfaces. How to avoiding scraping ? A solution could be to publish RDFa / microformats using common standard semantics, directly from the bugtracker pages at bugs URLs (see my wishlist for debbugs on that subject : 590931: Would be great to integrate RDFa metadata into debbugs pages). Now some pointers to historical details on the same subject and Debian (old posts by Lars Wirzenius) : Overall, the discussion was quite interesting, but didn t reach any sensitive progress regarding the state of the art. Much needs to be worked on, mainly on standardizing APIs, semantic descriptions of bugs, and implementation in tools like SD for instance. Stay tuned for more progress.

Gerfried Fuchs: Why new Design isn't deployed yet

This blog post is actually a response to a talk that was given at the debconf last week. It was marga's talk about Making Debian Rule, again. Now that the beta versions of the talks are available for download I'm able to proper quote what I want to respond to: Is there a reason why we haven't yet updated the website?, perhaps someone can answer. I feel the need to respond to this as I am the main person driving that effort. Unfortunately the question was asked in a talk with no-one involved in that work around instead of addressing those people directly, and the question wasn't brought to my attention before the talk so that I could have provided an answer to offer in the presentation already. Also, as disclaimer, this is my personal story for it and doesn't need to get shared by the other people involved. I hope it can be seen as an answer anyway. As a little of background, Kalle's proposal went a long path before even I found out about it. I was simply in awe in several ways about it because it wasn't just a great mockup of the page (which it actually is, IMHO) but was accompanied by thoughts about not only the main site but also about several sub sites. Also, it was accompanied by patches, so it was seemingly ready to get deployed right ahead. I started working on it, got accused of being discouraging while doing that, got told that it would require a proper vote for a decision and can't happen just as, but those are just the Debian way of communication. I nevertheless did set up several test sites for some over the time to play with it. Some did work out better (like the git one, though this one came last and Kalle improved the CSS over time a fair bit), some weren't ready yet (like wiki, missing e.g. the coloring for the version diff, or the packages one where the separation between the different sections isn't that visible yet). Time passed, other things demanded their attention too. Like the thing where our system administrators did send me a request along about a new www-master server that needs to get set up from scratch. Given that the documentation about the required packages for the website was lacking a fair bit (to say the least) it did require a lot of attention, especially when doing things from scratch the wish to document the dependencies properly is just natural. There are still some issues with that as can e.g. be seen on this page in the empty table at the bottom. The same issue also appears on my testsite for the new design. The last part though is currently the biggest blocking issue for both efforts. There is no way to move forward with either without having that addressed. Simon Paillard did a great job on helping the move along so far, keeping the thread about the server move requirements that I started on the debian-www mailinglist updated with information about what's still pending. Unfortunately Kalle, Simon and me are also facing some private time constraints (amongst other duties that require attention) which weren't helping to move forward at a bigger pace. Unfortunately not very many people actually has shown interest in helping out neither, after all it's easier to go and ask questions about why it's not done yet to the completely wrong audience. My plan is to switch many sites at the same time instead of one after the other simply for the effect but also for less distraction for our users, including less people repeatedly pestering about when this or that site gets done, too. At least we need a final discussion on what should go into the header links and what should be at the bottom before a switch can be made anyway. And for some there is still the opinion that such a switch is nothing that we are rightfully allowed to decide without a vote, so that's also part of the reason to not just deploy it. I hope this answers the question to some extend and like said, if only I would had been addressed about this before the talk was given an answer to it would had been ready for presentation right ahead instead of having a pretty terse statement getting relayed through IRC (thanks, Yoe!) because I was fortunate enough to be around and watching the talk at that time. Feel free to discuss it further or offer your help on debian-www, especially if you are familiar with working on CSS files.

/debian permanent link Comments: 3 Flattr this

Olivier Berger: Great talk Distributed bug tracking for Debian with SD at DebConf

This is a quick report and notes about the great presentation made at DebConf on Distributed bug tracking for Debian with SD by Christine Spang (http://penta.debconf.org/dc10_schedule/events/591.en.html) Hopefully, the slides are gonna be published soon, but in the meantime, you have them in the video recording (see link above). For those who don t know SD, they may refer to a talk given by its author, Jesse Vincent, at the recent RMLL/LSM 2010 : Peer to peer issue tracking with SD and Prophet (video recording coming soon : EDIT: a premiere recording is available here). She draws 2 main use cases in her speech, which is exactly the kind of features we ve been thinking about for fetchbugs4.me : What would be even better is to be able to link all of these bugs to each-other but I m not sure this is yet possible. Christine started implementing debbugs (Debian s bugtracker) support in SD. For reference, there s a wishlist about it in 590156 : sd: Provide sync-ing with debbugs. The implementation is still in alpha state, in read-only mode only. Christine has implemented a new method getmachinereadablebuglog() for debbugs SOAP interface. It seems to send some Perl Data::Dumper output, which is probably fine for SD as SD is written in Perl, like debbugs but it may be better if it was some more generally accepted format, like JSON maybe. Among the topics discussed, was the absence of any standard API to connect to bugtrackers, meaning that SD needs to include many different client code modules for each of the trackers different APIs over different kinds of protocols. I ve raised (through my IRC comments, watching the presentation through the stream), that there s a proposal for such a standard that exists : OSLC-CM. Still, it s not widely implemented, in particular in Open Source tools (see another post of mine on the subject). I think it would be great if SD could be a client of OSLC-CM compatible trackers, hence I ve filed a wishlist. Also, I suggested that maybe it would be interesting to be able to browse the mail threads of comments on Debian bugs, much like with bts mbox show 12345 command (which opens mutt on the bugs thread, for me), when in offline mode, with SD. Maybe the bts command could just use SD as a backend for all its operations. Another very interesting feature may be to allow transfering a Debian bug to an upstream bugtracker, through SD. Dunno if that will be possible. Finally the code may be found : in github. I m looking forward to testing it.

David Bremner: Batch processing mails from caff

What is it? I was a bit daunted by the number of mails from people signing my gpg keys at debconf, so I wrote a script to mass process them. The workflow, for those of you using notmuch is as follows:
$ notmuch show --format=mbox tag:keysign > sigs.mbox
$ ffac sigs.mbox
where previously I have tagged keysigning emails as "keysign" if I want to import them. You also need to run gpg-agent, since I was too lazy/scared to deal with passphrases. This will import them into a keyring in ~/.ffac; uploading is still manual using something like
$ gpg --homedir=$HOME/.ffac --send-keys $keyid 
UPDATE Before you upload all of those shiny signatures, you might want to use the included script fetch-sig-keys to add the corresponding keys to the temporary keyring in ~/.ffac. After
$ fetch-sig-keys $keyid
then
$ gpg --homedir ~/.ffac --list-sigs $keyid  
should have a UID associated with each signature. How do I use it At the moment this is has been tested once or twice by one person. More testing would be great, but be warned this is pre-release software until you can install it with apt-get. I have a patched version of the debian package that I could make available if there was interest.

Alessio Treglia: Manage your font collection with Font Manager

Already available in both Maverick and Debian sid, it provides many interesting features. I m talkin about Font Manager, a small application written in C and Python by Jerry Casiano, which allows users to easily install, remove and compare font files on own system. Here are few nice screenshots:

To install the application, as usual type:

sudo apt-get install font-manager

Let me know what you think! ;)

Links

David Moreno: DebConf 10

I was in New York for three full weeks, before, during and after DebConf 10 took place. Unfortunately, I wasn't able to attend talks or go hang out with folks as much as I would've wanted. Most of the time I was busy at the office with a new product launch we have been preparing for the last few months. However, I was able to come and say hi to a few people, attend the C&W party and talk with friends and ran into. I was, however, following all conversations on IRC so I was pretty informed of all of what was going on. Some random thoughts: DebConf 10 rocked and I appreciate everybody's participation in it. See you in Banja Luka, next year!

Andrew Pollock: [life] Citizen Zoe

I'm totally gobsmacked by how fast this happened, but Zoe's certificate of Australian Citizenship arrived today. I only sent off the application on Saturday, and it would have arrived in Washington DC at the Australian Embassy on Monday morning. For us to get it back on Wednesday means they processed it either the same or next day and got it back into the mail on Tuesday. Pretty amazing, given that the form itself sets the expectation of up to 12 weeks, and Embassy-specific web page sets the expectation of 3-4 weeks. Zoe with her Australian Citizenship certificate Next step, get an Australian passport.

John Goerzen: A Debconf Thanks

There was something at Debconf10 in NYC that I owe a huge thank-you to the entire community for. I was completely surprised at how many people made a point to say hi to me, comment that they liked reading about my blog, and encouraged me to keep posting. It was dozens for sure. I had never really thought that many people took much of an interest in what was, to me, some sort of cross between a journal and a way for family to stay in touch. I deeply appreciate all the encouragement, and the welcome that was extended to Terah, Jacob, and Oliver when they were present at Debconf events a few times. Thank you so much to all my Debian friends I got to talk to at Debconf. It was a wonderful week and I hope to see you all again before too long. I am happy to be a part of this community.

11 August 2010

Romain Beauxis: Mingw32-ocaml 3.12.0

An updated version of the ocaml cross-compiler package, based on ocaml 3.12.0, has just been uploaded to Debian experimental ! Any report and test on the package would be very welcome ! I have personally tested it with Liquidsoap and built a win32 version of the software. Since this build implies many external modules as well as C objects, I am pretty confident in the cross-compiler uploaded to experimental.. About the cross-compiler: the ocaml cross-compiler is the result of the hard work done by Richard Jones for Fedora. The Debian package is merely a backport (and adaptation to ocaml 3.12.0) of his patches. If you care about the future of the cross-compiler, the best you can do is work with upstream to find how to push the needed changes there in order to have a plain support for it. I have personally no time for starting this process but I could try to describe the patches to an intereste contributor. Warning: some are REALLY hacky :-)

Julian Andres Klode: APT2 this time in C

As I wrote a few hours ago on deity@l.d.o (see http://lists.debian.org/deity/2010/08/msg00057.html), APT2 is back again. The first time, I tried Vala; but this time I wrote it in C (with the help of GLib, but no GObject) and the cache uses GVariant instead of an SQLite database. It s really basic at the moment (no solver, package installation/removal), but it will improve. Read operation should be faster than with APT, although writes are slower (this will be fixed by reusing unchanged parts of the cache). See the announcement for further information.
Filed under: APT2

Joerg Jaspert: Flight "fun"

I am currently sitting somewhere around Gate A2 at Singapore Airport, waiting for my connection flight. Not my original planned one. Not the one I got first rebooked to I was in Taiwan the last 3 weeks, having had about the best time ever. The plan for the flight back was pretty simple. Today, that is Tuesday 10th August, I take the Singapore Airlines flight from Taipei to Singapore, leaving Taipei at 1810h, arrival Singapore 22:35h. I wait a little time and then board the flight from Singapore to Frankfurt, leaving at 23:50h and touchdown in Frankfurt at 06:30h. So far so easy. So far not working at all. The flight from Taipei had technical problems (they later said problems with the computer of the engines ) and was delayed by a bit more than an hour. Not enough time to get the connection flight. So the ground staff in Taipei arranged a flight for me leaving Singapore at 13:50h the next day, Hotel and Taxi and such things to be organized by Singapore ground staff. Fine fine, so over to Singapore. Got here at 23:35h, left airplane at 23:45h. Definitely not working for the original connection, so the rebooking sounded good, When I left the airplane there was ground staff waiting for me, with the information that they got another reschedule done for me. I am now supposed to fly via Zurich, getting another connection from there to Frankfurt. So over to the transfer desk to get them to handle my luggage and actually get me a usable seat. Fine fine, all worked. Leaving the transfer desk, heading over to the gate (boarding meant to start at 00:10h, so not much time left), I have the pleasure to listen to an announcement: The flight to Zurich is delayed by approximately 2 hours. Oh fun, the planned connection flight in Zurich leaves about 2 hours after the planned arrival of this flight. Back to the transfer desk, asking whats up. Simple solution, I m supposed to fly to Zurich and contact the transfer desk there, it will deal with it. Lets see what happens and if I ever get back home. :) Nice side detail - I am now supposedly getting onto an A380. Lets see if this thing is really as great as they always say. Update: Now I am home. When we started in Singapore the Flight captain announced we have good weather helping us and will end up with being only 40 minutes late in Z rich. Wellwell, I think he just used up much more gas than usual, flying faster, but whatever, it worked out, 40 minutes late it was. (I cant believe always seeing 70mph of head wind being good ). So then I had to wait at that airport. And I must say: It sucks. Taipei airport - free wifi, just some funny redirections in browser before it works. Singapore - contact a staff member of the airport, get a code, works for 6 hours, fine. Z rich - the suckers want me to pay. A lot. Useless crapshit. Way to expensive. Bad service there. The next flight then finally got me to my flights destination, Frankfurt. Had to walk through half the airport to get to the luggage, but there it actually came out first. So fine. Train home after a short visit at my work was easy, and I think I will fall asleep soonish. Ohmy. Wish I would have stayed in Taipei

Russell Coker: Ethernet Interface Naming

As far as I recall the standard for naming Linux Ethernet devices has always been ethX where X is a number starting at 0. Until fairly recently the interface names were based on the order that device drivers were loaded or the order in which the PCI bus was scanned. This meant that after hardware changes (replacing network cards or changing the BIOS settings related to the PCI bus) it was often necessary to replace Ethernet cables or change the Linux network configuration to match the renumbering. It was also possible for a hardware failure to cause an Ethernet card to fail to be recognised on boot and thus change the numbers of all the others! In recent times udev has managed the interface naming. In Debian the file /etc/udev/rules.d/70-persistent-net.rules can be edited to change the names of interfaces, so no matter the scanning order as long as an interface retains it s MAC address it will get the correct name or at least the name it initially had. One of the down-sides to the way this operates is that if you remove an old Ethernet card and replace it with a new one then you might find that eth1 is your first interface and there is no eth0 on the system this is annoying for many humans but computers work quite well with that type of configuration. I ve just renamed the interfaces on one of my routers by editing the /etc/udev/rules.d/70-persistent-net.rules file and rebooting (really we should have a utility like /sbin/ip with the ability to change this on a running system). I have decided to name the Ethernet port on the motherboard mb0. The PCI slots are named A, B, and C with A being the bottom one and when there are two ports on a PCI card the one closest to the left side of the system (when viewed from the front the right side when viewed from the read) is port 0 on that card. So I have interfaces pcia0, pcia1, pcib0, pcib1, and pcic0. Now when I see a kernel message about the link going down on one of my ports I won t have to wonder which port has the interface name eth4. I did idly consider naming the Ethernet devices after their service, in which case I could have given names such as adsl and voip (appending a digit is not required). Also as the names which are permitted are reasonably long I could have used names such as mb0-adsl, although a hyphen character might cause problems with some of the various utilities and boot scripts I haven t tested out which characters other than letters and digits work. I may use interface names such as adsl for systems that run at client sites, if a client phoned me to report Internet problems and messages on the console saying things like adsl NIC Link is Down then my process of diagnosing the problem would become a lot easier! Does anyone else have any good ideas for how to rename interfaces to make things easier to manage? I have filed Debian bug report #592607 against ppp requesting that it support renaming interfaces. I have also filed Debian bug report #592608 against my Portslave package requesting that provide such support although it may be impossible for me to fix the bug against Portslave without fixing pppd first (I haven t looked at the pppd code in question for a while). Thanks to Rusty for suggesting this feature during the Q/A part of my talk about Portslave for the Debian mini-conf in LCA 2002 [1].

Bdale Garbee: TeleMetrum at Apogee Components

Exciting news! As of today, TeleMetrum starter kits are available through Apogee Components in addition to the Garbee and Garbee web store!

Gregor Herrmann: RCBC - results and prizes

At the beginning of DebCamp, Zack started the RCBC the Release Critical Bug squashing Contest. Two weeks later not only DebConf10 is over but we also have results for the RCBC:

In summary: It was a huge success, we managed to fix And the winners are: All details can be found at http://wiki.debconf.org/wiki/DebConf10/RCBC.

The winners who haven't collected their prizes at the closing ceremony will get them be snail mail.

Finally let me thank

Marco T&uacute;lio Gontijo e Silva: Immix on GHC Summer of Code report #11

My project. This post assumes that the reader has read my last post. I ve been trying to find a reasonable method (which computer to use, single user or multi user) to benchmark for some time, and now an e-mail from Simon Marlow calmed me down. He suggests that in this stage it s not necessary to run a long benchmark, but only a short one, that will be enought to have an idea about what each change is causing to performance. For some time I was struggling with the benchmarks to get a reasonable result. I ve started more than once to write a blog post analysing the results I ve got, until in the middle of the post I notice the results are not consistent. This usually happens when there s a big distortion in benchmarks that shouldn t present no change. For instance, in a patch that only changes code that is executed with +RTS -w -RTS, there was a difference of 50% on execution time in default mode, that is, without +RTS -w -RTS. I think that only disabling CPU scaling and running a small number of tests, which make it easier to reproduce a possibly inconsistent result, will be enough to get a good data. I ve made a stack of patches, splitting each part of my work. The stack does not reflect my preferences between the patches, that is, the complete stack applied is not what I think that is the best change to GHC. It s only a set of patches applied linearly, in a way that it s easy to see the impact in performance of each change. The patches are: Don t check for swept blocks in -DS.
The checkHeap function assumed the allocated part of the block contained only alive objects and slops. This was not true for blocks that are collected using mark sweep. The code in this patch skip the test for this kind of blocks.
This patch is only needed to make it possible to run the binary with the parameters +RTS -w -DS -RTS. I ve already explained this patch. I didn t measure the difference in performance before this patch, since it only changes code that is not going to be executed in the benchmarks, since none of them are runned with -DS. Immix: allocate and free memory on lines. This is the main patch with the initial implementation of Immix. As it is the initial implementation, it has it s problems, that will be treated in the following patches. The comparison was done using 3 programs from nofib/gc: fibheaps, fulsom and constraints, as suggested by Simon Marlow. Two tests were made for each version of the compiler code: one using the default GC strategie (copy-collection with mark-compact if there s few heap space left), another one using mark-sweep (or Immix after my changes). When comparing the default GC strategie between before and after this change, the program got 0.4% slower, with the collection of the generation 0 being 0.1% slower and the collection of the generation 1 0.4% slower. Comparing the mark-sweep with Immix, it got 13.7% slower, 15% in GC0 and 7.2% in GC1. The change in memory used is irrelevant. Chose between allocating in line or blocks in todo_block_full The first improvement I ve made is to change the place where the allocation was searching for free lines. Most of the changes were done in the function alloc_for_copy(), in rts/sm/Evac.c. This is bad because this function is called a lot of times, and should be kept fast. So I moved them to ctodo_block_full() in rts/sm/GCUtils.c. It got about 9.9% and 9.6% faster in default and sweep, which makes it 1% faster and 2.8% slower than in the original code. Improvements in sweep() I was bothered that the code I ve written to free memory in lines was messy, and I thought it could be simplified and maybe even turn faster. The general execution time increased 0.4%, but the GC1 time, which is the only one that should be affected, was reduced by 0.6%. In the default GC strategie, there was also a 0.2% increase in time, so I guess this is inside the margin of change. I have to investigate better on this one, but it seems to have improved slightly the performance. Line before inscresing block size. In a conversatio with Simon Marlow, we thought about these two options: one is to search for a line after trying to increase the block size, and that s what I did in Chose between allocating in line or blocks in todo_block_full; the other is trying to search a line first, and just increase the block size if a block (and not a line) was being used for allocation and there is no line available. With this patch I ve changed it to use the second one. Using sweep, the GC time got 0.5% slower, and with default it got 0.1%. rts/sm/Sweep.c: Mark all BF_MEDIUM blocks as BF_FRAGMENTED In my first implementation, the blocks that contain objects bigger than a line (of medium size) are marked with BF_MEDIUM, and are treated as the usual block in mark-sweep: if the block is empty, it s freed; if it s very fragmented, it s marked to be collected using copy-collection. In this patch, instead of making it use the rules of mark-sweep, I just mark it as fragmented and make it be collected by copy-collection. This made the GC0 time be reduced by 3%, but caused an increase of 509.7% in the GC1 time. This was also the first time the memory used was reduced, by 25.4%. The results in the default mode are insignificant. What it seems to me is that the copy-collection code is more efficient with memory usage, but also more slow. Marking objects as BF_FRAGMENTED make this algorithm be used even for full objects. rts/sm/Sweep.c: Don t use 3/4 heuristics to mark as BF_FRAGMENTED The mark-sweep algorithm considered that an object was fragmented if more than 1/4 of the word groups are completely unmarked. I kept using this heuristic, and this change removes it, making more use of the Immix, and less use of copy-collection. This didn t change the memory used, and made the code 10.4% slower in sweep. In default GC mode, there was no relevant difference, as expected. Allocate in lines in minor GCs. In the initial implementation I was only allocating on lines only in major GCs, because I needed the mark stack, which was only available in these kind of allocations. In this change, I created the mark stack on all GCs, and used the allocation on lines. The results for the default mode are insignificant, and the code got 9.0% faster in sweep, using the same memory. Selection I ve made a selection with the presumably better patches, which are: Don t check for swept blocks in -DS, Immix: allocate and free memory on lines, Chose between allocating in line or blocks in todo_block_full, Improvements in sweep() and Allocate in lines in minor GCs. This is an attempt to achieve the best set of patches possible, to see how it improved the original code. Comparing the default strategie for this selection and the original code, it got 3.7% slower. Comparing both sweep, it got 3.9% slower. Comparing the original default with the final sweep, it got 4.2% slower and uses 4.7% more memory. There s a lot of room for improvement, and I m willing to hear suggestions of what I could change in the code to achieve this improvement.
The results are available. Every time I had doubt about a comparison, I d run both versions again to check. This is way there are some backup files. The data presented here is not from all the most recent versions of each measurement, but from the ones that I thought were more similar in conditions.

Next.

Previous.