Search Results: "ron"

6 May 2011

Raphaël Hertzog: People behind Debian: Steve Langasek, release wizard

Steve Langasek has been contributing to Debian for more than a decade. He was a release manager for sarge and etch, and like many former release managers, he s still involved in the Debian release team although as a release wizard (i.e. more of an advisory role than a day-to-day contributor). Oh, and he did the same with Ubuntu: on the picture on the left, he just announced the release of Ubuntu 10.04 from his Debian-branded laptop. ;-) He has also been maintaining PAM in Debian for as long as can I remember and does a great job at that. He s very knowledgeable and fully deserves his place within the Debian Technical Committee. I m glad he still has the time to participate on several important Debian mailing lists because his contributions are always very useful. I m sure you ll notice this just by reading his answers below. My questions are in bold, the rest is by Steve. Who are you? I m 32 years old, have been running Linux since my first year in college back in 96, and have been a Debian developer now for ten years. Along the way I ve been involved in maintaining a variety of server packages, worked on the Alpha port for a while, did a stint as a release manager for a couple of years, and serve on the technical committee. This year I m also celebrating my ten year anniversary with my lovely wife Patty, who many know as an erstwhile front-desk volunteer at DebConf. God only knows why she puts up with my late-night hacking! These days in my day job I m a manager on the Ubuntu Platform team at Canonical, working to help make Ubuntu a daughter distribution that the Debian community can be proud of. What s your biggest achievement within Debian or Ubuntu? There s no doubt that my biggest achievement in Debian has been overseeing the release of two Debian releases as release manager. On the other hand, the scope of a release is so huge, and it represents the output of so many developers working together, that it would be arrogant to claim the release itself as an achievement of my own. Also, sarge and etch have long since been rotated off of the mirrors so no one cares about them anymore. ;) For a more personal and lasting contribution in the distro itself, I m very proud of writing pam-auth-update. It s a small piece of code, but one that Debian was missing for a long time it s made a big difference to PAM module integration in packages! What are your plans for Debian Wheezy? My top priority for this cycle is to see multiarch through. We re still not far enough along in Debian for most developers to see any difference and once we are, the first thing people are going to see is a fair bit of breakage when we start breaking a lot of assumptions about paths that have been hard-coded upstream. But I m still excited by the progress that is being made here. We should be able to ship wheezy without any ia32-libs package. We might even be able to get rid of all the biarch library packages, including those used by the toolchain itself. 54 packages in testing build-depend on gcc-multilib right now, in order to build 32-bit code to ship in the amd64 package; a bunch of those should go away with absolutely no reduction in functionality, saving us a bit of space in the archive and saving the maintainers a lot of complexity in their packages, while at the same time giving us much better support for cross-compilation than we ve ever had before. It s a tall order, certainly, but the pieces are falling into place one by one. My second priority is to get a policy in Debian around packages integrating upstart jobs. It would of course benefit Ubuntu to have many packages back in sync with Debian, but if all we wanted was to sync with Debian, we could mostly just make debhelper ignore upstart jobs in Debian, prefer them in Ubuntu, and call it good. I m interested in making sure Debian also gets the benefits of being able to use upstart, because as Linux has become increasingly asynchronous (doing more in parallel at start up), the traditional sysvinit has not been able to keep up. There are all kinds of bugs now related to network startup, for instance, that we don t have a good answer for in a sysvinit model but that we can fix with an event-based system. Upstart has been around for a while now, but we ve been slow to integrate it into Debian because it only works on Linux. It would be a shame if right after the first Debian GNU/kFreeBSD technology preview, packages all stopped working on kFreeBSD because they started to assume the availability of upstart! Unfortunately, having been so cautious we now have systemd on the scene, which not only doesn t support non-Linux but seems to be in the process of trying to gobble up other, non-Linux-specific components of the desktop stack. So I have to wonder what the future holds for the free desktop on non-Linux kernels. If you could spend all your time on Debian, what would you work on? Well, based on my previous experiences when I did spend all my time on Debian, I think the answer here is QA / release work. :) Otherwise, I don t know. My hands are full enough now with multiarch that it s hard for me to see what the Next Thing would be. You re a member of the technical committee. In the interview of Bdale Garbee, I have argued that it s not working well. What s your point of view on this topic? Well, I feel a constant low level guilt about my own poor level of activity in the TC; but that doesn t translate into a belief that the system is broken. This is, after all, the decision making body of last resort for technical disputes in Debian, and as such it should really be used sparingly. And if a reputation for glacial deliberation means more developers work out their disputes on their own rather than asking the TC to step in, I think that s actually a healthy thing! I do still wish we were more effective at resolving those issues that do come our way, but there s no silver bullet for this. Though the funny thing is, I ve noticed that the majority of issues that get referred to the TC nowadays never even need us to make a decision; a short conversation with the disputants is often enough to get them to come to an agreement. What s the biggest problem of Debian? By and large, I think Debian is still doing a great job at what it s best at delivering a rock-solid distribution that users can rely on. If I would highlight one problem in Debian, though, it would be that I think we re becoming less innovative as time goes on. Part of that comes from being such a large project that we re bound to be more conservative as an institution; but even though the three pet Debian projects of mine that I mentioned above are fairly innovative (multiarch, pam-auth-update, upstart), each of these has landed first in Ubuntu rather than in Debian. Always with a clear intent of pushing back up into Debian, of course, but it just wasn t possible to do this work within Debian for the first cut without much longer delays. I worry that if Debian is no longer the place to try new things, that we re going to miss out on attracting contributions from the folks who are inspired to make Free Software better and not simply to make it stable. I m not sure how to address this, though. Maybe improved conversations with derivatives such as (but not limited to) Ubuntu, about what crack of the day is being tried where and how that can be integrated into Debian once it s proven to work? I don t think that team-based maintenance or low-threshold NMUs do anything to address this, though, as the kinds of innovation that matter most are ones that require discussion and consensus-finding not just routing around inactive maintainers. Do you have wishes for Debian Wheezy? Well, I d like to see the armhf port get on its feet and become an official port. Over the lifetime of the arm and armel ports, the state of the art on ARM has changed quite a bit; it would be great to see Debian taking advantage of this richer platform, to let people make better use of their hardware via Debian. As a former release manager, you re now a release wizard . I guess you have seen it on debian-devel, there are proposals to not freeze testing and to use another distribution starting as a snapshot of testing to finalize the new stable release. According to your experience, what needs to happen to make this possible? Frankly, I ve stayed out of that discussion because I don t think what s being asked for is possible. I think proponents of a freezeless release have seriously underestimated the amount of work required on the part of the release team to wrangle testing into a releasable product, and that anything that makes propagation of fixes into the pending release more time consuming will make Debian worse on the whole, not better. If people really want to avoid long freezes for the Debian release, the best way they can help this happen is by making Debian more releasable on an ongoing basis, by helping to hold our packages to our shared standards for quality (i.e., by fixing RC bugs!). The biggest factor in long freezes for Debian is the slow rate at which we bring the RC bug count down during the freeze. Back in the sarge, etch days we used to have really great bug squashing parties that would get people together on weekends to hack through RC bugs by the dozens. I don t see that happening as much anymore. I d really like us to get back to that, but my few attempts at this so far since retiring as release manager have led me to think I m really terrible at organizing parties of any kind. :) On the other side, as seen at http://bugs.debian.org/release-critical/, the RC bug count for testing at the beginning of the release cycle keeps getting higher and higher. I d love to know why that is so we can address it. I know we ve gotten better at detecting some classes of RC bugs; that s part of it, but I don t think it explains the whole trend. Is there someone in Debian that you admire for their contributions? Wow, what kind of arrogant jerk would I be if I didn t admire anyone in Debian for their contributions? Debian is and always has been an amazing community of top-notch developers; there are certainly too many I admire to list them all here. Joey Hess certainly makes the list, for his longstanding example of code speaking louder than words and for his ability to get to the heart of common problems and come up with elegant solutions. So does Russ Allbery, who by all accounts had his ability to feel anger in response to email burned out of him at a young age in a flame-related accident on Usenet. ;-) The list goes on, but here I think I have to follow Joey s example and cut the words short.
Thank you to Steve for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

5 May 2011

Andrew Pollock: [life] Zoe at 12 months

I got in early this time and started working on this post a day early. Well, we've made it. We've all managed to survive parenting to the 1 year mark. Zoe continues to do fabulously. Today she had her 12 month paediatrician appointment, and weighed in at 19 pounds and 8 ounces (8.8 kg), which has brought her up from the 25th percentile for weight to around the 50th percentile. She's also up from the 25th to the 50th percentile for height. She got four shots today, which she was singularly unimpressed about even before she was jabbed. Somewhat surprisingly (to us) Zoe still isn't walking. She'll stand on her own if you distract her with something, but otherwise she'll fall down and prefers to crawl everywhere. She'll cruise around with the help of furniture, and she'll happily walk while pushing something. Go figure. I really thought she'd be walking into her 12 month appointment at the paediatrician. Developmentally, she's waving and pointing now. I'll usually get a wave goodbye out of her when I head out the door to work in the mornings. She seems to be picking up things pretty quickly now. One of her traits has been she'd always toss her sippy cup on the floor when she was done taking a drink from it. She's been doing this for months. In one sitting, I taught her to hold it out to me and say "ta!" instead, which is great, because now at mealtimes, I'm not picking up the sippy cup off the floor a dozen times. We had another trip back to Australia last month, and this one was purely vacation, so we had a nice couple of weeks of pottering around with family. Zoe again handled the flights very well, although the jet lag upon returning to the US seemed a lot worse than I remembered after our first trip. The pilot was pretty punchy with the seat belt sign on the way to Australia, so we had to keep taking Zoe out of the bassinet while she was asleep. Incredibly we were able to do this about four times before she woke up. She had a bit of a cold before we left for Australia, and that turned into a nasty cough, which was diagnosed as bronchitis the first week we were there. We got some antibiotics for her, and that cleared the cough up. She also had a couple of spontaneous vomiting incidents, which were out of character for her (but Sarah contracted some kind of tummy bug herself, so it wasn't entirely surprising). Despite these health issues, she was still in excellent spirits the whole time, and I think she thoroughly enjoyed herself. We went to Surfers Paradise for a couple of nights in the first week, and rented an apartment on the 23rd floor of the Chevron Renaissance, and pottered around the Gold Coast a little bit. We went down to Palm Beach, where we used to holiday every year when I was a kid, and caught up with my cousin who lives there, and Zoe had fun on the beach. That weekend, we held an early first birthday party for family and friends in a park and had a barbecue lunch. The following week we spent a few days on the Sunshine Coast, with some of Sarah's extended family, and pottered around on the beaches up there. Our flight back ended up being delayed 9 hours, but fortunately we found out before we got to the airport, so we had a nice, unexpected extra day in Brisbane. It also meant that we flew back for Zoe's night time sleeping time, which made for a pretty good flight (but made the jet lag worse for her once we got back). I wonder what the next year will bring us? At a guess, more teeth, talking and walking. Zoe at (almost) 12 months

Russell Coker: Osama bin Laden

After almost 10 years the Americans finally found Osama bin Laden, unfortunately they were unable to take him alive. The Reid Report has a good summary of what happened [1]. Sam Varghese wrote an interesting analysis of the political aspects of this event in Pakistan [2], and he s a lot less positive about it than most people. Later he wrote about the inconsistencies in the reports, it seems that bin Laden was executed while unarmed [3]. When dealing with someone who is no stranger to suicide attacks and who has stated an intention to never be taken alive it s not unreasonable to shoot quickly, it s just a pity that they couldn t have been honest from the start. Dr. Pamela Gerloff wrote an insightful article for Psychology Today titled Why We Should Stop Celebrating Osama Bin Laden s Death [4]. She asks the rhetorical questions What kind of nation and what kind of species do we want to be? Do we want to become a species that honors life? Do we want to become a species that embodies peace? and suggests that we should mourn the series of tragedies that led up to this situation and to feel compassion for anyone who, because of their role in the military or government, American or otherwise, has had to play any role in killing another . Daniel R. Hawes wrote an insightful article for Psychology Today that s quite different from Dr. Gerloff s article [5]. He is quite glad that bin Laden is dead. But he considers how the people who have lost friends and relatives on 9-11 might feel about this and says that the shouts and celebratory chants that rang around America today seemed to me to carry a certain element of irreverence for those affected most deeply by the September 11 attacks and the entire military operations that followed . To represent the people who are partying now there is a post by Jenny Lind Schmitt that is totally unworthy of a site such as Psychology Today [6]. Jenny told her small children about al Quaeda: They hate you because you are American. Their god is destruction, and they would kill you, a little child, if they had the chance, just because you are American. She also describes al Quaeda as lunatics . Describing all your enemies as lunatics is quite common in general conversation, but it s not suitable for a web site that discusses psychology (where some respect for people with mental health issues is expected). Teaching children to be bigoted against Islam is a bad thing too. It seems to me that a factor in the violence that comes from the middle-east is a result of women just like her telling their children similar things but with country names and religions switched. Finally some capacity for empathy is required for someone to have any insight into psychology, someone who can t understand such things can t be competent to discuss psychology. In stark contrast to Jenny (who bears an irrational hatred in spite of apparently not having any close connection to the events in question) there is a TED talk by Phyllis Rodriguez (who s son was killed in the 9-11 attacks) and Aicha el-Wafi (who s son is in jail for being a member of al Quaeda and was accused of being part of the 9-11 plot) [7]. The two women became friends after 9-11 and work together in the cause of peace. From reading some interviews that Google turned up Aicha seems like a very intelligent woman, I think that the TED talk with her section translated from French to English didn t seem to show this. Perhaps people who speak French and English would get more from the TED talk than I did. Also perhaps if she gave a talk to a French audience and it was subtitled then the result would be better. Now of course the less intelligent people on the right-wing are trying to spin this to say that Bush deserves credit for getting bin Laden. The Reid Report has a good analysis of the history of the hunt for bin Laden and it doesn t make Bush look good [8]. Also the Reid Report has an interesting analysis of the operation to get bin Laden and the possibility for peace now that he s gone [9]. LA Times has an article about the use of the name Geronimo as a code-word for bin Laden [10]. But compared to the use of words such as crusade and paladin by the US armed forces this hardly rates a mention.

Jeff Licquia: FHS Refresh

I ve been busy tonight spamming mailing lists and otherwise getting the word out: the LSB workgroup is preparing to update the FHS. This update has been a long time in coming; FHS 2.3 (the current version) was released back in 2004. Since then, a lot has happened, and it s starting to look like the FHS is holding things back due to the lack of updates. For the longest time, the FHS was cared for by its original editors: Dan Quinlan, Rusty Russell, and Chris Yeoh. We should all be grateful that they created a useful and well-written standard one that has been resilient enough to remain useful for six years without changes. Even though it s time to move on, we should not forget that we are building on a strong foundation they laid for us. So, you may be asking: how can I help? Glad you asked! We re tentatively shooting for a goal of releasing FHS 3.0 before July, though that s not written in stone. But we don t want to wait much longer than we ve already had to.

Anand Kumria: Django and Whiskey.

No, not programming whilst inebriated. Although the effects of the Ballmer Peak are well known. I mean using wsgi and Apache together with Django. But, perhaps, I should explain my (current) layout of projects. I now use:
 /path/to/project/
                 .git
                 .gitignore
                 project_name/
                             urls.py
                             settings.py
                             production_settings.py
                             [...]
                 apache/
                       project_name.conf
                       django.wsgi
                 env/
                    [...]
                 sitestatic/
                 requirements.txt
                 fabfile.py
This allows me to have revision control of things meta to the project and of the project itself. This setup is relatively new, I've only been using it in about 4 projects now. Which meant that my wsgi file had to be slightly modified. This took me a lot longer than I expected to get going. But if you decide to use a layout similar to mine, this might help.
import os
import sys
import site
## Assumptions:
# Assume that the Django project and the environment are at the same level.
# i.e.
# /home/
#      /<project root>/                                     <- *
#                     /<project name>/                      <- *
#                                    /apache/
#                                           django.wsgi
#                                    production_settings
#                     /env/
#                         /lib/
#                             /pythonX.Y/
#                                       /site-packages/     <- *
#                     /sitestatic
#
# Directories marked with '<- *' need to be imported for everything to work
PROJECT_NAME = "control"
PYVER = "%d.%d" % (sys.version_info[0], sys.version_info[1])
APACHE_DIR = os.path.abspath(os.path.dirname(__file__))
PROJECT_ROOT = os.path.abspath(os.path.join(APACHE_DIR, ".."))
site_packages = os.path.join(PROJECT_ROOT, 'env/lib/python%s/site-packages' % PYVER)
site.addsitedir(os.path.abspath(site_packages))
sys.path.insert(0, os.path.join(PROJECT_ROOT, PROJECT_NAME))
sys.path.insert(0, PROJECT_ROOT)
# import from down here to pull in possible virtualenv django install
os.environ['DJANGO_SETTINGS_MODULE'] = 'production_settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
One slight tweak I have also done is to rename the django.wsgi file, to be the environment file. e.g. devel.wsgi. That means the only hardcoded thing in the wsgi file is the PROJECT_NAME. Apply this patch on top of the file above:
--- apache/django.wsgi  2011-05-06 00:38:57.266316530 +0100
+++ apache/devel.wsgi   2011-05-05 14:23:33.708685680 +0100
@@ -24,14 +24,15 @@
 PYVER = "%d.%d" % (sys.version_info[0], sys.version_info[1])
 APACHE_DIR = os.path.abspath(os.path.dirname(__file__))
 PROJECT_ROOT = os.path.abspath(os.path.join(APACHE_DIR, ".."))
+ENVIRON_TYPE = os.path.basename(__file__).rstrip('.wsgi')
-site_packages = os.path.join(os.path.dirname(PROJECT_ROOT), 'env/lib/python%s/site-packages' % PYVER)
+site_packages = os.path.join(PROJECT_ROOT, 'env/lib/python%s/site-packages' % PYVER)
 site.addsitedir(os.path.abspath(site_packages))
 sys.path.insert(0, os.path.join(PROJECT_ROOT, PROJECT_NAME))
 sys.path.insert(0, PROJECT_ROOT)
 # import from down here to pull in possible virtualenv django install
-os.environ['DJANGO_SETTINGS_MODULE'] = 'production_settings'
+os.environ['DJANGO_SETTINGS_MODULE'] = '%s_settings' % ENVIRON_TYPE
 import django.core.handlers.wsgi
 application = django.core.handlers.wsgi.WSGIHandler()
If you have any suggestions for improvements, or thoughts on how to remove the PROJECT_NAME, let me know!

4 May 2011

Daniel Kahn Gillmor: test-driven development: refactoring, difficulties

liw recently wrote about test-driven development:
This all sounds like a lot of bureaucratic nonsense, but what I get out of this is this: once all tests pass, I have a strong confidence that the software works. As soon as I've added all the features I want to have for a release, I can push a button to push it out
To my mind, this is only one of the benefits. He doesn't describe another major benefit, which is the confidence with which you can take on re-factoring in projects with well-developed test infrastructure. If your project has no test infrastructure at all, and you make a deep and/or potentially invasive change, you might well produce something that is heinously broken for users who have a different pattern of using the tool than you do. But if you have a well-developed, reasonable test suite with fairly wide coverage, you can make a deep or invasive change, and be confident that -- if the tests all pass -- the stuff you release isn't going to be too horrific. And if you do break something with a change which the test suite didn't cover, that's an indication that the test suite is lacking. Hopefully, you can factor out a problem report into its own test, so that future changes will ensure that the behavior doesn't regress too. The upshot of more-confident re-factoring is that your development can be much bolder, you can roll out new features more quickly, and you can spend less time agonizing about whether you've got the various abstraction layers exactly right the first time through. These are all good things (though i do think the agony of abstraction perfectionism is well-warranted in some contexts, like API definitions, and wouldn't want test-driven development to make people give up on that necessary task). when testing can't cover everything Some things are just hard to test well. for example, if you have 20 different boolean options to a command-line tool, you can't realistically test them in all combinations and permutations; that would be over a million tests. User experience is also notoriously difficult to test, as are tools that rely heavily on network interaction, which can be flakey and unpredictable. other downsides Test suites themselves require maintenance, as the components they rely on can change over time. More than once, i've had a test suite failure that was really a failure of the test infrastructure, not the code itself. But in those same projects, that's usually followed or preceded by a test suite failure that picks out a particularly nasty or subtle bug in the tested code that might have persisted for quite a while unnoticed. So the test suite does in some sense create more work all around. more and better testing is good for Debian Even when we know that coverage isn't perfect, and even with its additional overhead, well-integrated tests (at the unit level and more generally) are worth the tradeoffs. It's worth doing because of the robustness and guarantees that we can give each other with regularly-tested code. It's a more stable foundation which surprisingly also gives us more flexibility going forward. This is good for free software in general. It also helps us find our bugs before our users do, so it's better for our users. So more and better testing directly supports the two main priorities outlined in the Debian Social Contract.

Gerfried Fuchs: Different Interpretations

Mostly everything in life boils down to the same troublesome issue: people are reading different things into what they read, and interpret them regularly in a way it wasn't meant to. It seems that in certain areas a culture of interpreting things in a bad way instead of good or asking how they were actually meant has established the rules of (not) working together but rather against each other and around each other. At times I would like to account it to language barriers, or cultural differences, but it happens with people from all areas so that reasoning would be too easy. Even artists manage to do that, and in that certain area it creates something extremely creative and thoughtful. This blog entry thus contains three songs and six videos: Two different interpretations of the same lyrics. Maybe this is able to stir some thinking process whether the interpretation that one found for a given situation might be biased or even just looking from the wrong angle. Enjoy! One thing I'd like to mention, and that is two cross references to former blog entries. For the first song, James Iha played as guitarist in The Smashing Pumpkins before he joined A Perfect Circle. The second cross reference is with respect to my former blog entry about the Wise Guys: They did also cover Mad World, in the Gary Jules' interpretation but of course in a capella.

/music permanent link Comments: 6 Flattr this

3 May 2011

Craig Small: IPv6 and address translation

We re in that in-between place where there is plenty of IPv4 out there, but sometimes you either need or want to get to IPv6 addresses. With the IPv4 address allocations out of ARIN now empty there is going to be an increased need of NAT either between the address families or within it. This is a quick overview of the different sorts of systems that do this address translations. 6tunnel The best way to describe this is port forwarding across the address families. You can have a IPv4 port on your local device that when you connect to it forwards the connection via IPv6 to a remote device, perhaps on a different port. You can also do it the other way, so connecting to a local IPv6 port will connect to a remote IPv4 port. Of course remote could be another port on the same computer. Squid You could replace squid with any other sort of proxy server that can work as a reverse proxy and understands IPv6. From the deployments I ve heard of, this arrangement where the IPv4 only webservers are front-ended with a dual stack IPv4/IPv6 squid (or other) reverse proxy server or set of servers. It is a real simple way of getting your webserver onto IPv6, if you cannot do it in the webserver itself. The usual advantages and disadvantages of using a reverse proxy apply here. The better long-term solution is to have the servers just run IPv6 natively, but this is a reasonable stop-gap. That s a brief overview of the various packages out there that can help with address translation to either get your servers understanding IPv6 or a client to get to an IPv6 server. Ideally these are only temporary measures but as some methods people use to write programs (the latest I ve seen has weird hard-coded 4 byte offset tree-table with 256 long list inside good luck fixing that!) they may be temporary for some time. Tayga This program implements NAT64 which is a 1:1 address translation. It can work both ways so IPv6 clients can connect to IPv4 servers and vice-versa. You would normally use this for your own hosts, rather than for connecting to the internet as you need to specify prefixes. It works like the old style static nat ranges in IPv4 where 1.1.1.10-100 would map to 2.2.2.10-100 tnat64 Now this package is a little back-to-front to the others in that it takes an existing IPv4 only application and makes it able to connect, at least at the socket level, to a IPv6 server. It works by preloading a library which overloads the socket functions like socket() and connect() so it can attempt to find IPv6 servers. totd This is a small DNS proxy which can be used to translate IPv4 A records to IPv6 AAAA records. You often use it with translation technologies like Tayga.
totd This is a small DNS proxy which can be used to translate IPv4 A records to IPv6 AAAA records. You often use it with translation technologies like Tayga.

Kai Wasserb ch: Taking over the maintenance of Chronicle

Steve Kemp recently announced his resignation from Debian. That left a few of his packages without a maintainer, one of them was Chronicle. As a Debian Developer using Chronicle, I've adopted this package. The upstream development will still be mostly done by Steve. The packaging is done in a Git repository. The 4.6-1 upload consists of a new upstream version and some housekeeping stuff on the packaging side (like switching to a minimal debian/rules, fixing some Lintian warnings and complying with the latest version of the Debian Policy). I'd like to take this opportunity and thank Steve for all his work for Debian. Hopefully he'll be back to Debian in the future.

Rapha&#235;l Hertzog: My Debian activities in April 2011

This is my monthly summary of my Debian related activities. If you re among the people who support my work, then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. GNOME 3 packaging Right after the GNOME 3 release, I was eager to try it out so I helped the pkg-gnome team to update some of the packages. I did some uploads of totem, totem-pl-parser, gvfs, mutter, gnome-shell, gnome-screensaver. I also kept people informed via my blog and prepared a pinning file for adventurous users who wanted to try it out from experimental (like me). One month later, I m still using GNOME 3. There are rough edges still, but not so many. And I m starting to get used to it. Debian Rolling planning Debian Rolling is a project on my TODO list for quite some time. I decided it was time to do something about it and started a series of articles to help clarify my ideas while getting some early feedback. My goal was to prepare a somewhat polished proposal before posting it to a Debian mailing list. But as usual with Murphy s law, my plan did not work out as expected. Almost immediately after my first post the discussion started on debian-devel:
At this point it s a discussion thread of several hundreds of messages (there are several screens of messages like the one above). Many of the sub-threads have been interesting, but the general discussions mixed too many different things so that there s no clear outcome yet. Lucas Nussbaum tried to make a summary. Obviously I must adjust my plan, there s lots of feedback to process. I accepted to drive a DEP together with Sean Finney to help structure the part of the discussion that focuses on allowing development to continue during freezes. But I m also eager to fix the marketing problem of testing and have the project recognize that testing is a product in itself and that end-users should be encouraged to use it. Package Tracking System maintenance The Package Tracking System is an important tool for Debian developers, and it has been broken by some change on the Bug Tracking System. I worked around it quite quickly so that few people noticed the problem but Cron kept reminding me that I had to properly fix it. I ended up doing it last week-end. While working on the PTS, I took the opportunity to merge a patch from Jan Dittberner to enhance the news RSS feed that the PTS provides. And I also integrated information from backports.debian.org (thanks to Mehdi Dogguy for reminding me #549115). Multiarch update Not much new this month. I fixed two bugs in the multiarch dpkg branch thanks to bug reports from Ubuntu users (LP 767634, LP 756381). I m still waiting on Guillem Jover finishing his review of the multiarch branch. I m pinging him from time to time but it looks like multi-arch is no longer in his short term priority list. :-( I ve been running this code for more than 2 months and it works fine. I want to see it merged. I m ready to update my code should anything need to be changed to please Guillem. But without any feedback we re in a deadlock. Misc dpkg work While fixing a bug in update-alternatives (found in one of the valid reports on launchpad), I noticed that there was room for improvements in the error messages output by update-alternatives. I changed them to reuse the same strings that were already used in other parts of dpkg. The result is that there are a few strings less to translate (always a nice thing for the poor translators who have to deal with the thousands of strings that dpkg contains). I also tried to fix some of the most cryptic error messages in dpkg (see #621763) but that work is stalled at the request of Guillem. Book update We (me and Roland Mas) are almost done with the update of our French book for Debian Squeeze. It will hit the shelves in July or September. I m starting to prepare the fundraising campaign to make an English translation of it. We ll use ulule.com for this. On my blog I have been pleased to interview Meike Reichle, it s the first women that I have interviewed in the series but it s certainly not the last one. I also interviewed Adam D. Barratt, one of our tireless release managers. Thanks Many thanks to the people who gave me 180.35 in March and 235.37 in April. That represents 1.5 and 2 days of work for those months. See you next month for a new summary of my activities.

8 comments Liked this article? Click here. My blog is Flattr-enabled.

Luke Faraone: Your release sucks.

I look forward to Ubuntu s semiannual release day, because it s the completion of 6ish months of work by Ubuntu (and by extension Debian) developers.I also loathe it, because every single time we get people saying This Ubuntu release is the worst release ever! .Ubuntu releases are always rocky around release time, because the first time Ubuntu gets widespread testing is on or after release day.We ship software to 12 Million Ubuntu Users with only 150 MOTUs who work directly on the platform. That s a little less than 1 developer with upload rights to the archive for every 60,000 users.1 Compared to Debian, which (at last estimate in 2010) had 1.5 million uniques on security.debian.org, yet has around 1000 Debian Developers.Debian has a strong testing culture; someone once estimated that around of Debian users are running unstable or testing. In Ubuntu, we don t have good metrics on how many people are using the development release that I m aware of (pointers welcome), but I d guess that it s a very very small percentage. A common thread in bug reports, if we get a response at all, goes on as follows:

Triager:2 Is this a problem in $devel?

User: I ll let you know when it hits final

Triager: It s too late then. Then we ll want you to test in the next release. We have to fix it BEFORE its final

User: Ok, I ll test at beta.

Triager: That s 2 weeks before release, which will be too late. Please test ASAP if you want us to have time to fix it Of course, there are really important bugs with hardware support which keep on cropping up. But if they re just getting reported on or around release day, there are limits to what can be done about them this cycle.We need to make it easier for people to run early development versions, and encourage more people to use them (as long as they re willing to deal with breakage). I m not sure whether unstable/testing is appropriate for Ubuntu, and I m fairly confident that we don t want to move to a rolling release (currently being discussed in Debian, summary). But we badly need more developers, and equally importantly, more testers to try it out earlier in the release process.To users: please, please try out the development versions. Download a LiveCD and run a smoketest, or check if bugs you reported are in fact fixed in the later versions. And do it early and often.

  1. This number, like all other usage data, is dated, and probably wasn t even accurate when it was first calculated
  2. Developer, bugcontrol member, etc. Somebody who is not experiencing the problem but wants to help.

30 April 2011

Russell Coker: Autism Awareness and the Free Software Community

It s Autism Awareness Month April is Autism Awareness month, there are lots of articles about Autism, most of them are at best misleading if not wrong. The most common error seems to be describing non-verbal Autistic children and then claiming that Autistic people comprise about 1% of the population. The number of non-verbal Autistic children is a lot less than 1% of the population, and at least 1/3 of the people on the Autism Spectrum have good verbal skills and often talk more than average. To be precise, they are describing the symptoms of people who would be diagnosed with Severity 3 according to the American Psychiatric Association s Diagnostic and Statistical Manual (the DSM) as applying to all people who are on the Autism Spectrum even though of the currently diagnosed people probably about 1/3 might be classified as Severity 2 and 1/3 as Severity 1. It seems that severity 1 Autism is going to be the replacement for Asperger Syndrome in the new DSM-5. The problem with misleading information in regard to what comprises Autism is that parents really don t want to have a child who s abnormal and search for evidence to make their wishes come true. At the extreme there are parents who can convince themselves that a 4yo non-verbal child isn t on the Autism Spectrum. When people are given the impression that Autism is only about non-verbal children then they will probably think that a 6yo who can give a half-hour lecture on a topic of interest (regardless of whether the audience is interested) isn t on the Autism Spectrum. There is also a similar issue with undiagnosed adults, I would have probably got assessed at least a decade earlier if I had learned more about Autism instead of believing some of the stories. In a quick Google search for Autism Awareness Month I haven t found any clear and accurate information on what Autism is or what should be done about it, so it seems to me that the awareness project has largely been a failure. The Autism page on Wikipedia [1] and the page for Asperger Syndrome [2] are good places to start reading. By merely pointing to those two Wikipedia pages I ve made this post a lot more useful than every article I ve seen in the mainstream media. To find some useful and insightful posts about Autism awareness you have to add some search terms such as neurodiversity, which turns up posts such as a post by Leah Jane about Autism Acceptance [3]. It seems that the Autism Awareness articles that are most useful at informing people will only be found by people who are already aware of some of the issues. Most People Who Read this are Geeks The majority of the readers for my blog come from aggregators that cater to fairly geeky people. Geek culture is largely Aspie culture. So I think that most people who read my blog have a good general knowledge of what people who might be diagnosed with Asperger Syndrome or High Functioning Autism are like they just think that such people are merely a bit more geeky than is typical for the Free Software community. So it s not as if all Autistic people are hidden away somewhere because they can t live independently. Many of us work in the computer industry and in academia. Probably the majority of people who have a university degree related to science or engineering have had a professor who was on the Autism Spectrum, probably most people who have any significant experience in the computer industry have had colleagues who are on the Spectrum. After having become used to the geek culture I initially found it difficult to believe that I could be on the Autism Spectrum because I knew heaps of people in the Free Software community who are at least as geeky as me but apparently weren t on the Autism Spectrum. I now think that the majority of people who are as geeky as me would get diagnosed with Asperger Syndrome (or High Functioning Autism under DSM-5) if they were assessed. Adult Diagnosis If you are wondering whether you might be on the Autism Spectrum then it s worth reading the pages linked from the DSM 5 draft process have the full diagnostic criteria (both for DSM-4 and DSM-5) [4]. Also Glenn Rowe has a web page with tests designed by Professor Simon Baron-Cohen [5] and Leif Ekblad has a project to do long-term analysis of changes to personality test scores which starts with a quick Aspie quiz [6] (and also has lots of links to other relevant sites). If you think that you might be on the Autism Spectrum (which means Asperger Syndrome in the case of most people who read my blog) then I recommend that you get assessed if possible. As assessment by a psychologist isn t cheap there is the option of self-diagnosis in which case Chaotic Idealism s post about self-diagnosis is worth reading [7].

Thomas Girard: ACE+TAO Debian packaging moved to git

We recently converted Debian ACE+TAO package repository from Subversion to git. This was a long and interesting process; I learned a lot on git in the course. I had been using git for a while for other packages: BOUML, dwarves and GNU Smalltalk. But I did not really get it. A preliminary study led by Pau[1] showed that out of the following three tools: the last one was giving results that look better.
The conversion svn-all-fast-export requires physical access to the repo, so the Alioth SVN repo was copied on my machine svn-pkg-ace/ before running the tool:
svn-all-fast-export --identity-map authors.txt --rules pkg-ace.rules svn-pkg-ace
Here's the content of the pkg-ace.rules configuration file that was used:
create repository pkg-ace
end repository
match /trunk/
  repository pkg-ace
  branch master
end match
match /(branches tags)/([^/]+)/
  repository pkg-ace
  branch \2
end match
The author mapping file authors.txt being:
markos = Konstantinos Margaritis <email-hidden>
mbrudka-guest = Marek Brudka <email-hidden>
pgquiles-guest = Pau Garcia i Quiles <email-hidden>
tgg = Thomas Girard <email-hidden>
tgg-guest = Thomas Girard <email-hidden>
The tool sample configuration file merged-branches-tags.rules recommends to post-process tags, which are just a branch in SVN. That's why the configuration file above treats branches as tags. The conversion was indeed fast: less than 1 minute.
Post-conversion observations Invoking gitk --all in the converted repo revealed different kind of issues:
  • svn tags as branches: http://thomas.g.girard.free.fr/ACE/tags-as-branches.png Branches are marked with green rectangles, and tags with yellow arrows. What we have here (expected given our configuration of the tool) are branches (e.g. 5.4.7-5) corresponding to tags, and tags matching the SVN tagging commit (e.g. backups/5.4.7-5@224). We'll review and fix this.

  • merged code that did not appear as such: http://thomas.g.girard.free.fr/ACE/missing-merge-metadata.png Branches that were not merged using svn merge look like they were not merged at all.

  • commits with wrong author: http://thomas.g.girard.free.fr/ACE/wrong-author.png Before being in SVN, the repository was stored in CVS. When it was imported into SVN, no special attention was given to the commit author. Hence I got credited for changes I did not write.

  • obsolete branches: http://thomas.g.girard.free.fr/ACE/obsolete-branches.png The tool leaves all branches, including removed ones (with tag on their end) so that you can decide what to do with them.

  • missing merges: http://thomas.g.girard.free.fr/ACE/missing-merge.png The branch 5.4.7-12 was never merged into the trunk!

Learning git Based on observations above, I realized my limited knowledge won't do to complete the conversion and clean the repository. There are tons of documentation on git out there, and you can find a lot of links from the git documentation page. Here's the one I've used:
The Git Object Model It's described with pictures here. You really need to understand this if you haven't already. Once you do, you understand that git is built bottom-up: the plumbing then the porcelain. If you can't find the tool you need, it's easy to write it.
git fast-import The Migrating to Git chapter explains how you can use the git fast-import tool to manually import anything into git. I've used it to create tags with dates in the past, slightly changing the Custom Importer example in the book:
#!/usr/bin/env ruby
#
# retag.rb
#
# Small script to create an annotated tag, specifying commiter as well as
# date, and tag comment.
#
# Based on Scott Chacon "Custom Importer" example.
#
# Arguments:
#  $1 -- tag name
#  $2 -- sha-1 revision to tag
#  $3 -- committer in the form First Last <email>
#  $4 -- date to use in the form YYYY/MM/DD_HH:MM:SS

def help
  puts "Usage: retag <tag> <sha1sum> <committer> <date> <comment>"
  puts "Creates a annotated tag with name <tag> for commit <sha1sum>, using "
  puts "given <committer>, <date> and <comment>"
  puts "The output should be piped to git fast-import"
end
def to_date(datetime)
  (date, time) = datetime.split('_')
  (year, month, day) = date.split('/')
  (hour, minute, second) = time.split(':')
  return Time.local(year, month, day, hour, minute, second).to_i
end
def generate_tag(tag, sha1hash, committer, date, message)
  puts "tag # tag "
  puts "from # sha1hash "
  puts "tagger # committer  # date  +0000"
  print "data # message.size \n# message "
end
if ARGV.length != 5
  help
  exit 1
else
  (tag, sha1sum, committer, date, message) = ARGV
  generate_tag(tag, sha1sum, committer, to_date(date), message)
end
graft points (graft means greffe in French) Because of missing svn:mergeinfo some changes appear unmerged. To fix this there are graft points: they override git idea of parents of a commit. To create a graft point, assuming 6a6d48814d0746fa4c9f6869bd8d5c3bc3af8242 is the commit you want to change, currently with a single parent 898ad49b61d4d8d5dc4072351037e2c8ade1ab68, but containing changes from commit 11cf74d4aa996ffed7c07157fe0780ec2224c73e:
me@mymachine$ echo 6a6d48814d0746fa4c9f6869bd8d5c3bc3af8242 11cf74d4aa996ffed7c07157fe0780ec2224c73e 898ad49b61d4d8d5dc4072351037e2c8ade1ab68 >> .git/info/grafts
git filter-branch git filter-branch allows you to completely rewrite history of a git branch, changing or dropping commits while traversing the branch. As an additional benefit, this tool use graft points and make them permanent. In other words: after running git filter-branch you can remove .git/info/grafts file. I've used it to rewrite author of a given set of commits, using a hack on top of Chris Johnsen script:
#!/bin/sh

br="HEAD"
TARG_NAME="Raphael Bossek"
TARG_EMAIL="hidden"
export TARG_NAME TARG_EMAIL
filt='
    if test "$GIT_COMMIT" = 546db1966133737930350a098057c4d563b1acdf -o \
            "$GIT_COMMIT" = 23419dde50662852cfbd2edde9468beb29a9ddcc; then
        if test -n "$TARG_EMAIL"; then
            GIT_AUTHOR_EMAIL="$TARG_EMAIL"
            export GIT_AUTHOR_EMAIL
        else
            unset GIT_AUTHOR_EMAIL
        fi
        if test -n "$TARG_NAME"; then
            GIT_AUTHOR_NAME="$TARG_NAME"
            export GIT_AUTHOR_NAME
        else
            unset GIT_AUTHOR_NAME
        fi
    fi
'
git filter-branch $force --tag-name-filter cat --env-filter "$filt" -- $br
(Script edited here; there were much more commits written by Raphael.)

Important

It's important to realize that the whole selected branch history is rewritten, so all objects id will change. You should not do this if you already published your repository.

The --tag-name-filter cat argument ensures our tags are copied during the traversal; otherwise they would be untouched, and hence not available in the new history.

Hint

Once git filter-branch completes you get a new history, as well as a new original ref to ease comparison. It is highly recommended to check the result of the rewrite before removing original. To shrink the repo after this, git clone the rewritten repo with file:// syntax -- git-filter-branch says it all.

Cleaning up the repo To recap, here's how the ACE+TAO git repo was changed after conversion:
  1. Add graft points where needed.

  2. Clean tags and branches. Using git tag -d, git branch -d and the Ruby script above it was possible to recreate tags. During this I was also able to add missing tags, and remove some SVN errors I did -- like committing in a branch created under tags/.

  3. Remove obsolete branches.

  4. Merge missing pieces. There were just two missing debian/changelog entries. I did this before git filter-branch because I did not find a way to use the tool correctly with multiple heads.

  5. Fix commit author where needed. Using the shell script above Raphael is now correctly credited for his work.

That's it. The ACE+TAO git repository for Debian packages is alive at http://git.debian.org/?p=pkg-ace/pkg-ace.git;a=summary.
[1]http://lists.alioth.debian.org/pipermail/pkg-ace-devel/2011-March/002421.html
[2]available in Debian as svn-all-fast-export

29 April 2011

Ian Wienand: Toolchains, libc and interesting interactions

When building a user space binary, the -lc that gcc inserts into the final link seems pretty straight forward link the C library. As with all things system-library related there is more to investigate. If you look at /usr/lib/libc.so; the "library" that gets linked when you specify -lc, it is not a library as such, but a link script which specifies the libraries to link, which also includes the dynamic linker itself:
$ cat /usr/lib/libc.so
/* GNU ld script
   Use the shared library, but some functions are only in
   the static library, so try that secondarily.  */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a  AS_NEEDED ( /lib/ld-linux-x86-64.so.2 ) )
On very old glibc's the AS_NEEDED didn't appear; so every binary really did have a DT_NEEDED entry for the dynamic linker itself. This can be somewhat confusing if you're ever doing forensics on an old binary which seems to have these entries for not apparent reason. However, we can see that that /lib/libc.so itself does actually require symbols from the dynamic linker:
$ readelf --dynamic /lib/libc.so.6   grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
Although you're unlikely to encounter it, a broken toolchain can cause havoc if it gets the link order wrong here, because the glibc dynamic linker defines minimal versions of malloc and friends -- you've got a chicken-and-egg problem using libc's malloc before you've loaded it! You can simulate this havoc with something like:
$ cat foo.c
#include <stdio.h>
#include <syslog.h>
int main(void)
 
   syslog(LOG_DEBUG, "hello, world!");
   return 0;
 
$ cat libbroken.so
GROUP ( /lib/ld-linux-x86-64.so.2 )
$ gcc -o -Wall -Wl,-rpath=. -L. -lbroken -g -o foo foo.c
$ ./foo
Inconsistency detected by ld.so: dl-minimal.c: 138: realloc: Assertion  ptr == alloc_last_block' failed!
Depending on various versions of things, you might see that assert or possibly just strange, corrupt output in your logs as syslog calls the wrong malloc. You could debug something like this by asking the dynamic linker to show you its bindings as it resolves them:
$ LD_DEBUG_OUTPUT=foo.txt LD_DEBUG=bindings ./foo
Inconsistency detected by ld.so: dl-minimal.c: 138: realloc: Assertion  ptr == alloc_last_block' failed!
$ cat foo.txt.11360   grep "\ malloc'"
     11360:	binding file /lib/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol  malloc' [GLIBC_2.2.5]
     11360:	binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol  malloc' [GLIBC_2.2.5]
     11360:	binding file /lib/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol  malloc' [GLIBC_2.2.5]
Above, because the dynamic loader comes first in the link order, libc.so.6's malloc has bound to the minimal implementation it provides, rather the full-featured one it provides internally. As an aside, AFAICT, there is really only one reason why a normal library will link against the dynamic loader -- for the thread-local storage support function __tls_get_addr. You can try this yourself:
$ cat tls.c
char __thread *foo;
char* moo(void)  
	return foo;
 
$ gcc -fPIC -o libtls.so -shared tls.c
$ readelf -d ./libtls.so    grep NEED
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 0x00000001 (NEEDED)                     Shared library: [ld-linux.so.2]
 0x6ffffffe (VERNEED)                    0x314
 0x6fffffff (VERNEEDNUM)                 2
Thread-local storage is worthy of a book of its own, but the gist is that this support function says "hey, give me an address of foo in libtls.so", the magic being that if the current thread has never accessed foo then it may not actually have any storage for foo yet, so the dynamic linker can allocate some memory for it lazily and then return the right thing. Otherwise, every thread that started would need to reserve room for foo "just in case", even if it never cares about moo. But looking a little closer at the symbols of libc.so is also interesting. libc.so doesn't actually have many functions you can override. You can see what it is possible to override by checking the relocations against the procdure-lookup table (PLT).
Relocation section '.rela.plt' at offset 0x1e770 contains 8 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
00000035b000  084600000007 R_X86_64_JUMP_SLO 00000000000a2100 sysconf + 0
00000035b008  02e000000007 R_X86_64_JUMP_SLO 0000000000075e50 calloc + 0
00000035b010  01dd00000007 R_X86_64_JUMP_SLO 0000000000077910 realloc + 0
00000035b018  029300000007 R_X86_64_JUMP_SLO 00000000000661c0 feof + 0
00000035b020  046f00000007 R_X86_64_JUMP_SLO 00000000000768c0 malloc + 0
00000035b028  000400000007 R_X86_64_JUMP_SLO 0000000000000000 __tls_get_addr + 0
00000035b030  01b400000007 R_X86_64_JUMP_SLO 0000000000076dd0 memalign + 0
00000035b038  086000000007 R_X86_64_JUMP_SLO 00000000000767e0 free + 0
i.e. instead of jumping directly to the malloc defined in the libc code section, any internal calls will jump to this stub which, the first time, asks the dynamic linker to go out and find the address of malloc (it then saves it, so the second time the stub just jumps to the saved location). This is an interesting list, seemingly without much order. feof, for example, stands out as worth checking out a bit closer why would that be there when fopen isn't, say? We can track down where it comes from with a bit of detective work; knowing that the value of the symbol feof will be placed into 0x35b018 we can disassemble libc.so to see that this address is used by the feof PLT stub at 0x1e870 (luckily, objdump has done the math to offest from the rip for us; i.e. 0x1e876 + 0x33c7a2 = 0x35b018)
000000000001e870 <feof@plt>:
   1e870:       ff 25 a2 c7 33 00       jmpq   *0x33c7a2(%rip)        # 35b018 <_IO_file_jumps+0xb18>
   1e876:       68 03 00 00 00          pushq  $0x3
   1e87b:       e9 b0 ff ff ff          jmpq   1e830 <h_errno+0x1e7dc>
From there we can search for anyone jumping to that address, and find out the caller:
$ objdump --disassemble-all /lib/libc.so.6   grep 1e870
000000000001e870 <feof@plt>:
   1e870:	ff 25 a2 c7 33 00    	jmpq   *0x33c7a2(%rip)        # 35b018 <_IO_file_jumps+0xb18>
   f19f7:	e8 74 ce f2 ff       	callq  1e870 <feof@plt>
$ addr2line -e /lib/libc.so.6 f19f7
/home/aurel32/eglibc/eglibc-2.11.2/sunrpc/bindrsvprt.c:70
Which turns out to be part of a local patch which probably gets things a little wrong, as described below. The sysconf relocation is from a similar add-on patch (being used to find the page size, it seems). libc, like all sensible libraries, uses the hidden attribute on symbols to restrict what is exported by the library. The benefit of this is that when the linker knows you are referencing a hidden symbol it knows that the value can never be overridden, and thus does not need to emit extra code to do indirection just in case you ever wish to redirect the symbols. In the above, it appears that feof has never been marked as hidden, probably because no internal glibc functions used it until that add-on patch, and since it is not considered an internal function the linker must allow for the possibility that it will be overridden at run time and provide a PLT slot for it. There are consequences; if this was on a fast-path then the extra jumps required to go via the PLT may matter for performance and it may also cause strange behaviour if somebody had preloaded something that took over feof. Note, this is different from saying that your library can override symbols provided by libc.so; such as when you LD_PRELOAD a library to wrap an open call. What you can not override is the open call that say, the internal libc.so function getusershell does to read /etc/shells. Having the malloc related calls as preemptable seems intentional and sane; although I can not find a comment to the effect of "we deliberately leave these like this so that users may use alternative malloc implementations", it makes sense so that libc.so is internally using the same malloc as everything else if you choose to use something such as tc-malloc. tl;dr? Digging into your system libraries is always interesting. Be careful with your link order when creating toolchains, and be careful about symbol visibility when you're working with libraries.

Steve Kemp: Goodbye, world.

Today I resigned from the Debian project. The following packages are up for adoption: I'll remove myself from Planet Debian tomorrow, assuming the keyring revokation isn't swift and merciless. ObQuote: This space is intentionally blank.

Rapha&#235;l Hertzog: Do we need project-wide support for Debian rolling?

The discussion about Debian rolling started sooner than expected on debian-devel (see the thread here). I initially wanted to iron out the biggest problems through discussions on my blog and try to submit a somewhat polished proposal instead we ended up discussing the same things both on -devel and on my blog. That said it s not that bad (except for the time I lost to have similar conversations in both places) because the debian-devel discussions included members of the release team and it looks like they are not fundamentally opposed to the idea. Despite this, the introduction of a frozen suite branched off from testing/rolling is not really consensual (yet?). But the idea of officially supporting testing on a best-effort basis appears to have almost no opposition. While some will undoubtedly believe that this is a useless exercise, I believe it would help if the project stated this in a somewhat official manner. My answer to the question in the title is thus: YES I am thus considering to submit a general resolution to that effect. My current draft is below.
Title: Debian endorses usage of testing by end-users, and renames it to rolling The Debian project recognizes that the Debian testing distribution initially created to make it easier to prepare and test the next stable release has become a useful product of its own. It satisfies the needs of users who are looking for the latest stable versions of software and who can cope (or even appreciate) a system that s constantly evolving. The Debian project decides to endorse this usage and will strive to provide a good experience to users of testing . To better communicate this policy change to our users, testing will be renamed rolling . While we believe that this is a good move, we would like to remind our users that Debian is a volunteer project and that our resources are not infinite. Package maintainers are contributing to Debian on a best-effort basis. This means that they might not be able to properly support their package(s) in all distributions. In that case, the project recommends that maintainers apply the following priorities:
  1. Support in stable (security updates, release critical bugs)
  2. Preparation of the next stable release
  3. Support in rolling
Note that this general resolution could have amendments with s/rolling/current/ and that would solve the bikeshedding over the name that started several times already. I deliberately separated Preparation of the next stable release and Support in rolling so that priorities are clear even if we decide to not freeze rolling and to introduce a frozen distribution to finalize the next stable release. I also did not go into too much details on the implications that it might have, it s best to leave that up to each contributor/team/etc. I hesitated to add a paragraph stating that we want to try to gradually improve the usability of testing but in the end I think it would be somewhat redundant. We re always trying to do our best when we decide to take on something. All comments welcome (even if you just agree and would be willing to second such a GR). Update: I will tweak the draft included in this article when I get good suggestions. Thanks to Lucas Nussbaum for the first one.

10 comments Liked this article? Click here. My blog is Flattr-enabled.

28 April 2011

Bastian Venthur: Installing Dropbox in Debian/Sid

A few months ago dropbox was a Debian packge in non-free. Recently I noticed that there are several new versions available and tried to notifiy the maintainer and ask him to update the package. But unfortunately it looks like the package is not available any more (Here s why). However getting Dropbox is quite easy if you know what to do. The obvious choice would be to go to dropbox.com and download either the source package and compile it yourself or try to install the precompiled Ubuntu packages. The first option is not really an option for most users and the second one does currently not work on Debian/Sid (aka unstable). However, there is an easy solution: on the download page search for the CLI script (currently it is named dropbox.py) Download that and run it with python dropbox.py start -i. That will download the propritary binaries into ~/.dropbox-dist. Now go into that folder and start dropboxd. There you go, you have a fully working systray app for dropbox. Depending on your desktop environment you might want to put dropboxd in your autostart . So you don t have to start the daemon every time you login. And that s it. It is not really ideal, since a Debian package would be much better for updates, security, yada yada, but it s still better than nothing.

Rapha&#235;l Hertzog: No freeze of Debian s development, what does it entail?

The main feature of rolling is that it would never freeze. This is not without consequences. Possible consequences It can divert developers from working on the release No freeze means developers are free to continue their work as usual in unstable. Will it be more difficult to release because some people will spend their time working on a new upstream version instead of fixing RC bugs in the version that is frozen? Would we lose the work of the people who do lots of NMU to help with the release? It makes it more difficult to cherry-pick updates from unstable No freeze also means that unstable is going to diverge sooner from testing and it will be more difficult to cherry-pick updates from unstable into testing. And the release team likes to cherry-pick updates that have been tested in unstable because updates that comes through testing-proposed-updates have often not been tested and need thus a more careful review. Frozen earth My responses to the objections Those are the two major objections that we ll have to respond to. Let s try to analyze them a bit more. It s not testing vs rolling On the first objection I would like to respond that we must not put testing and rolling/unstable in opposition. The fact that a contributor can t do its work as usual in unstable does not mean that he will instead choose to work on fixing RC bugs in testing. Probably that some do, but in my experience we simply spend our time differently, either working more on non-Debian stuff or doing mostly hidden work that is then released in big batches at the start of the next cycle (which tends to create problems of its own). I would also like to argue that by giving more exposure to rolling and encouraging developers to properly support their packages in rolling, it probably means that the overall state of rolling should become gradually better compared to what we re currently used to with testing. The objection that rolling would divert resources from getting testing in a releasable shape is difficult to prove and/or disprove. The best way to have some objective data would be to setup a questionnaire and to ask all maintainers. Any volunteer for that? Unstable as a test-bed for RC bugfixes? It s true that unstable will quickly diverge from testing and that it will be more difficult to cherry-pick updates from unstable into testing. This cannot be refuted, it s a downside given the current workflow of the release team. But I wonder if the importance of this workflow is not overdone. The reason why they like to cherry-pick from unstable is because it gives them some confidence that the update has not caused other regressions and ensures that testing is improving. But if they re considering to cherry-pick an update, it s because the current package in testing is plagued by an RC bug. Supposing that the updated package has introduced a regression, is it really better to keep the current RC bug compared to trading it for a new regression? It sure depends on the precise bugs involved so that s why they prefer to know up-front about the regression instead of making a blind bet. Given this, I think we should use testing-proposed-updates (tpu) as a test-bed for RC bug fixes. We should ask beta-testers to activate this repository and to file RC bugs for any regression. And instead of requiring a full review by a release manager for all uploads to testing-proposed-updates, uploads should be auto-accepted provided that they do not change the upstream version and that they do not add/remove binary packages. Other uploads would still need manual approval by the release managers. On top of this, we can also add an infrastructure to encourage peer-reviews of t-p-u uploads so that reviews become more opportunistic instead of systematic. Positive reviews would help reduce the aging required in t-p-u before being accepted into testing. This changes the balance by giving a bit more freedom to maintainers but still keeps the safety net that release managers need to have. It should also reduce the overall amount of work that the release team has to do. Comments welcome Do you see other important objections beside the two that I mentioned? Do you have other ideas to overcome those objections? What do you think of my responses? Does your experience infirm or confirm my point of view?

18 comments Liked this article? Click here. My blog is Flattr-enabled.

Jan Hauke Rahm: Save the environment and come to LinuxTag 2011 in Berlin

It s the time of the year again. LinuxTag is right up and, of course, Debian is part of it. But, it wouldn t be us if we didn t have any problems at all. Well, the situation isn t real bad but could deserve some improvement. Improvement about what, you ask? Isn t that obvious? It s man power that we need (women shall feel included :)). Now, LinuxTag 2011 is going to happen, with or without you, on May 11th to 14th. Needless to say that you ll be missing quite a lot if you don t attend. I mean, even Zack will be there. And so will I (most probably at least). And even more Who s willing to miss that, really? So, pack your stuff, get a train ticket or your car or your friend s car or the car of a friend of your friend who knows a friend who doesn t know you but still wants to give you his car which you don t even need since you already have the car of that friend of your friend and now you have two cars and can even bring another bunch of fellas who probably own cars themselves and now you re thinking about the environment because of all the cars that will come to Berlin and because of that you suddenly buy a train ticket and leave all three cars at home and in the end only one thing counts: you ll be there. Oh, and before you drive off to your local train station, please put your name into our neat little list so we can count on you as booth staff that s because we need man power in case you missed that section or somehow forgot it while thinking about that guy who is the friend of your friend that owns a car but rather comes by train as it saves the environment.

Patrick Schoenfeld: Directory-dependent shell configuration with zsh (Update)

For a while I've been struggling with a little itch. I'm using my company notebook for company work and for Debian related stuff. Now, whenever I switch between those two contexts, I had to manually fix the environment configuration. This is mostly related to environment variables, because tools like dch et cetera rely on some, which need to be different for the different contexts, like DEBEMAIL.
A while ago I had the idea to use directory dependent configuration for that purpose, but I never found the time and mood to actually fix my itch.
Somewhere in the meanwhile I applied a quick hack ("case $PWD in ...; do export...; esac") to my zsh configuration to ease the pain, but it still did not feel right.


For the impatient: Below you find a way to just use whats described here. The rest of the article just contains detailed information on how to implement something like this.


The other day I were cleaning up and extending my zsh configuration and it came to my mind again. I then thought about what my requirements are and how I could solve it. First I thought about using a ready solution, like the one in the Grml Zsh configuration, but at that point I did not remember it (it needed a hint by a co-worker *after* I finished the first version of my solution). Then I came up with my requirements:

It lead to a configuration approach as a first start. When thinking about how to represent the configuration I looked into the supported data types in zsh. zsh supports arrays, which is perfect for my need. I came up with something like that:
typeset -A EMAILS ENV_DEBIAN ENV_COMPANY
EMAILS=(
"private" "foo@bar.org"
"company" "baz@foo.org"
"debian" "schoenfeld@debian.org"
)
ENV_DEBIAN=(
"DEBEMAIL" "$EMAILS[debian]"
"EMAIL" "$EMAILS[debian]"
)
ENV_COMPANY=(
"DEBEMAIL" "$EMAILS[company]"
)
The next part was selecting the right profile. In the first version I used the old case logic, but it was breaking my separate logic and configuration paradigm. Approximate at this point the co-worker lead me to the grml approach, which I borrowed an idea from:


# Configure profile mappings
zstyle ':chpwd:profiles:*company*' profile company
zstyle ':chpwd:profiles:*debian*' profile debian

and the following code to lookup profile based on $PWD:

1 function detect_env_profile
2 local profile
3 zstyle -s ":chpwd:profiles:$ PWD " profile profile profile='default'
4 profile=$ (U)profile
5 if [ "$profile" != "$ENV_PROFILE" ]; then
6 print "Switching to profile: $profile"
7 fi
8 ENV_PROFILE="$profile"
9

For an explanation: zstyle is a zsh-builtin which is used to "define and lookup styles", as the manpage says, or put different: Another way to store and lookup configuration values.
Its nice for my purpose, because it allows storing patterns instead of plain configuration values which can be compared against $PWD easily with all of the zsh globbing magic. This is basically whats done in line 3. zstyle then sets $profile to the matching zstyle configuration in the :chpwd:profiles: context or to 'default' if no matching zstyle is found.

The (almost) last part is putting it together with code to switch the profile:

1 function switch_environment_profiles
2 detect_env_profile
3 config_key="ENV_$ENV_PROFILE"
4 for key value in $ (kvP)config_key ; do
5 export $key=$value
6 done
7
The only non-obvious part in this are lines 3 and 4. Remember, the profiles were defined as ENV_PROFILE, where PROFILE is the name of the profile. We cannot know that key in advance, therefore we have to construct the right environment variable from the result of detect_env_profile. We do that in line 3 and lookup this environment variable in line 4.
The deciding aspect for that is the P-flag in the parameter expansion. It tells zsh that we do not want the value of $config_key, but instead the value of $WHATEVER_CONFIG_KEY_EXPANDS_TO.
The other flags k and v tell zsh that, from the array, we want both: keys and values. If we'd omitted those flags it would have given us the values only.
We then loop over that to configure the environment. Easy, hu?

We would be finished, if this would do anything. The code above needs to be called. Lucky for us thats pretty easy to achieve, as zsh has a hook for when a directory is changed. Making all this work is simply a matter of adding something like this:

function chpwd()
switch_environment_profiles

Now, one could say, that the solution in the grml configuration has an advantage. It allows calling arbitrary commands on profile changing, which might be useful to *unset* variables in a given profile or whatever you could think of.
Well, its a matter of three lines to extend the above code for that feature:


# Taken from grml zshrc, allow chpwd_profile_functions()
if (( $ +functions[chpwd_profile_$ENV_PROFILE] )) ; then
chpwd_profile_$ ENV_PROFILE
fi

to the end of switch_environment_profiles and now its possible to additionally add a function chpwd_profile_PROFILE which is called whenever the profile is changed to that profile.


USAGE: I have put the functions into a file which can be included into your zsh configuration, which can be found on github.
Please see this README and the comments in the file itself on further usage instructions.

Next.

Previous.