Search Results: "xam"

31 May 2009

Ingo Juergensmann: IPv6: AnimatedExplanations and Ipv6 Act Now

For everyone who is wondering why IPv6 is a good idea, there are now two new websites that explain the benefits and background of IPv6. For example Geert Coppens from AnimatedExplanations.com wrote me last week that their site offers several short instructional/educational movies. One of those movies covers IPv6:


What is IPv6? - Animated Explanations

He writes as well:

The movie was made available with as purpose to spread the adoption of IPv6. If this movie is also of interest to you, feel free to embed or forward.


So spread the word - and movie! ;-)

The other site is IPv6actnow.org, which is operated by the RIPE NCC and deals with all expects of IPv6: why is IPv6 necessary, what are the benefits or how to migrate? There FAQs, Forums and video testimonials. So, if you need some infos for your boss to argue about the introduction of IPv6 in your network, be sure to visit that site as well!

Russell Coker: LG U990 Viewty

I have just got a LG U990 Viewty mobile phone [1]. It s a 3G phone and came free on the $29 monthly cap plan from Three (minimum monthly spend is $29 - but this is free if you use $29 per month). My previous plan was the $29 cap but had a minimum spend of $20 per month, as I never happened to use less than $29 per month I am not paying more. For a modern mobile phone the actual phone functionality is a sideline. If a device was strictly designed to be a phone then I think it would be very similar to the Nokia phones that were available 5+ years ago - the Nokia I had in 1999 performed every phone function that I desired of it. Like most modern phones the LG U990 Viewty suffers in it s phone functionality from the desire to make it do non-phone tasks and from the desire to cripple it to meet the desires of the carriers (not the desires of the users). For example the home screen will always have at least two buttons for paid Three services and I have no configuration option to remove them. Replacing them with speed dial options for a couple of numbers that I regularly call would be handy. As the main screen is a touch-screen there is no excuse for this, they should allow the software to be reconfigured with more useful options. Eventually the Android will kill most of the other phones and this problem of phones being designed to suit the telephone companies instead of the users will be solved. One of the most annoying mis-features of the phone is that it doesn t properly handle address book entries with multiple phone numbers. One of my friends turns his mobile off when he is at home, so I regularly call his mobile and then immediately call his home number if the mobile is unavailable. With my previous two mobile phones I could press the dial button to bring up the list of previous calls and then use the arrow buttons to select from the other numbers that are attached to the same address-book entry, so with three button presses I would be dialing his other number. With the Viewty I have to go back to the address book. The compelling feature of the Viewty is the camera and display. It has a 5MP camera which makes it the second highest resolution camera-phone offered by Three - the LG Renoir (KC910) has 8MP but needs a $99 plan for it to be free. It also has a 240*320 resolution touch-screen display (in a quick search in January when I bought my phone the best resolution display I could find on a phone is 240*400 in the LG Renoir). While the camera is documented as being 5mp there are no specs available about the resolution of the CCD. I want to use the native resolution of the CCD for pictures (I think interpolation is a waste of space). The CCD might actually be 5mp, a picture of the 1400*1050 resolution screen on my Thinkpad allows me to read all the text even when small fonts are in use, so the CCD resolution must be significantly greater than 1400*1050. This is a really important feature as the Viewty will work well for making screen-shots for bug reports about crashed computers (several of my clients have expressed interest in getting one after seeing such a demonstration). One annoying problem is that the camera software takes a while to load, my trusty Sony digital camera starts a lot faster and the LG U890 phone I used for the past two years is also a lot faster and more convenient. This won t work well for photographing unexpected events. When I am traveling by public transport I will photograph the relevant pages of my street directory as I can zoom in to photos of the maps and read the street names, it saves some weight when traveling. The 2G micro-SD card (which incidentally cost $10 from OfficeWorks) will allow me to store a lot of maps. One interesting feature is the video recording capabilities. It can do 640*480 resolution at 30fps (which is pretty good) and 320*240 resolution at 120fps (they claim that you can film a balloon popping). In my quick tests the standard 640*480*30fps mode works well, but the 120fps mode requires much brighter light than most of my test environments, so I have not yet got it working properly. The phone has a reasonable voice recording function, it can record considerably more than 34 hours of audio and the quality is reasonably good if you use an external microphone. It is however quite poor if you use the built-in microphone for a dictaphone function, it seems that quality is poor at any distance. I had wanted to record my LCA mini-conf talks with my phone but unfortunately forgot to bring the adapter for the external microphone. It s a pity that the phone doesn t have a standard microphone socket as I have misplaced my Viewty microphone, when designing the phone they should assume that misplacing attachments is a common occurrence and design it to use common parts. I recently spoke to a journalist who uses his mobile phone to record interviews. He said that his phone supported phone calls, voice recording, and taking pictures - all the essential tasks for his work. It seems that the Viewty would be better than most phones for journalistic work apart from the issue of low quality voice recording when you have misplaced your external microphone. The text editor is unfriendly in the keyboard mode (I have not tried hand-writing recognition), one thing I don t like is the fact that the letters jump when you press them. This does allow changing a letter by moving the stylus before releasing the press (some people consider this a great feature). There are no cursor control keys (which is a serious omission), and the keyboard doesn t resemble a real keyboard. My iPaQ is far better for writing (I once wrote a full-length magazine article on an iPaQ). The stylus is quite strange and interesting. In the picture of the front of the phone the stylus is compacted with it s lid on. In the picture of the back of the phone the stylus is extended with the lid off. The end of the stylus clips in to the lid so that removing the lid drags the central part out of the body. It s an interesting design and with the string on the lid allows the stylus to be attached to the phone when it s not being used. But I have never used it. Even with an iPaQ (which had a proper stylus that attached firmly inside the body of the device) I often used a fingernail on the touch screen. I have not felt the need to ever use a stylus with my Viewty. One final noteworthy thing is the support for Google services. It seems to have client support for YouTube, Google Maps, GMail, and Blogger. This seems to be a major win for Google, the Viewty is one of the most popular phones at the moment and I expect that lots of people who buy them will now have an incentive to use the Google services. Between these sorts of deals and the Android I think that it will be necessary to have some sort of anti-trust action against Google. Google are generally doing good things for the users. I have been quite satisfied to use Google search, Google advertising on my blog, and Gmail. Also I have been moderately happy with Blogger (it was good when I started blogging) and Google Maps is useful on occasion. So generally I am happy with Google, but monopolies are bad for the users so I think that if things continue on their current trend then Google may have to be split into several little Googlets some time in the next few years.

30 May 2009

Marc 'Zugschlus' Haber: The grub drama

This is a rant. A rant which goes to the grub maintainers, and one that could go nearly identically to many people in the KDE environment or many other open source projects. I really like grub. I really like grub 0.97 despite that it s been unmaintained for years and not booting on two of my important machines. I should like grub 2 because its configuration looks more straightforward and for its better features - direct booting of .iso images, from LVM and RAID. But actually, I have learned to hate grub 2 since it is not finished and badly documented, and that its existence is already being used as an excuse for grub 0 s development having stopped years ago (and it being renamed to grub-legacy to clearly show that it s the unloved child) - and things looks like this is not going to change any time soon. grub 0.97 is the de facto standard in booting Linux, its most prevalent advantages probably being its capability of reading filesystems and its command shell, which allows booting kernels and other software that one wasn t planning to boot when the system was shut down. There is a small fraction of people who still use lilo despite its disadvantages, for example on achitectures that are not supported by grub, if one wants to have the boot FS on LVM, if one just prefers lilo for non-technical reasons or on systems that grub doesn t work on. The last three systems, by the way, that I still run lilo on are Pyramid pizzaboxes with supermicro-made mainboards from 2002, which just refuse to boot if grub is used and the vendor just says that grub 0.97 is still beta and that they do not support grub on their systems: Use Lilo. Grub s last upstream release, 0.97, dates back to 2005, and upstream has stopped developing this grub release years ago. grub 2, the new way to go, had its first release in 2003, and the last stable release that people are supposed to use dates back to februar 2008. grub 2 is clearly a step in the right direction, but there are still bugs, and the most prevalent show stopper against grub 2 is the almost complete absence of any documentation. I have, for example, not yet found good documentation about the new config file format, which is a completely different deal from what we are used to with grub 0. The commands manual pages do little more than list the (obvious!) command line options without giving much more information about what the actually do, let alone usage examples. The documentation (which used to be really excellent for grub 0), has been moved to a wiki, which has some useful content but cannot keep up with the excellent documentation that grub 0 used to have. Additionally, grub2 does not seem to have a grub shell which can be called from within the host system, which makes installing grub 2 in exotic situations much harder than it used to be with grub 0. I am also quite disappointed that despite the grub upstream web pages claiming that grub 2 is under active development, the last release is more than fifteen months in the past while there are still so many shortcomings. The Debian maintainers of grub 2 are regularly packaging grub 2 subversion snapshots, which is a good idea in face of upstream s release policy which is quite different from what s being told in the wiki. I am, however, reluctant to rely on a development snapshot of a software to boot productive systems. Guys, you cannot declare a widely used, well-documented software legacy when the replacement is still in a nearly unuseable shape and the docs are hidden in the helpful files named .c and .h. This is simply unacceptable. It s a drama that the docs are still in such a pitiful shape fifteen months after your last release. But, grub is not the only free software project that so blatantly scores developer fun over the needs of the end user. I know that it is a boring task to fix bugs and to work around design shortcomings in the existing version of software, and that it is much more fun to hack away on new designs and new code, but dropping old versions before the new version is up to par with the old one is annoying the users and driving people away to other software packages. This applies to grub, a quite low-level program, but also to larger and more complex projects like KDE who has recently pulled the same stunt by dropping development and support for KDE 3.5 while KDE 4 is still missing many many features and KDE 3.5 s stablity, thus driving people (back) to GNOME or alternative desktop environments. This we give a damn about people who actually use our software, our compilers run without users as well attitude is a show of disrespect for the user base and makes me very sad as this does indeed show that free software is in no way superior to its commercial counterparts. Sadly, I do not have a solution for this issue since I know that free software developers are usually volunteers who do not have to report to anybody, and that they do not need to care for their user base. But I hope that people begin to reconsider their attitude and think about what kind of horrible damage is being done to the user base and to free software s reputation.

Francois Marier: Troubleshooting Postgres Performance Problems

There are great resources to help tune PostgresQL for performance, but say you've got a Postgres database which suddenly becomes very slow. How can you figure out what's going on?
General system stateA slow database server will typically be running out of:
So the first thing you should do is to get an overview of the system state using common UNIX tools such as:
If you notice that memory is exhausted, you could play with the following postgresql.conf settings:
shared_buffers
work_mem
effective_cache_size
If you see a lot of I/O, then try adjusting these settings:
checkpoint_segments
autovacuum_naptime
wal_buffers
Finding slow queries
On the database side, start by determining whether:
To do this, make sure that the following setting is turned on in your postgresql.conf:
stats_command_string = on
Then fire up psql dbname and look at the currently executing queries:
SELECT usename, current_query, query_start
FROM pg_stat_activity;
(You can also add the procid column to the query if you want to map a query to the process ID you see in top.)

If you can't get anything useful out of pg_stat_activity, you may want to enable slow query logging by adding this to your postgresql.conf:
log_min_duration_statement = 100
(All queries which take more than 100 ms to execute will be printed to the main Postgres log file.)
Examining a particular queryOnce you have identified a slow query, you can time it by running this command before executing the query manually:
\timing
To get an idea of where Postgres spends its time when executing that query, look at the query plan:
EXPLAIN your_query;
The numbers you see there are estimates. To run the query and get actual numbers, use this instead:
EXPLAIN ANALYZE your_query;
If you still can't figure out why a certain query takes so long, have a look at the ratio between live and dead rows in the relevant tables:
ANALYZE VERBOSE tablename;
Having too many dead rows is often a sign of insufficient vacuuming. You might want to turn autovacuum on if it isn't already, or to make it a bit more aggressive by tweaking these settings:
autovacuum_vacuum_scale_factor
autovacuum_analyze_scale_factor

Troubleshooting database performance is a bit of a black art, many thanks to Mark Kirkwood for sharing his Postgres wisdom with me.

If there's anything else you have successfully used to find the cause of your performance woes, please feel free to leave a comment.

28 May 2009

Sandro Tosi: Update on the Matplotlib book

Some people asked me how's the book writing is going. Ah you're right, it's a loooong time I didn't say anything of the progress, and here it is.

I've just turned the half of the book: about 10 days ago I've sent the 5th chapter (out of 10) and up to now I've covered this (high-level) contents by chapter:

  1. installation & setup
  2. first contact with mpl
  3. some more stuff (like graph types)
  4. OO style and some advanced things
  5. GTK embedding
6th (and current) chapter is about Qt, so even KDE guys will be happy :)

I found several problems when working with GUI design programs: both Glade and Qt Designer made me scream a lot; but probably it's just me that I'm not so used to GUIs :)

Other arguments will be: Wx, web, real use cases.

In particular for the last part (real cases) I'd like to hear some proposals from you. I've already got something in mind, but users opinion will help me direct my work better.

Sorry, I've got to "drop/reduce" the science chapter (it was superseded by the real use-cases one) since I don't think this is the right place for it. Of course, a couple of "science examples" might come into the mentioned chapter, but your proposals have to be at a low-medium level.

27 May 2009

Biella Coleman: Journal of Legal Anthropology

Something I have been thinking a lot about lately is why social science and humanities journals have been slower to move toward the land of open access in comparison to the hard sciences. There are a few obvious reasons but there are others which are still a mystery to me. I understand why existing journals can t easily pry away from established relations and obligations so I am not all that surprised that these journals, whether in the hard sciences or soft , have not gone open access. But perhaps newly minted journals are in better position to start right off the bat with an OA agreement. This is what the International Journal of Communication recently did and I am sure there are other examples. So today I was disappointed to find out that the following new Anthropology journal theJournal of Legal Anthropology has seemingly gone down a traditionalist copyright route. But I don t just want to point fingers here as I know editors are often in a very difficult position when seeking sponsorship and support for a new journal. That is, achieving OA, I understand is no walk in the park. And yet given their mission and given that it is a legal journal, it also makes sense to have some sort of open format:
International in scope, we hope it will be accessible beyond a specialist legal anthropology area and, in practice, both widen what is understood within the discipline of anthropology as legal and position the legal as also socio-cultural in terms of contemporary anthropology. The journal is produced by anthropologists interested in making anthropology accessible (translatable) in other settings and disciplines, and by legal practitioners with support from academics working in human rights, conflict and related areas
A walled garden is not suited for the flowers of access to grow. But perhaps they tried and failed. If this is the case, it would also be good to learn of these experiences, which can be used in future cases to pave the path toward greater access.

Biella Coleman: Journal of Legal Anthropology

Something I have been thinking a lot about lately is why social science and humanities journals have been slower to move toward the land of open access in comparison to the hard sciences. There are a few obvious reasons but there are others which are still a mystery to me. I understand why existing journals can t easily pry away from established relations and obligations so I am not all that surprised that these journals, whether in the hard sciences or soft , have not gone open access. But perhaps newly minted journals are in better position to start right off the bat with an OA agreement. This is what the International Journal of Communication recently did and I am sure there are other examples. So today I was disappointed to find out that the following new Anthropology journal theJournal of Legal Anthropology has seemingly gone down a traditionalist copyright route. But I don t just want to point fingers here as I know editors are often in a very difficult position when seeking sponsorship and support for a new journal. That is, achieving OA, I understand is no walk in the park. And yet given their mission and given that it is a legal journal, it also makes sense to have some sort of open format:
International in scope, we hope it will be accessible beyond a specialist legal anthropology area and, in practice, both widen what is understood within the discipline of anthropology as legal and position the legal as also socio-cultural in terms of contemporary anthropology. The journal is produced by anthropologists interested in making anthropology accessible (translatable) in other settings and disciplines, and by legal practitioners with support from academics working in human rights, conflict and related areas
A walled garden is not suited for the flowers of access to grow. But perhaps they tried and failed. If this is the case, it would also be good to learn of these experiences, which can be used in future cases to pave the path toward greater access.

25 May 2009

Michael Prokop: Boot an ISO via Grub2

With grub2 you can directly boot an (iso9660) ISO using its loopback option. This is great because it provides another nice rescue scenario - for example when using grml (the Debian based Linux Live-CD for sysadmins). You no longer need to extract kernel and initrd from the ISO to be able to boot it using the isofrom bootoption. All you need to do is put a plain grml ISO to your harddisk. I ve tested it with grml-small 2008.11 and grml-small 2009.05-rc1 using grub2 from Debian/unstable (1.96+20090317-1). In the following example the ISO file grml-small_2008.11.iso is available in directory grml on device /dev/sda1. You can boot it from the interactive grub shell using:
loopback loop (hd0,1)/grml/grml-small_2008.11.iso
linux    (loop)/boot/grmlsmall/linux26 isofrom=/dev/sda1/grml/grml-small_2008.11.iso boot=live quiet vga=791 noeject noprompt
initrd   (loop)/boot/grmlsmall/initrd.gz
boot
So if you are using grub2 already consider adding an entry like the following to your grub config (/boot/grub/grub.cfg):
menuentry "grml-rescue system from harddisk (ISO = grml-small_2008.11.iso)"  
  loopback loop (hd0,1)/grml/grml-small_2008.11.iso
  linux    (loop)/boot/grmlsmall/linux26 isofrom=/dev/sda1/grml/grml-small_2008.11.iso boot=live quiet vga=791 noeject noprompt
  initrd   (loop)/boot/grmlsmall/initrd.gz
 
This entry gives you the option to boot grml from your harddisk even if you don t have the possibility to boot via CD/USB/PXE/ . Tip: starting with grml 2009.05(-rc1) you can also use findiso=/path/to/grml.iso instead of isofrom=/dev/sdX/path/to/grml.iso. findiso is a dynamic version of isofrom as it looks for the specified ISO file on all disks where it usually looks for the .squashfs file (so you don t have to know the device name compared to isofrom= ). The above configuration using findiso is even easier to set up and would look like this:
menuentry "grml-rescue system from harddisk (ISO = grml-small_2008.11.iso)"  
  loopback loop (hd0,1)/grml/grml-small_2008.11.iso
  linux    (loop)/boot/grmlsmall/linux26 findiso=/grml/grml-small_2008.11.iso boot=live quiet vga=791 noeject noprompt
  initrd   (loop)/boot/grmlsmall/initrd.gz
 

24 May 2009

Jan Wagner: [vacation] 30/05/09 - 14/06/09 Manerba del Garda

I m on vacation the 1rst two weeks of June in Manerba del Garda at Lago di Garda. If some dudes are around for beering / chatting / keysigning, feel free to contact me til Friday.
In case of urgent actions (for example #530271), I m without regular internet access there. Feel free to NMU if needed, but try to avoid breaking things. For secuirty related issues please coordinate with team@security.debian.org.
Since I m probably most of the rest of the week busy with organising our camping, please send a short ping, if I m not responding by time, please go ahead.

Christian Perrier: News from the samba packaging team

Some (not so) short news from the samba packaging team... Recent package uploads:
  • Last "stable" upstream branch: 3.3.4. 3.3.4-2 is in unstable, quietly ageing before entering testing. 3.3.4-1 is in testing. In short, we have the latest upstream in squeeze
  • Next upstream version: 3.4.0~pre1 is in experimental. That includes many packaging changes as upstream is now providing samba3 and samba4 altogether, which lead to source tree reorganization. Good news: there are no more "FHS" patches and upstream included needed changes for the file layout to finally be able to cope with the FHS entirely.
  • We maintain many backports:
    • 3.3.4 was just uploaded to lenny-backports on backports.org. Maybe soon to come a 3.3.4 upload for etch-backports
    • unofficial backports on http://pkg-samba.alioth.debian.org: here we will maintain "the latest upstream version of the same upstream branch than what we have in Debian stable suites". That seems complicated to say and an example says it better: lenny has 3.2.5, so this repository has upstream 3.2.11 with the same packaging than 3.2.5. We made an exception for etch: instead of backporting the latest 3.0 upstream (etch has 3.0.24, upstream has 3.0.34), we backported the latest 3.2: we consider that nobody is now seriously considering to run 3.0, except those people who are very happy with the official packages in etch.
    • All these backports will be useful when tracking down bugs and ask our users if they reproduce them with the latest upstream versions, as samba setups are often very tricky to reproduce in our own environments.
Other things are coming, such as the work on having samba3 and samba4 packages coexist. Jelmer Vernooij prepared everything after our discussions at the SambaXP conference and it should be uploaded "soon". A few patches we have should be integrated upstream as well, therefore making the Debian diff smaller and smaller. Also, Luk Claes joined the maintenance team and helped setting up the unofficial repository for backports. The bug count is going up again these days as I don't spend much time triaging the bugs and some are very tricky to reproduce. As usual, help would be welcomed. I'm fairly sure that several of these are user errors but I often lack time to prove this enough for closing the bug. We should go to to 50-60 bugs or so.

Steve Kemp: Everything is different, but the same.

I've successfully upgraded my primary web/mail/misc host from Debian Etch to Debian Lenny. There were a few minor problems, but on the whole the upgrade was as painless as I've come to expect. In the past I'd edited my Exim4 configuration to add quite a few ACL checks, for example rejecting mails based upon spoofed/bogus HELO identifiers, and rejecting messages that didn't contain "Subject" or "Date" headers. The Debian Exim4 configuration may be split into multiple files (which is how I prefer it on the whole). The idea that you just add new files into the existing hierarchy and they'll magically appear in the correct location when a real configuration file is generated. On the whole this works well, but sometimes editing files in-place is required, and it was these local edits that caused me pain. Fixing things up was mostly not a challenge, primarily it was a matter of removing ACLs until exim4 started without errors - all my spam checking is handled ahead of exim4 these days, except for the last-ditch spam filtering with a combination of procmail-fu and the crm114 classifier package. Taking a hint from Bubulle's weblog I decided to nuke my CRM114 spam database to avoid any possible version-mismatch issues so now I'm having to classify a lot of "unsure" messages. Happily my memory of doing this last time round is that the initial training of spam/ham takes a day or so to complete. Anyway now I can start looking to advantage of the things new to Lenny. But probably not until I'm sure things have calmed down and upgraded correctly.
steve@skx:~$ uptime
 05:00:31 up 260 days, 14:23,  2 users,  load average: 0.95, 0.51, 0.31
steve@skx:~$ cat /etc/issue
Debian GNU/Linux 5.0 \n \l
ObFilm: Bill & Ted's Excellent Adventure

23 May 2009

Joey Hess: website backup RFC

So, I'm considering a backup system that has as its goal to let you backup all your data from third party websites, easily. The idea I have for its UI is you enter in the urls for all the sites you use, or feed it a bookmark file. By examining the URLs, it determines if it knows how to export data from each site, and prompts for any necessary login info. The result would be a list of sites, and their backup status. I think it's important that you be able to enter websites that the system doesn't know how to handle yet; backup of such sites will show as failed, which is accurate. It seems to almost be appropriate to have a web interface, although command-line setup and cron should also be supported. For the actual data collection, there would be a plugin system. Some plugins could handle fairly generic things, like any site with an RSS feed. Other plugins would have site-specific logic, or would run third-party programs that already exist for specific sites. There are some subtle issues: Just because a plugin manages to suck an RSS feed or FOAF file off a site does not mean all the relevant info for that site is being backed up. More than one plugin might be able to back up parts of a site using different methods. Some methods will be much too expensive to run often. As far as storing the backup, dumping it to a directory as files in whatever formats are provided might seem cheap, but then you can check it into git, or let your regular backup system (which you have, right?) handle it.
Thoughts, comments, prior art, cute program idea names (my idea is "?unsilo"), or am I wasting my time thinking about this?

Diego Escalante Urrelo: Python common mistakes

When starting with Python or just when you haven t read PEP 8 you usually fall into some frequent mistakes. Mostly because you take your experience in other languages to Python and your mind gets confused with the new conventions (or something like that). Some of these errors and their explanation:
  • if a == None:
    Really common. None is a singleton, so you can (well, actually, have) to compare to it like if a is None or if a is not None .
  • if (condition):
    This is not really pythonic, if doesn t take () around the condition in python, () purpose is grouping, you can use them in an if but only to clarify a really complex condition or similar stuff. Please don t clutter code with unneeded characters.
  • if len(list):
    This is a misunderstanding of Python way of things, any empty sequence (string, list, tuple) is False. So if you want to make sure a list is empty, just check if it s False. Same for tuples, and in some ocassions for strings.
  • if type(obj) is type(1):
    This is killing kittens, there s a builtin function for this purpose, it s called -surprise- isinstance(object, type). For example isinstance(123, int)
Read more about this and other conventions in PEP 8.

21 May 2009

Ingo Juergensmann: Using Google via IPv6

As some may know Google is reachable via IPv6 by choosing http://ipv6.google.com/ in your browser. But this might have the disadvantage that you will end rather quick on IPv4 addresses again. Another possible way to use Googles services is to use a nameserver that is allowed to query Google for AAAA records.
For Sixxs.net users you can use the nameservers of Sixxs.net as forwarders, but when you put these forwarders into main section of your bind config, all queries will be forwarded to Sixxs.net nameservers. You might prefer to choose the forwarder just when asking for google.com or other domains of the Googleversum. Example is like this:

CODE:
zone "google.com"
type forward;
forwarders
2001:b18:0:1000:2e0:81ff:fe61:ae0d;
2001:1418:10:2::2;
;
forward first;
;

zone "google.de"
type forward;
forwarders
2001:b18:0:1000:2e0:81ff:fe61:ae0d;
2001:1418:10:2::2;
;
forward first;
;


That way you have (IMHO) better control about your DNS queries, but have to take care for each domain you want to forward to those nameservers. Of course, if you don't have concerns about using those forwarders for all of your queries, you can use them in your main section. The above IPv6 addresses are part of nscache.eu.sixxs.net. Some more detail can be found at Sixxs.net forum.

Josselin Mouette: Reporting useful bugs

Here are a few hints if you want your reports to be treated more promptly. When testing fails, try unstable If you are running testing, not all bug reports are interesting. More specifically, the relevant ones are about insufficient dependencies that let slip into testing a non-working combination of packages. Think of the I upgraded foo, then bar broke category. Otherwise, the reason why reportbug looks for newer versions of a package in unstable is that, in most cases, the bug is fixed in unstable. If you can, always try the unstable version before reporting the bug. Report bugs against the failing package This is related to printing in a GNOME program, so I report this against libgnomeprint. Always file reports against the failing program. If there is a library involved, reportbug will include the necessary information so that the maintainer can easily reassign. If you re not sure of the affected package, use a metapackage (for example, gnome for GNOME-related packages). Do not second-guess the origin of the bug After upgrading libfoo, bar starts crashing, so I m reporting it against libfoo. Unless you can tell which function is at fault and how this is ABI breakage, you should report the bug against the program that crashes. The program s maintainer will reassign if needed, and will hit the library s maintainer with a hammer if needed. Don t forget to explain what s wrong The frobniz in libfrob is broken, you should change /usr/share/libfrob/blah line 13 to something else. I don t know what libfrob is about, but I have some problems that look frobniz-related. Do not second-guess, always start explaining what bug you are seeing first. Your attempts at investigating it can be interesting, but don t forget to explain what bug lead you on investigating. What is the use case we should cover? I m trying to do that with your package and it looks extremely complicated/doesn t work as expected/fails miserably. Your package is clearly broken! If your attempts to do something that should be trivial, given the purpose of the package, fail, then you are probably looking at the problem at the wrong angle. There may be a much simpler, though radically different, way to do what you are trying to achieve. In these situations, always include in your report what you are trying to do and why you want to do it. strace is not a general-purpose debugging tool Whether you observe a crash, a deadlock or a livelock, strace is not the tool that will give us information. The only thing you do by adding a strace log to the bug report is making it unreadable with its thousands of lines of output. For the specific cases that require strace for debugging, these traces are welcome but if you re not sure, don t attach them. The developers will ask them for you. Otherwise, the general-purpose debugging tool is gdb. Use reportbug There is a good reason why some packages include bug control files and scripts. Use a reporting tool that takes them into account. Unless you are requesting an enhancement, always use reportbug. Please avoid reportbug-ng since it only provides parts of the required information.

19 May 2009

Neil Williams: libqof2

It's taken a lot longer than I expected (mostly due to getting other things done in Emdebian etc.) but now that Lenny is out, QOF can finally drop all the previously deprecated code and move to libqof2, so the upstream code has now been released via Alioth (0.8.0) and the binaries have been uploaded to Debian and are waiting in NEW.

QOF provides a set of C language utilities for performing generic structured complex queries on a set of data held by a set of C/C++ objects. This framework allows programmers to add query support to their applications without having to hook into a SQL Database. QOF is a large part of my attempts to get some degree of data freedom to complement the source code freedom. This remains a problem with certain applications, especially financial ones, so I've tried to make it easier to both import and export data between programs using QOF and to convert the default QOF data storage formats into other common formats using simple tools and scripts based on XSL or perl. Having a SQL-type query interface accessible via the command line is a huge bonus when dealing with large data sets.

There is a patch for gnotime that makes the final adjustments to handling the removed code. pilot-qof and gpe-expenses already build cleanly without the deprecated code but each of these is also pending a new upstream release, so rather than need a binNMU, I'm simply going to make those releases when QOF clears NEW.

Having said that, pilot-qof and gpe-expenses introduce new binary packages themselves so they will spend time in NEW too. pilot-qof will be the last of the set to be released because the new upstream version can use symbols from the new gpe-expenses release to make it easier to move data between applications. However, that support is optional at build time and could be done via a Debian revision, so I might do it that way.

This also marks the completion of the migration of QOF from SourceForge to Alioth - principally for easier bug tracking and a usable mailing list archive. The QOF VCS is in svn.debian.org.

The mailing list for QOF development is therefore at alioth:
QOF-devel and archives also.

I'm still working on two other QOF-based utilities - one that can handle simple accounts and one to handle simple invoices. The invoice handling is already partially implemented in pilot-qof but I'm looking at a GTK+ frontend with a robust command line interface so that scripts can update the invoice data behind-the-scenes. Hopefully, some form of arbitrary-format parser can be arranged that can handle extracting data from whatever form the client happens to send - typically PDF. It'll need tweaking for each client, but it could work. The objects used by gpe-expenses and pilot-qof (including invoices) are exposed via QOF and can be mixed together in any new QOF application, easily able to read data from multiple sources, merge data sets, query data sets and export into other formats. The core currency and expense types come from pilot-link but are then enhanced to support GnuCash conventions.

For information on another QOF application called "estron" - an idea for a data-driven query interface - see http://estron.alioth.debian.org/. Problem 1 with estron: the examples all rely on postgres and I just can't get that to setup properly as either a test system or as a Debian package. Problem 2: estron was intended to be an interpreter that could be written "in itself" but whilst this is possible for the simpler examples, doing more complicated stuff like calculating values from other values makes the estron XML very obtuse. The current codebase uses GOB2 to

QOF is now stable and I don't expect to need another SONAME bump for quite a while. If anyone is interested in taking on estron or helping with the Cash and/or Invoice frontends, let me know. The problem with estron currently is bitrot - the GTK+ frontend has too many gtk1.2 hangovers which now fail to build.

18 May 2009

Marco d'Itri: Supporting unsupported USB devices

Some common classes of USB devices are supported by the same drivers because they share an identical or almost identical hardware design, but since each manufacturer uses its own Vendor and Product IDs the drivers need to be updated for each one. Some popular examples are 3G modems and devices which present a serial interface. As an interim solution until the kernel can be updated, the Linux device model provides a sysfs parameter called new_id which can be used to notify a driver that it should recognize a VID/PID pair as supported. The following example uses udev to add support for the Sheevaplug to ftdi_sio, the driver for the widely used USB to UART chips manufactured by FTDI. First we need a rule to automatically load the driver when the device is plugged in: udev cannot load it by itself since it does not know about the Sheevaplug VID and PID:
# if no driver has claimed the interface yet, load ftdi_sio
ACTION=="add", SUBSYSTEM=="usb", ENV DEVTYPE =="usb_interface", \
        ATTRS idVendor =="9e88", ATTRS idProduct =="9e8f", \
        DRIVER=="", \
        RUN+="/sbin/modprobe -b ftdi_sio"
And then we instruct udev to add the new VID and PID after the driver has been loaded:
# add the sheevaplug VID and PID to the list of devices supported by ftdi_sio
ACTION=="add", SUBSYSTEM=="drivers", \
        ENV DEVPATH =="/bus/usb-serial/drivers/ftdi_sio", \
        ATTR new_id ="9e88 9e8f"
While we are configuring udev we can also make it create a symlink for the console interface, since the FTDI chip in the Sheevaplug provides two serial interfaces: interface 0 is connected the JTAG lines and interface 1 to the serial console:
# create a convenience symlink for the console device
ACTION=="add", KERNEL=="ttyUSB*", \
        ATTRS interface =="SheevaPlug JTAGKey FT2232D B", \
        ATTRS bInterfaceNumber =="01", \
        SYMLINK+="sheevaplug"
This last rule shows an important misfeature of udev: while the keys ending in "S" can match nodes at any level of the devices chain, the matched nodes must be all at the same level. In this case this means that a single rule cannot have keys matching both the interface number and the VID and PID: this would require multiple rules and GOTO directives, so my rule matches the interface node which is available at the same level.

17 May 2009

Holger Levsen: Appropriate talk about porn: it's abuse

Russel, I agree that some porn sites are facing interesting technical challenges, but I wouldn't suggest using them as example for anything, just like a talk about the challenges of torture is always inappropriate in a human society.

Porn is, except for self made porn and some other corner cases, abuse and oppression. Simple as that.

And please, before you try to argue against that simple statement, read the lyrics of no answer for a dancer from one of my favorite bands, consolidated, and think for more than ten minutes. If we were living in really free societies, porn could be fun.

Which reminds me of a quote from Rebecca L. Eisenberg: "A truly radical view of pornography recognizes that we are what we consume -- and if we are going to alter the face of society, and destroy Victorian and religious notions like gender altogether -- as I believe we must -- we must be willing to evaluate and change every aspect of our lives, including the images we feed ourselves as we fuck, as terrifying as it sounds. Like it or not, revolution is a scary concept." The complete text where this quote is from can be found online on her webpage.

P.S: If you hear me playing "the internet is for porn" (this musicalsong) or remember me doing so - can you now see the great irony? I would have hoped you've seen it before. Or at least wondered, what the hell I was doing.

16 May 2009

Steve Kemp: Humans don't drink blood.

I've said it multiple times, but all mailing list managers suck. Especially mailman. (On that topic SELinux is nasty, Emacs is the one true editor, and people who wear furry boots are silly.) Having setup some new domains I needed a mailing list manager, and had to re-evaluate the available options. Mostly I want something nice and simple, that doesn't mess around with character sets, that requires no fancy deamons, and has no built in archiving solution. Really all we need is three basic operations:
  • Confirmed opt-in subscription.
  • Confirmed opt-out unsubscription.
  • Post to list.
Using pipes we can easily arrange for a script to be invoked for different options:
# Aliases for 'list-blah' mailing list.
#
^list-blah-subscribe$:   " /sbin/skxlist --list=list-blah --domain=example.com --subscribe"
^list-blah-unsubscribe$: " /sbin/skxlist --list=list-blah --domain=example.com --unsubscribe"
^list-blah$:             " /sbin/skxlist --list=list-blah --domain=example.com --post"
The only remaining concerns are security related. The most obvious concern is that the script named will be launched by the mailserver user (Debian-exim in my case). That suggests that any files it creates (such as a list of email addresses - i.e. list members) will be owned by that user. That can be avoided with setuid-fu and having the mailing list manager be compiled. But compiled code? When there are so many lovely Perl modules out there? No chance! In conclusion, if you're happy for the exim user to own and be able to read the list data then you can use skxlist. It is in live use, allows open lists, member-only lists, and admin-only lists. It will archive messages in a maildir, but they are ignored for you to use if you see fit. List options are pretty minimal, but I do a fair amount of sanity checking and I see no weaknesses except for the use of the Debian-exim UID. ObFilm: Blade

14 May 2009

Antti-Juhani Kaijanaho: Asynchronous transput and gnutls

CC0
To the extent possible under law,
Antti-Juhani Kaijanaho has waived all copyright and related or neighboring rights to
Asynchronous transput and gnutls. This work is published from Finland.
GnuTLS is a wonderful thing. It even has a thick manual but nevertheless its documentation is severely lacking from the programmer s point of view (and there doesn t even seem to be independent howtos floating on the net). My hope is to remedy with this post, in small part, that problem. I spent the weekend adding STARTTLS support to the NNTP (reading) server component of Alue. Since Alue is written in C++ and uses the Boost ASIO library as its primary concurrency framework, it seemed prudent to use ASIO s SSL sublibrary. However, the result wasn t stable and debugging it looked unappetizing. So, I wrote my own TLS layer on top of ASIO, based on gnutls. Now, the gnutls API looks like it works only with synchronous transput: all TLS network operations are of the form do this and return when done ; for example gnutls_handshake returns once the handshake is finished. So how does one adapt this to asynchronous transput? Fortunately, there are (badly documented) hooks for this purpose. An application can tell gnutls to call application-supplied functions instead of the read(2) and write(2) system calls. Thus, when setting up a TLS session but before the handshake, I do the following:
                gnutls_transport_set_ptr(gs, this);
                gnutls_transport_set_push_function(gs, push_static);
                gnutls_transport_set_pull_function(gs, pull_static);
                gnutls_transport_set_lowat(gs, 0);
Here, gs is my private copy of the gnutls session structure, and the push_static and pull_static are static member functions in my sesssion wrapper class. The first line tells gnutls to give the current this pointer (a pointer to the current session wrapper) as the first argument to them. The last line tells gnutls not to try treating the this pointer as a Berkeley socket. The pull_static static member function just passes control on to a non-static member, for convenience:
ssize_t session::pull_static(void * th, void *b, size_t n)
 
        return static_cast<session *>(th)->pull(b, n);
 
The basic idea of the pull function is to try to return immediately with data from a buffer, and if the buffer is empty, to fail with an error code signalling the absence of data with the possibility that data may become available later (the POSIX EAGAIN code):
class session
 
        [...]
        std::vector<unsigned char> ins;
        size_t ins_low, ins_high;
        [...]
 ;
ssize_t session::pull(void *b, size_t n_wanted)
 
        unsigned char *cs = static_cast<unsigned char *>(b);
        if (ins_high - ins_low > 0)
         
                errno = EAGAIN;
                return -1;
         
        size_t n = ins_high - ins_low < n_wanted
                ?  ins_high - ins_low
                :  n_wanted;
        for (size_t i = 0; i < n; i++)
         
                cs[i] = ins[ins_low+i];
         
        ins_low += n;
        return n;
 
Here, ins_low is an index to the ins vector specifying the first byte which has not already been passed on to gnutls, while ins_high is an index to the ins vector specifying the first byte that does not contain data read from the network. The assertions 0 <= ins_low, ins_low <= ins_high and ins_high <= ins.size() are obvious invariants in this buffering scheme. The push case is simpler: all one needs to do is buffer the data that gnutls wants to send, for later transmission:
class session
 
        [...]
        std::vector<unsigned char> outs;
        size_t outs_low;
        [...]
 ;
ssize_t session::push(const void *b, size_t n)
 
        const unsigned char *cs = static_cast<const unsigned char *>(b);
        for (size_t i = 0; i < n; i++)
         
                outs.push_back(cs[i]);
         
        return n;
 
The low water mark outs_low (indicating the first byte that has not yet been sent to the network) is not needed in the push function. It would be possible for the push callback to signal EAGAIN, but it is not necessary in this scheme (assuming that one does not need to establish hard buffer limits). Once gnutls receives an EAGAIN condition from the pull callback, it suspends the current operation and returns to its caller with the gnutls condition GNUTLS_E_AGAIN. The caller must arrange for more data to become available to the pull callback (in this case by scheduling an asynchronous write of the data in the outs buffer scheme and scheduling an asynchronous read to the ins buffer scheme) and then call the operation again, allowing the operation to resume. The code so far does not actually perform any network transput. For this, I have written two auxiliary methods:
class session
 
        [...]
        bool read_active, write_active;
        [...]
 ;
void session::post_write()
 
        if (write_active) return;
        if (outs_low > 0 && outs_low == outs.size())
         
                outs.clear();
                outs_low = 0;
         
        else if (outs_low > 4096)
         
                outs.erase(outs.begin(), outs.begin() + outs_low);
                outs_low = 0;
         
        if (outs_low < outs.size())
         
                stream.async_write_some
                        (boost::asio::buffer(outs.data()+outs_low,
                                             outs.size()-outs_low),
                         boost::bind(&session::sent_some,
                                     this, _1, _2));
                write_active = true;
         
 
void session::post_read()
 
        if (read_active) return;
        if (ins_low > 0 && ins_low == ins.size())
         
                ins.clear();
                ins_low = 0;
                ins_high = 0;
         
        else if (ins_low > 4096)
         
                ins.erase(ins.begin(), ins.begin() + ins_low);
                ins_high -= ins_low;
                ins_low = 0;
         
        if (ins_high + 4096 >= ins.size()) ins.resize(ins_high + 4096);
        stream.async_read_some(boost::asio::buffer(ins.data()+ins_high,
                                                   ins.size()-ins_high),
                               boost::bind(&session::received_some,
                                           this, _1, _2));
        read_active = true;
 
Both helpers prune the buffers when necessary. (I should really remove those magic 4096s and make them a symbolic constant.) The data members read_active and write_active ensure that at most one asynchronous read and at most one asynchronous write is pending at any given time. My first version did not have this safeguard (instead trying to rely on the ASIO stream reset method to cancel any outstanding asynchronous transput at need), and the code sent some TLS records twice which is not good: sending the ServerHello twice is guaranteed to confuse the client. Once ASIO completes an asynchronous transput request, it calls the corresponding handler:
void session::received_some(boost::system::error_code ec, size_t n)
 
        read_active = false;
        if (ec)   pending_error = ec; return;  
        ins_high += n;
        post_pending_actions();
 
void session::sent_some(boost::system::error_code ec, size_t n)
 
        write_active = false;
        if (ec)   pending_error = ec; return;  
        outs_low += n;
        post_pending_actions();
 
Their job is to update the bookkeeping and to trigger the resumption of suspended gnutls operations (which is done by post_pending_actions). Now we have all the main pieces of the puzzle. The remaining pieces are obvious but rather messy, and I d rather not repeat them here (not even in a cleaned-up form). But their essential idea goes as follows: When called by the application code or when resumed by post_pending_actions, an asynchronous wrapper of a gnutls operation first examines the session state for a saved error code. If one is found, it is propagated to the application using the usual ASIO techniques, and the operation is cancelled. Otherwise, the wrapper calls the actual gnutls operation. When it returns, the wrapper examines the return value. If successful completion is indicated, the handler given by the application is posted in the ASIO io_service for later execution. If GNUTLS_E_AGAIN is indicated, post_read and post_write are called to schedule actual network transput, and the wrapper is suspended (by pushing it into a queue of pending actions). If any other kind of failure is indicated, it is propagated to the application using the usual ASIO techniques. The post_pending_actions merely empties the queue of pending actions and schedules the actions that it found in the queue for resumption. The code snippets above are not my actual working code. I have mainly removed from them some irrelevant details (mostly certain template parameters, debug logging and mutex handling). I don t expect the snippets to compile. I expect I will be able to post my actual git repository to the web in a couple of days. Please note that my (actual) code has received only rudimentary testing. I believe it is correct, but I won t be surprised to find it contains bugs in the edge cases. I hope this is, still, of some use to somebody :)

Next.

Previous.