Search Results: "cas"

30 September 2009

Romain Beauxis: About free software and yellow pages...

Even if it has happened in the past that I could support a pragmatic approach in the context of free software and media technologies, there is something that offenses me in the context of free audio players. In the world of audio streaming servers on for net radios, there are two majors competitors. In the one hand, Shoutcast, the commercial proprietary streaming server released by Nullsoft, which is also the editor of Winamp, the popular media player. On the other hand, Icecast, which started as an open source implementation of the protocol used by Shoutcast, but was rewritten in its version 2 release to support the free codecs from xiph. Now, appart from the respective quality of the two servers, the real competition takes place in another place: availability of the radios using this or that server. For that purpose, both have implemented Yellow Pages. Shoutcast's yellow pages are here, whereas Icecast/Xiph's yellow pages are there. The great thing with yellow pages is their integration into the audio player. Then, the user can browse the list of radios, and test them. It is very good both for convenience no need to download a playlist or enter manually a complicated URL and advertisement a single place to list all possible radios. The yellow pages from Shoutcast are very popular. They are integrated in Winamp of course, but also in many other popular players, in the PSP, in the iPhone, etc... The Shoutcast yellow pages are reserved for shoutcast users. Of course, the protocol has been reversed it is kind of dummy but if your radio get caught registering to Shoutcast's yellow pages while not using their proprietary software, your radio gets blacklisted and you loose a whole lot of listeners (and money too). For that reason, many net radios want to use Shoutcast as their broadcasting server. From the point of view of Nullsoft, it seems a good call. At least, they play their game and make their money out of it, so let's put this appart. But, what about the free software world ? The obvious idea would be that the free software projects should push altogether for the use of free alternatives, hence pushing for a support of Icecast's yellow pages in the free players. Of course, free software being also a matter of choice, it should be possible to also support Shoutcast's yellow pages. After all, both protocols are very simple, so it is no big deal to implement.. Well, unfortunately, that is not exactly what happens.. Guess what, Amarok and VLC only implement Shoutcast's yellow pages... Given that VLC is a massively popular audio player that spreads among a lot of plateforms, I find it quite a pity that they do not do something to encourage its users to use free alternatives to the proprietary Shoutcast yellow pages, and similarly for Amarok. The adoption of free alternatives only works with massive demand, and we should not hope that proprietary softwares help in any way, but, at least, free softwares should be a bit more aware of what they're advocating when they choose to implement only the proprietary support...

27 September 2009

John Goerzen: How To Record High-Definition MythTV Files to DVD or Blu-Ray

I ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk. MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920 1080i down to DVD resolution of on a good day 720 480. I couldn t have that. Background My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate. ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920 1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn t be displayed. Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn t want to re-encode any video, both for quality and for time reasons. Attempts So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later. Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed. Next, I tried avidemux. Quite the capable program and I thought I could use it to cut my file into DVD-sized bits. But I couldn t get it to let me copy the valid MPEG TS into another MPEG TS it kept complaining of incompatible formats, but wouldn t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn t really what I was after. Then, I tried mpgsplit. Didn t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner. dvbcut I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R. It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box. So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:
File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
Eeeeepp. So apparently the ISO 9660 filesystem can t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2 s file was surprisingly just a few KB short. But the rest of them weren t. I didn t want to have to go back and re-split the data to be under 4GB. I also didn t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes. So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:
#!/bin/bash
set -e
if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi
if [ " id -u " != "0" ]; then
   echo "This program must run as root."
fi
EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi
# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009
SECSIZE=2048
# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000
echo "Allocating image..."
dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1
echo "Creating filesystem..."
mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"
echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"
echo "Done."
That was loosely based on a script I found on the Arch Linux site. But I didn t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive. So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.

26 September 2009

Gunnar Wolf: The bad and the worse; Representative democracies' minimums

Martin rants about the German electoral system. From his rant, I'll pick up only two points And I'll try to connect with Toxicore's excelent (Spanish) blog post, where he quotes political analist Denise Dresser. Dresser has made a great point: Our probably-imposed, legitimacy-impaired president Felipe Calder n has requested the society to talk good about Mexico, to project a positive image of the country. Dresser says, yes, there is a lot of good to talk about the country, and we should emphasize on its richness and beauty, invite people to come and visit, to know what the country is really like. But at the same time, it is our duty to talk bad about the bad areas and decisions of our government, as that is the best (if not the only) thing many of us can do to really get things to happen That is what we can do to push our country's good things forward, to make the country sustainable, to pull attention towards what needs (such as the very very deplorable cases of censorship, human rights violation, ecosystem predation we have seen in the last years). Anyway... What did I want to comment about Martin's post? He criticizes Germany's law requiring a 5% quota for a party to have parliamentary representation. In Mexico, the minimum is 2%. Most people agree, though, that it is too low, and that we should push to increase it. Why? Because the money that is spent in supporting the party system. In Mexico, when a political party fails to get 2% of the vote, it is basically disbanded and it is very hard for it to regroup, to compete again. Many people believe we should aim to a political system with as few political parties as possible (such as the semi-democratic system they have in the USA). I strongly prefer the system found in most European (and even many South American) countries where there is a real wealth of ideological positions represented, and where governments have to be formed by agreeing to form coalitions, as it is almost impossible for them to get full majority. I would much rather see Mexico march towards a parliamentary-based political system, away from the presidential one. Of course, that is almost impossible to expect. With the current political system, we are bound to have forever few monolithic, meaningless political parties. We will likely converge on three blocks, following the current three major blocks (leftoid PRD, centroid PRI, rightoid PAN). They are different in some important senses, yes, but in general they are much the same. I don't hold any hopes to ever see something like the Pirate Party appearing in our system...

Ana Beatriz Guerrero Lopez: How to get your bugs solved in Debian+KDE

Imagine you have found a bug in your Debian running KDE, a nasty one, and you want to get it resolved. What you should do? This is centered on KDE and Debian but most of it is useful in general. First Case: You have a problem and you have no idea whether it is a bug or not. Even worse, you don t know which package to report it to. This happens more often than you might believe; it even happens to experienced people. Your first step should be to checking user forums and mailing lists [0] to see if someone else has encountered the same issue. You might find that your problem is already known (and maybe easy to solve). In the case that you do not find your problem there, then you can ask for help and post a mail or message explaining your problem. Pick one mailing list or forum board, the one you think is more appropriate, then post your issue there. The most detailed you are about your problem, the bigger is the likelihood of you obtaining help. It could happen that: If you do not get any answer it might be that nobody got into the same problem and they do not how to help you. But it is also possible you were not clear about your problem. Also, you need to be patient; it is unlikely someone will answer your question 2 minutes after you posted it.
If you find out more about your problem, it is a good idea to send another email or post updating your first message. If after 5-7 days you do not get any answer, you can try asking for help in another forum or mailing list. If you are given pointers to what software is causing the problem you can go to the second case. [0]
Mailing lists: Forums: Second case: You know where the problem is. Where you should report your bug? First, you should check in both the Debian and the KDE bug tracking systems [1] to see if the bug is already reported. If that is the case, and you are able to give more information about it, update the bug report. If the bug is not reported, then report it in the KDE bug tracking system if you think it is a problem in the application or in the Debian BTS if it is a packaging problem. If you re not sure what kind of bug it is, you can go to the first case (in the beginning of this post) and ask in the users forums and mailing lists [0]. However, you won t always get it right, and in some cases you will be pointed to the other bug tracking system. Do not take it personally; KDE developers can not help you with packaging problems and Debian packagers can t always help you with the application bugs. It might be that your problem is solved in the development version, so if possible check what is going on in the development version before report. When reporting the problem, give all the details you can about your problem. If you can detail the steps to reproduce the bug, even better. Also, if you are asked for more information, reply the best you can. Somebody is trying to help you. Be nice !:) Also, remember to be patient when reporting bugs or being asked for more information: both KDE and Debian are big projects with a lot of traffic on their respective bug reporting systems (esp. KDE), so sometimes there is not a quick reply from the developers. [1] In Debian, a lot of people report upstream issues in the bug tracking system and they think it is good and what they have to do. The truth is that in a very few cases, such as security bugs or data loss bugs, this is a good idea. But most of the time it is not useful to report problems to the people who can not solve them. Do not expect Debian/KDE maintainers to forward your problem upstream (a problem which they may well not be able to reproduce), then back to you when upstream ask for more information, then back with the information to upstream it is time consuming and we have big (wo)manpower problems. In any case, if you think the bug is very important and should be in the Debian BTS, you can report it in Debian as well as reporting it to KDE. Make sure you mention the KDE bug in the Debian bug. Third case: your problem is not a bug, just a feature request. Until now, I have talked about bugs, when you find something that is clearly not working right in your system. But what about when you want to ask for a new feature in a piece of software? In very few cases this belongs in a Debian wish list. Most feature requests apply upstream and you should tell the about your idea. If you do not tell them, it is unlikely they will implement it. Still, after reporting, you should accept it if upstream thinks it is not a good idea or it is not interested enough to implement it. Post updated on the 30th September, thanks to Rupert Swarbrick for grammar fixes.

Julian Andres Klode: Results of the APT2 config parser testing


Thanks to those who have tested it (and/or will test it). The results where helpful and resulted in Bug#548443 filed against localepurge and GNOME Bug #596429 against glib. The first one is a case where quotes where used inside a value, although this has never been defined to work, and the second one is a problem with GLib s GScanner not ignoring multi-line C-style comments although it was configured to do so. I also fixed some bugs in APT2, like the missing build-depends on libgee-dev and the configuration parser now accepts . , _ , + in the option name. I also talked with Eugene about some differences in the way cupt and APT2 handle quotes and about some other parts of the configuration format. Seems this was a good day. Posted in APT2

Martin F. Krafft: Voting denied, enter pirates

This coming Sunday, Germany is voting its parliament for the next four years. I wanted to participate remotely, but they didn t let me. The letter, inviting me to order the voting forms, was sent too late. Instead of ordering and waiting for the voting forms, hoping for them to arrive back in Germany before the municipal agency stops all their work at 16:59 on Friday afternoon, I filled in the fields in a way to give my mother full authority to submit my vote. On the document, it only said that she would need a notary procuration, which she has. When my mother tried to vote in my name, they refused her: the vote has to be confidential, and proxies may only vote for disabled people, or those who cannot read. I can understand half of the motivation behind this it guards against abuse but the bottom line is that I am now unable to cast a vote. Democracy stabbed by bureaucracy. Well, democracy to the limited degree that we have it in Germany. I do not mean to criticise at this time that voting in Germany means to place trust into people you d rather not trust, but once they get majority, they can do whatever they want for four years. The specific shortcoming in our democracy on which I would like to focus is the five percent clause , which states that a party needs to reach 5% of votes before they can enter the parliament; if those 5% are not reached, all votes for the party are basically lost. I suspect that this clause is what s keeping back a lot of people from electing Germany s pirate party, which focuses on the importance of freedom in today s media-centric and interconnected world. It seems doubtful that the Piratenpartei will reach the limit this round, but given all the attention they recently received, as well as their plain cause, I would not be surprised if they did. I think it is important for people to vote the party they want, without letting the 5% limit discourage them. Anything else would be undemocratic. Let s return to the pirates: it s paramount for the government to consider freedom and related issues, and to put clueful people on the job. The Piratenpartei is the only party that approaches the issue properly, all the others are just too happy to succomb to castration of personal freedom for such lofty goals as anti-terrorism, or the containment of child pornography. I don t like terrorists, and I think child pornography is among the most horrific ways to abuse defenceless humans in irrevocable ways that I can imagine. But I also don t like losing my deodorant to airport security, cameras tracking my every move, (diletantic) censorship on the Internet, and lack of control over my own data. Time to put an end to that, and to solve problems without castrating freedom. Benjamin Franklin is credited with saying: they that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. The Piratenpartei is probably not capable of leadership at this point, for they are young. But they can help the leading party maintain a bearing. That s what the Greens did, successfully albeit slowly, and I appreciate that the pirates are approaching the matter before it will be too late. The media have not missed a chance to try to stomp over them in recent months, but at the end of the day, after all the slander and some obvious mistakes on the side of the party, I still see a group of people prepared to fight for freedom with an open and honest approach. The run-ins with the media are hardly about content, it seems more that the Piratenpartei is simply unable to become part of the sensationalist mash which is our media-politician ensemble. And I think that s good. I appreciate a party communicating not via their PR office, but with new media, like blogs, or e-mail. I appreciate that their people maintain the personality of a human trying to do politics, not a politician trying to come across as human. If my municipality had let me vote, I would have selected the pirates. They may not make the 5% this year, but I would not let that be a reason against them. If one believes in democracy, then it s important to do it right.

25 September 2009

Mauro Lizaur: TODO-list before 2012

Among other random-stuff, I have this mental-TODO list for quite some time, just in case we dissapear (probably it'll be just like in Y2K):

[ ] Form a Death-freaking-uber-grind-metal band and tour all over the world.
[ ] Travel to Japan
[ ] See Faith No More live
[X] See Racing Club win a championship.
[X] Shoot a cultish bizarre short-film about the (almighty) Cthulhu
[X] Make a living with free software
[X] See Opeth live
[X] Become a Debian Developer \o/

So I'd like to thank to those who helped me during and before the process (advocating, sponsoring, am-ing, etc).

/me waves to the crowd, grabs his oscar-award and leaves the scenario.

Ryan Niebur: Padre 0.47 released!

Version 0.47 of Padre, a Perl IDE, has been released! Thanks to everybody who contributed! The majority of the changes for this release are lots of bug fixes, as well as improvements and refactoring of Padre s code. Among the bug fixes are many fixes for Windows-specific problems. There s also improvements to the beginner mode, and improvements to the included Perl documentation, making Padre even better for people who are learning Perl! This release had the biggest Changes entry of all of the Padre releases, with 41 Changes entries in 12 days, so I m not going to list all of the changes. But here are some of the most important ones: The complete list of changes can be found here.

24 September 2009

Eddy Petrișor: I'm voting Remus Cernea (Green Party) - Eu votez Remus Cernea (Partidul Verde)

Romania is closing really quickly to the presidential elections.

The options for voting seem to be non-existent and is basically about choosing the smallest evil and I was planning on not voting.

That was until I found out that a personality from the civic society, Remus Cernea has announced that he wants to candidate for Presidency.

He is a candidate from the Green Party and he is in the process of collecting support signatures. 200 000 - two hundred thousands - signatures to be able to candidate for presidency in a country of less than 20 millions.

Yes, Romania has one of the most undemocratic and tailored-for-the-current-in-power election laws. Here (in Romanian) is a summary made by Remus Cernea himself about the election laws in Romania.

A full report was sent by the Romanian Green Party to the European Green Party Committee and the European Greens have expressed their concern about the current state of the law in Romania.



Since I want to support him and since he has been actively ignored (ro) by mass media for the past month (since his announce), I decided to help collecting the signatures. This post is also about making the information flow to the people via the Internet.


So, if, after reading and informing yourselves about Remus Cernea and his civic actions:
you decide that you'd want him in the election, you can contact me (eddy.petrisor+remuscernea _at_ gmail.com) for details how to make this happen.

David Welton: LangPop.com Updates

It took me a while to get the September update of LangPop.com out. The reason: I had to redo the book statistics. Previously, I got those from Amazon's web service, which worked ok as far as those things go. However, recently, I ran into several problems:
  • They started requiring signatures on the requests, which meant I had to stop and fiddle with my code, which I did not appreciate. (If you have this problem and use Ruby, get the amazon-ecs Ruby gem)
  • Their results suffered from some kind of regression in any case: C, C# and C++ all showed up with the same number of results. They didn't use to show the same numbers. Not good from my point of view.
  • They were completely unresponsive to requests for information or help.
  • And for good measure, they hid their information about their products API on some other web site, with no links from their web services page, which makes it kind of annoying to look for it, and suspicious that they may be removing it at some point in the future.
All of these together made me decide it was time to switch data providers. Where to go? The answer was obvious: back to my home state of Oregon, to Powell's Books, a huge independent book seller. If you ever go to Portland, do yourself a favor and go to Powell's. It is a city block full of books on multiple floors. When I lived in Portland, I would go there before eating to give myself a reason to leave (although they do have a cafe...). And that's just the regular book store. They have another huge space a few blocks away dedicated entirely to technical books. Powell's is one of my favorite things in Oregon, and a great experience. Anyway, I decided that they're big enough, and do enough business online that I'd ask them about getting data, so I sent some email asking after what options were available. I got a very helpful answer from a real person, CJ Stritzel, who was kind enough to explain what the options were, and work with me to help find the best one for what I needed. Definitely better than "/dev/null", the Amazon message board! The API is still something they're working on, but it works well enough for what I need, and I'm quite happy with the situation. It has caused some changes in the rankings, though: they show Java books as being by far and away the most popular, which has pushed that language into the top spot in the rankings. We'll see over the coming months; I may rework the queries I'm using some as their API stabilizes and I experiment with different combinations of keywords. In any case, thanks Powell's, and enjoy the new statistics everyone.

Dirk Eddelbuettel: digest 0.4.0 and 0.4.1

After almost two years, I just made a small update to the digest package for R by adding sha-256 as another supported hashing function. While at this, a few minor edits where made here or there. Version 0.4.0 also included a file src/Makefile.win but as that actually prevents a successful build on the Windows platform I promptly removed it for a (more-or-less brown-bag) release 0.4.1. Thanks to Uwe Ligges and his automated win-builder service that permits easy unattended build tests on that platform. The new package will appear shortly on its CRAN page; the R-Forge page has access to the Subversion repo etc and otherwise there's always my local my local digest page. Oh, and 0.3.1 was never blogged about as it was just a maintenance release that added a single cast and an edit to the DESCRIPTION file.

Paul Wise: Adobe CMap and AGLFN data now free software!

In March 2009 I contacted Adobe about the self-contradictory nature of the license for the Adobe Glyph List For New Fonts (AGLFN) data. I did so because one of the upstreams I am involved in (fonttools) embeds a copy of aglfn.txt and I noticed the license was a bit strange. An Adobe employee by the name of David Lemon replied to say that the license would become less free than it was. I then asked him to consider using a free software license for both the Adobe CMap and AGLFN data. We discussed the benefits and drawbacks of doing this for both Adobe and the free software community. David was initially skeptical about the balance between benefits and drawbacks for Adobe and I was feeling pessimistic about the situation. The conversation ended and several months passed, with me sending the occasional ping and David being busy. Then in June he told me a new license was coming in a month. So I sent another ping on September 20th to find out what the status of that was. A couple of days later I received a mail saying that the Adobe CMap data has now been released under the BSD license and that the Adobe AGLFN data will soon follow! Please note that while this means that modifications are technically allowed, they are still strongly discouraged for compatibility reasons. Adobe has assigned an emailable maintainer (currently Ken Lunde) for these files so there is no reason that modifications should not be done upstream. You can also edit the wiki to document things or post in the forum. If you happen to meet David, Ken or Dave McAllister (who manages the Open@Adobe program) somewhere, please thank them and offer beverages. So, what does this mean for free software projects? Debian CJK users will be able to read many more PDFs with CJK characters without using any non-free packages once the new CMap files reach main. I will need to package the new AGLFN data for Debian. I will need to remove the embedded AGLFN from fonttools upstream and adapt it to use the packaged one if available or suggest the user download it if not. GNU Classpath and the other software projects using AGLFN data might like to do the same. I may consider writing a library for loading the AGLFN data, mainly as an exercise in learning how to write a good library though. Oh, and perhaps I can convince (Kenshi MUTO) my minimal level of involvement in this is worth a beer if I ever meet him :D

23 September 2009

Jo Shields: directhex@debian.org

For those who don t follow my exciting life on the IRCosphere or Tweetoscape, I m now officially a Debian Developer, complete with snazzy email address. What this means in real terms is that the pkg-cli-apps, pkg-cli-libs and pkg-mono teams now have a second person with upload rights, alongside the talented but occasionally overworked Mirco Bauer. Which should lower the amount of time that contributor packages and patches spend waiting to be sponsored. As the cool kids say: woo!

Roland Mas: FusionForge news, September 2009

Here's another round of the semi-regular bulletin about FusionForge. First item: FusionForge 4.8.1 was released this week. It's not exactly an important update, but the 4.8 branch had been accumulating fixes over time and we felt that it would be good to push these fixes out. If you don't encounter particular problems, there's probably no need to upgrade in a hurry. A follow-up for the rewrite of the SCM subsystem: I now consider the Bazaar and Git plugins complete. The missing part, in both cases, was a proper integration of a repository browser and the collection of commit statistics; since one of my clients wants to use Bazaar and another one wants Git, both features have been completed recently. The code still lives on a branch based off 4.8 (for people who need a 4.8-based instance), but it's also been pushed into trunk so the next release will have it natively. Another branch I've been working on (for clients) was about making the Mediawiki plugin able to handle one wiki per project rather than one shared wiki. This is now possible with yet another 4.8-based branch, where the wiki creation is completely automated. A nice feature is that the FusionForge identification is used as a basis for Mediawiki, with different groups on the wiki depending on project membership and role in the forge. That allows specifying wiki permissions in a simple way, for instance to say that only project members can create new pages, authenticated users can only edit existing pages, and non-authenticated users are read-only. This code will be pushed to trunk in the coming weeks. Thanks to Alain Peyrat, we now have a buildbot running Hudson for unit tests and a few other things. The coverage isn't complete yet, but we hope to increase it as time passes. It's already proven useful, by ensuring at least correctness of PHP syntax, encoding and line-endings. I think that's about it for this time. Business as usual.

22 September 2009

Francois Marier: grub on a bootable USB rescue stick

If you've got a cheap USB key lying around which is too small to be useful, follow these instructions to create a rescue "stick" in case your boot loader stops working.

(This is based on these instructions but simplified and updated for grub 2.)

Before you start, make sure you've got both the parted and the grub-pc packages installed.
Make the USB stick bootablePlug in your USB stick and replace the device name and capacity with the appropriate values. (You can find the total capacity of the stick by typing print inside parted.)
$ sudo parted /dev/sda
(parted) rm 1
(parted) mkpart primary 1MB 4009MB
(parted) mkfs 1 fat32
(parted) toggle 1 boot
(parted) quit
Install grubAgain, substitute sda with the correct device name.
mount /dev/sda /mnt

mkdir -p /mnt/boot/grub
cp /usr/lib/grub/i386-pc/* /mnt/boot/grub/
echo '(hd0) /dev/sda' > /mnt/boot/grub/device.map
sudo grub-install --root-directory=/mnt /dev/sda

umount /mnt
That's it, now all you have to do is reboot with the stick plugged in and test booting your box manually with it.

Robert Collins: Python unittest API : Time to fix it


So, for ages now I ve been saying that unittest is, at its core, pretty sound. I incited a talk to this effect. I have a vision; I dream of a python testing library that:
  1. Is in the python core
  2. Is simple
  3. Is extensible
  4. Has tests take care of testing
  5. Has results take care of reporting
  6. Aids communication from test to test reader
Hopefully those are pretty modest and agreeable things to want. However we don t have this: nose is lovely but not in the core [and is a moderately complex API]. py.test is also not in the core, and has previously tripped my too-much-magic alerts. I must admit to not having checked if this is fixed yet. unittest itself is in the core but has some cruft which we should clean up, but more importantly is not extensible enough, which leads to extensions such as the zope testrunner having to muddy the waters between testing and reporting. The point Aids communication from test to test reader is worth expanding on: automated testing is something that doesn t need observation until the unexpected happens. At that point some poor schmuck such as you or I ends up trying to guess what went wrong. The more data that we gather and communicate about the event, the greater the chance it can be corrected without needing a repeat run under a debugger, or worse, single stepping through the code. There is a problem with assertFoo methods in unittest, something that I m not going to cram into this blog post. I will say, if you find the tendency of such methods to crawl to the base class frustrating, that you should look at hamcrest it and similar things have been very successful in the Java unit testing world; we can learn from them. Going back to my vision, we need to make unittest more powerfully extensible to allow projects like nose to do all the cool things they want to while still being unittest compatible. I don t mean that nose can t run unittest tests; I mean that unittest can t run nose tests: nose has had to expand the contract, not simply add implementations that do more. To that end I have a number of bugs which I need to file. Solving them piecemeal will create a fractured API particularly if this is done over more than one release. So I am planning on prototyping in other projects, discussing like mad on the testing-in-python list, and when it all starts to come together writing up a PEP. The bugs I have are:
  1. streams nicely: countTestCases must die/be made optional. This function is inherently incompatible with generative tests or anything beyond the simplest lightweight environments
  2. no way to wrap code around a single test. This would permit profiling, debugging, tracing, and I m sure other things more cleanly. (At the moment, one must turn on the profiler in startTestCase, and turn it off in stopTestCase. This is much more awkward than simply being in the call stack). Some care will be needed here, particularly for generative tests.
  3. code that isn t part of the implementation in the core needs to be able to work with the reporting code; allowing an optionally wider API permits extensions to be debuggable. This needs thought: do we allow direct access to TestResults? Do we come up with some added level of indirection and events ? I don t know.
  4. More data than just the backtrace needs to be included when an outcome is reporter. I ve started a discussion on the testing in python list about this. I m proposing that we use a dict of named content objects, and use the HTTP content-type abstraction to make the content objects introspectable and reliably handleable without tying the unittest object protocol to any given wire format loose coupling is good!
  5. The way we signal outcomes between TestCase and TestResult the addFailure etc methods is concerning: there are many grades of outcome that users of the framework may usefully wish to represent; in fact there are more than we probably want to put in the core. Finding a way to decouple the intent of a particular outcome from how its signalled would allow users more control while still being able to use the core framework. One particular issue in this area is that its possible with the current API to have a single test object succeed multiple times. Or fail (addFailure) then succeed (addSuccess). This causes no end of confusion, as test counts can mismatch failure counts, and so on.
I ve got some ideas about these bugs, but I m approaching a kiloword already, and I hope this post has enough to provoke some serious thought about how we can fix these 5 bugs, compatibly, and end up with a significantly better unittest module. We ll have been sucessful if projects like Trial, nose and the zope testrunner are able to remove all their code that duplicates standard library functionality or otherwise worksaround these bugs, and can instead focus on adding the specific test support needed by their environments (in the Trial and zope cases), or on UI and plug-n-play (for nose).

Uwe Hermann: Using CRM114 for spam filtering on Debian GNU/Linux

I've been using CRM114 as spam filter for a while now, and I'm quite happy with it. Due to bug #529720 though (incompatible upstream file format changes) I decided to start my setup from scratch with a recent CRM114 version from unstable. Here's a short HOWTO, hope it's useful for others. First you need to install crm114 and set up a few files in your $HOME directory.
  $ sudo apt-get install crm114
  $ mkdir ~/.crm114
  $ cd ~/.crm114
  $ cp /usr/share/doc/crm114/examples/mailfilter.cf.gz .
  $ gunzip mailfilter.cf.gz
  $ cp /usr/share/crm114/mailtrainer.crm .
  $ touch rewrites.mfp priolist.mfp
Edit ~/.crm114/mailfilter.cf and set the following variables (some are optional, but that's what I currently use):
  :spw: /mypassword/
  :add_verbose_stats: /no/
  :add_extra_stuff: /no/
  :rewrites_enabled: /no/
  :spam_flag_subject_string: //
  :unsure_flag_subject_string: //
  :log_to_allmail.txt: /no/
The :log_to_allmail.txt: /no/ option should probably stay at "yes" for the first few days until you have tested your setup and everything works OK. The ~/.crm114/allmail.txt file will contain all your mails, in case something goes wrong. Now set up empty spam and nonspam files like this:
  $ cssutil -b -r spam.css
  $ cssutil -b -r nonspam.css
Test the setup by invoking mailreaver.crm as follows, typing some test text and then pressing CTRL+d:
  $ /usr/share/crm114/mailreaver.crm -u ~/.crm114
  test
  [CTRL-d]
  ** ACCEPT: CRM114 PASS osb unique microgroom Matcher **
  CLASSIFY fails; success probability: 0.5000  pR: 0.0000
  Best match to file #0 (nonspam.css) prob: 0.5000  pR: 0.0000
  Total features in input file: 8
  #0 (nonspam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  #1 (spam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  X-CRM114-Version: 200904023-BlameSteveJobs ( TRE 0.7.6 (BSD) ) MF-35EB8B9A [pR: 0.0000]
  X-CRM114-CacheID: sfid-20090920_151224_574131_D290E589
  X-CRM114-Status: UNSURE (0.0000) This message is 'unsure'; please train it!
The output should look similar to the above. If there are errors instead, you should check your settings in ~/.crm114/mailfilter.cf. Now you have to setup a procmail rule for crm114:
  :0fw: crm114.lock
    /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114
  :0:
  * ^X-CRM114-Status: SPAM.*
  IN.spam-crm114
In my case this rule is also followed by a spamassassin rule, so all my mail goes through two different spam filters (will look into dspam and bogofilter also I guess, the more the better). Finally, in .muttrc I have the following configs so I can press SHIFT+x to mark a mail as spam, and SHIFT+h to mark it as non-spam (ham).
macro index X '  formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version   /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro index H '  formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version   /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'
macro pager X '  formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version   /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro pager H '  formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version   /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'
Important: crm114 is most effective if you start with empty CSS files (as shown above) and only train it by marking mails as spam/ham when it gets them wrong. The process will take a few hours or maybe a day (depending on how many mails per day you get), then the misclassification rate gets very low... Update 2009-09-23: Changed --spam/--nonspam to the correct options for mailreaver/mailtrainer, --spam/--good.

20 September 2009

Jeff Bailey: Home Media Server

I'm been wrestling with the idea of a home media server for a bit. Our home computing needs are pretty light, they only thing either of us really do outside of a web browser these days is ssh (me) or nethack (Angie). Outside of the laptop we have a pretty standard non-connected house. The only case where I'd really like to change this is music. We don't generally watch TV (and when we do, it's usually DVDs that we've actually purchased), but we do listen to a fair amount of music. The CDs tend to get scattered around the house and then mauled by our velociraptor offspring.

I looked at the Squeezebox, which would be perfect if it wouldn't mean replacing a whole bunch of perfectly good stereos with a whole bunch of nicer-than-we-need stereos.

The solution I've hit onto is this: a NAS with an FM transmitter on the site. The LineX one looks like it would do a nice job since it just acts as a USB speaker. The question then is figuring out if there's software or whatnot to drive such a thing.

What I think I want:

* 802.1n NAS with no moving parts. 2 64GB USB keys in a RAID-1 config would do the job nicely.
* Enough horsepower to play Oggs, MP3s and FLACs.
* Remote song selection through an Android app.

Anyone know if anything comes close to this? The best thing I can think of is perhaps Debian running on a Linksys NAS box. If I could get that far, then the software seems pretty straight forward if it doesn't exist in some form or other in the Debian archive already.

Uwe Hermann: Help add subtitle support for Miro

Miro 2.0 feed list If you ever wanted to support an open-source project but you are not a programmer, here's one (of many possible) ways to help: The Miro project (Internet TV / Video and Audio Podcast application for Linux, Windows, and Mac OS X) is seeking for pledges/donations that will be used to add subtitles support in Miro. To quote from the announcement:
We re hoping to build real subtitle support into Miro in the next couple months, but we need your help! So we ve started a Kickstarter project to raise $1,000 to develop this feature for Miro on all three platforms: Windows, Mac, and Linux. Can you pledge to help make it happen? One of the great things about the Kickstarter model is that unless we can reach $1,000, your pledge won t be charged.
[...]
(if you live in the United States, donations are tax deductible we are a 501c3 non-profit)
There are 11 days left to make a pledge.

19 September 2009

Sean Finney: dpkg triggers, the lost how-to document

this weekend while giving a bit of TLC to the php packages, i thought i'd finally get around to tackling #490023 and similar bugs, which meant learning about how to use the triggers feature provided by dpkg. unfortunately documentation for this feature was in fairly short supply, and after extensive searching (and by extensive searching i mean typing "dpkg triggers howto" into a google search) i had a couple short manpages (dpkg-trigger(1) and deb-triggers(5)) and an overly verbose and possibly out of date /usr/share/doc/dpkg/triggers.txt.gz. none of these documents really gave a clear "big-picture" for how to get going either. so, as is often the case, i got distracted from my intended task and ended up on a little side-project putting together a nice howto/tutorial for how to integrate dpkg triggers into a package. note that there may be misunderstandings or inaccuracies in the document, as it's based on an afternoon's worth of hacking and q&a directed at #debian-dpkg (thanks to Guillem and Rapha l for fielding those questions). so if i'm off on anything, please feel free to let me know! to get started:
git clone http://git.debian.org/git/users/seanius/dpkg-triggers-example.git

and for those who are too lazy even for that, the README follows: The debian package triggertest This package is a dead-simple "triggers in a nutshell". It shows a few different ways that triggers can be used in practice, to hopefully cover most of the general use cases for triggers. The purpose of triggers Triggers are used to ensure that during the standard package-management process certain operations always run, but not more than necessary. For example:
  • during an upgrade many packages might provide updated texinfo(5) documentation, and triggers are used to ensure that update-info-dir(8) is run without having to be run several unnecessary times from the maintainer scripts of package in question.
  • if a system has both apache2 and PHP's apache2 engine installed, apache2 will need to be restarted whenever new php extensions are installed or removed. Without triggers it would be very disruptive (and inefficient) to explicitly stop and start apache by each PHP extension.
  • update-initramfs(8) should be run once and only once after kernels are updated or certain other packages are installed/upgraded that would want to have a new initramfs generated.
  • other expensive operations that are desirable to be aggregated and consolidated during installs/upgrades, such as update-texmf(8) or scrollkeeper-update.
How triggers work in a nutshell Trigger-using packages can be classified in two behavioral categories:
  • Consumers: packages which declare triggers and thus can be "triggered"
  • Producers: packages which activate triggers (explicitly or implicitly)
When a consumer is triggered, its postinst script is run with the arguments:
postinst triggered "<trigger1> ... <triggerN>"

i.e. $2 contains an iterable list of activated triggers. Note that if a consumer is going to be normally configured (i.e. it is also being updated), then no triggering may occur and thus the standard control flow of the maintainer scripts should still take care to handle this. A "trigger" is declared for the consumer by shipping a file in DEBIAN/triggers (in the case of debhelper based packages, ./debian/consumer-package.triggers will JDTRT). Useful RTFM: deb-triggers(5). Example:
interest /path/to/a/directory
interest my-second-trigger

This declares that the package in question is interested in two triggers. The first trigger has a leading path separator in its name, which instructs dpkg that any filesystem modifications underneath this directory by other packages should cause the respective consumer(s) to be triggered. The second name is completely arbitrary (and also global, so you should probably pick something reasonably specific and identifiable), and will only cause a trigger to be activated when a producer
  1. ships its own triggers file with a corresponding "activate my-second-trigger", or
  2. invokes the trigger explicitly in a maintainer script by calling "dpkg-trigger my-second-trigger".
Another perhaps useful RTFM: dpkg-trigger(1). Examples contained in this source package This source package produces a single "consumer" package and a number of different "producer" packages. The consumer is triggered in various ways, such as:
  • Filesystem updates to a given directory from any other package
  • "activate foo" from a producer's triggers file
  • Activation from a producer's explicit call to dpkg-triggers
In this case the triggers correspond to an "update-foo" and "update-bar" command, which do nothing other than echo to the console that they are being run. In practice they'd correspond to updating some kind of database or restarting a daemon. More information For more information about triggers, see /usr/share/doc/dpkg/triggers.txt.gz, which will make a lot more sense having a working example like this package as reference. Especially useful is "Transition hints for existing packages", which can be used if you need to gracefully handle migrating in support for triggers across other packages not directly under your control. Corrections, suggestions, etc Are appreciated and should be sent to the author, below. -- Sean Finney seanius@debian.org Sat, 19 Sep 2009 15:57:48 +0200

Next.

Previous.