Search Results: "tar"

28 September 2009

MJ Ray: Concentration, Community and Cooperation

I read the news about the launch of Ubuntu User by Linux New Media AG with some sadness. I believe that free software is stronger because of the informal cooperation between different distributions that usually do different jobs. It lets us see what s growing in our neighbour s garden. I m a debian developer, but I don t subscribe to many debian-specific products or services because I like the broader view. Also, at a generally-economically-weak time when some more general publications seem to be dying and restarting, concentrating on the Ubuntu users seems like it probably won t be a big enough audience to sustain a title. The temporary loss of readers could also push other multi-distribution magazines over the brink. Now, I use and support Ubuntu systems from time to time (among others), but it feels like Ubuntu attracts this sort of anti-social single-distribution concentration more often than any other current one. Its users seem even more fixed on it than Gentoo s users, which is almost the opposite of what I expect from a distribution which claims Ubuntu and Free Software are about collaboration and working together . So why does this happen? Are any sociologists researching whether there are really narrower horizons among Ubuntu users than others? Have any Ubuntu community leaders questioned the lack of collaboration with Free Software in launching an Ubuntu-only magazine in a recession?

27 September 2009

John Goerzen: How To Record High-Definition MythTV Files to DVD or Blu-Ray

I ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk. MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920 1080i down to DVD resolution of on a good day 720 480. I couldn t have that. Background My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate. ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920 1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn t be displayed. Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn t want to re-encode any video, both for quality and for time reasons. Attempts So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later. Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed. Next, I tried avidemux. Quite the capable program and I thought I could use it to cut my file into DVD-sized bits. But I couldn t get it to let me copy the valid MPEG TS into another MPEG TS it kept complaining of incompatible formats, but wouldn t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn t really what I was after. Then, I tried mpgsplit. Didn t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner. dvbcut I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R. It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box. So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:
File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
Eeeeepp. So apparently the ISO 9660 filesystem can t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2 s file was surprisingly just a few KB short. But the rest of them weren t. I didn t want to have to go back and re-split the data to be under 4GB. I also didn t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes. So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:
#!/bin/bash
set -e
if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi
if [ " id -u " != "0" ]; then
   echo "This program must run as root."
fi
EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi
# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009
SECSIZE=2048
# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000
echo "Allocating image..."
dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1
echo "Creating filesystem..."
mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"
echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"
echo "Done."
That was loosely based on a script I found on the Arch Linux site. But I didn t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive. So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.

John Goerzen: Jacob Is Three

Jacob s third birthday was Saturday, and I thought I ought to give a bit of an update on him. I ll start with his exactness. Jacob has become very particular about certain things. This morning, I was getting him dressed for church. We had this conversation:
Me: OK Jacob, it s time to get your pull-ups on. Jacob: No, Dad! Not pull-UPS. I don t have free (3) of them! I have one. It a pull-UP. Me: OK. Let s put it on Now, let s put on your pants. Jacob: Daaaad! Not pants! They SHORTS!
So yes, Jacob is trying to correct me about plurals. Also, when I sing Home On The Range to him at bedtime, and get to the line that says, and the skies are not cloudy all day, Jacob will look outside, and point out, the skies ARE cloudy today! (Even if they re not.) But the big thing is guitars. Terah wrote about how this happened. But suffice it to say that he s been almost all-guitar, all the time. At dinner time, he ll hold a spoon across his chest and strum the lots of pretend strings over the big end. Outside, he ll find leaves from irisis, and excitedly yell, I have LOTS of guitars! Pretty soon it wasn t just him that needed to play guitars. He d hand an iris leaf or a spoon to me and say, shall you play it too? So I d start strumming my leaf, and he d smile happily for a second. Then, No, not like that! This way! He meant I was supposed to hold it in a certain position, but it was not always clear exactly what he wanted. Then at supper Friday night, we had some family here, and Jacob said, Shall we ALL play guitars? And he d keep repeating it until we all did it. And he spent about 5 solid minutes getting us each to hold our spoons in the proper musical position. Then he was happy. So you can imagine the excitement when he got a guitar from his grandparents on Saturday. It s a child-sized one, but a real one, with metal strings. He proudly went around sitting on many different chairs to try out his guitar on each one. He played it in the car on the way home. He strummed it at home. He skipped dessert so he could play. And he cried every time he had to put it down: at dinner time, at bedtime last night, when it was time to go to church today. The next thing he wanted was for everyone to sing while he played the guitar. That was a bit difficult, since, after all, it is a 3-year-old playing a guitar he doesn t know how to play. He seemed to really enjoy it though. I have some great photos of him playing his guitars I ll be uploading hopefully this week.

Carl Chenet: Python-keyring in Debian


I have been introducing the Python Keyring library in this previous post. At that time I was looking for a sponsor and thanks to Piotr O arowski, python-keyring is now available in Sid. What for ?
  • You can access the keyring you use on your system (Gnome-keyring, KWallet) to store your app s passwords.
  • It s a unified access to keyrings that the Python Keyring library provides, meaning you can easely write your own backend for your keyring software.
  • You can create your own keyring, crypted on a file, still with the same library.
  • Keyrings are cool and this library provides a great way to play with them, from your Python applications.
  • When mature, the Keyring library will be integrated in the official Python module getpass, which means you re not learning yet-another-soon-unmaintained library.
How ? The source package provides three binary packages :
  • python-keyring : the Keyring library without the support of both Gnome-keyring and KWallet.
  • python-keyring-gnome : the support for Gnome-keyring.
  • python-keyring-kwallet : the support for KWallet.
If you use passwords in your Python apps, it s worth to give python-keyring a try.

Steve Kemp: Looks like me an Vincent caught you boys at breakfast

It is interesting that Fran ois Marier recently posted a brief "howto" document on debugging problems caused by overly-agressive filtering with privoxy, as I've recently been having problems with that tool. My home network frequently changes configuration depending on what I'm concentrating upon, but every few months I'll start/cease using the following tools: Recently I was experimenting with XSS attacks against various browsers, which meant using them for real. As not all browsers have the same anti-advert setups I was running privoxy to filter out web-annoyances, and I spotted a major flaw with it. Unfortunately I can only describe the problem, not reproduce it, or track it down. I'm 80% certain the bug is in privoxy, but the stack is suitably high that determining that for sure is problematic. In short the issue is that HTTP requests would end up being sent to the wrong host: After that clicking around consistently sends requests to the first HTTP host which was accessed succesfully. So, for example, attempting to visit http://foo.com/bar/ will send the request to http://steve.org.uk/bar - which then gives a 404. In terms of setup I use a dnsmasq DNS cache, privoxy and iceweasel from Debian unstable. From the symptoms I'm not sure if iceweasel's "KeepAlive" system is to blame, or if privoxy has a bad cache of hosts. Perhaps it is dnsmasq returning bogus DNS data, or my cable connection itself having DNS issues. Anyway once the symptoms present themselves closing the browser and restarting the cache fixes it. Until the next time which might be hours or days later. I'd report it as a bug - but I don't know where it should be. Privoxy caching things it shouldn't? iceweasel having keepalive issues? dnsmasq returning wrong DNS entries? I'd ask "Have you seen this before, internet world?" but I guess if you have tracked it down it'd be fixed by now, and it clearly isn't! Anyway for the moment I've uninstalled privoxy. ObFilm: Pulp Fiction

Zak B. Elep: (Not) Moving to Manila

So, I'm moving back to Manila tomorrow. I hope the flood's subsided by then. UPDATE: Delaying this, after hearing some more news. I hope it gets better and not worse, as I hear there are a couple more storms coming in.

Eddy Petrișor: Remus Cernea - how media blocks information

I wrote in my last post about my decision to support Remus Cernea's candidacy to he Romanian Presidency.

The fact is that main stream media ignored for a whole month his candidacy and the fact that Radio Rom nia Actualit i (Radio News Romania) removed an internet poll (true, not a social poll, but one of the percepted image of the Internet users) from the site www.politicaromaneasca.ro which credited him with 30% of the preferences of the visitors.




Here (Romanian) is the article on Remus' blog in which he talks about this. And no, he did not set himslef on fire, he set on fire a picture of himself as a symbolic gesture against the media blockade.


As a proof that the poll was not doctored, the first article mentioning Remus Cernea in a search for "presedintia romaniei alegeri" is the 5th unsponsored hit on google.ro. There is an obvious correlation between the position of other candidates in the results and the hits on Google.


If you want to support Remus' candidacy, please sign the support list(ro) or even collect signatures (ro) for him. He needs 200 000 signatures to be able to candidate, a task which is even harder for a candidate without a big party behind him.

You can contact me at eddy.petrisor+remuscernea _at_ gmail.com, for more details or you can contact Remus directly.

Francois Marier: OpenStreetMap/OpenLayers and Privoxy

I have been meaning to switch to OpenStreetMap for a while now, but the fact that it didn't work with Privoxy was holding me back. I don't enjoy surfing the web without protection and Privoxy is one of the most convenient ways of enhancing one's browsing experience on Debian and Ubuntu.

So I decided to spend some time trying to figure out what was wrong. If you want to skip the details and just get the fix, scroll down to the bottom of this post for the solution.

Finding all affected URLsIn order to get a list of all of the URLs that Privoxy filters when one visits the front page of OpenStreetMap, I did the following:

Comparing the filesArmed with the list of all affected URLs, I downloaded them through Privoxy using wget into a privoxy/ directory:
$ http_proxy=localhost:8118 wget URL
Then I downloaded the same URLs without Privoxy into a noproxy/ directory:
$ http_proxy= wget URL
To find identical files, I ran md5sums in each directory:
md5sum *
After deleting all identical files, I was left with:
index.html
OpenLayers.js?1251388304
Which I diff'ed together:
colordiff -u noproxy/FILE privoxy/FILE
The first file, index.html did not have any relevant changes, but I noticed in the second one that a few of these
this.moveTo(this.position)
were replaced with
''.concat(this.position)
Searching for "concat" in /etc/privoxy/, I found that the "jumping-windows" filter was the culprit. After disabling it, all of the problems went away.

I have filed a bug upstream.

The SolutionHere's what you need to put in your /etc/privoxy/user.action:
 -filter jumping-windows 
.openstreetmap.org

26 September 2009

Gunnar Wolf: The bad and the worse; Representative democracies' minimums

Martin rants about the German electoral system. From his rant, I'll pick up only two points And I'll try to connect with Toxicore's excelent (Spanish) blog post, where he quotes political analist Denise Dresser. Dresser has made a great point: Our probably-imposed, legitimacy-impaired president Felipe Calder n has requested the society to talk good about Mexico, to project a positive image of the country. Dresser says, yes, there is a lot of good to talk about the country, and we should emphasize on its richness and beauty, invite people to come and visit, to know what the country is really like. But at the same time, it is our duty to talk bad about the bad areas and decisions of our government, as that is the best (if not the only) thing many of us can do to really get things to happen That is what we can do to push our country's good things forward, to make the country sustainable, to pull attention towards what needs (such as the very very deplorable cases of censorship, human rights violation, ecosystem predation we have seen in the last years). Anyway... What did I want to comment about Martin's post? He criticizes Germany's law requiring a 5% quota for a party to have parliamentary representation. In Mexico, the minimum is 2%. Most people agree, though, that it is too low, and that we should push to increase it. Why? Because the money that is spent in supporting the party system. In Mexico, when a political party fails to get 2% of the vote, it is basically disbanded and it is very hard for it to regroup, to compete again. Many people believe we should aim to a political system with as few political parties as possible (such as the semi-democratic system they have in the USA). I strongly prefer the system found in most European (and even many South American) countries where there is a real wealth of ideological positions represented, and where governments have to be formed by agreeing to form coalitions, as it is almost impossible for them to get full majority. I would much rather see Mexico march towards a parliamentary-based political system, away from the presidential one. Of course, that is almost impossible to expect. With the current political system, we are bound to have forever few monolithic, meaningless political parties. We will likely converge on three blocks, following the current three major blocks (leftoid PRD, centroid PRI, rightoid PAN). They are different in some important senses, yes, but in general they are much the same. I don't hold any hopes to ever see something like the Pirate Party appearing in our system...

Christoph Haas: This should work fine nowadays (about vino, Compiz and wrong optimism)

I ve been back from KDE (4.2!) to Gnome and with the mighty powers of Compiz my desktop is the envy of my Windows coworkers and still geeky functional. Well, somehow functional. For a while I wondered why I can t properly use VNC to remote-control my desktop from home. The connection works and I see the desktop but nothing seems to happen when I click or type. Now I figured out the desktop in the VNC client window is just not refreshed while my mouse and keyboard actions really do what they were supposed to on the VNC server. This is my sad tale of how I learned that a regression may mean driving the motorcycle to work in the middle of the night after cursing VNC. :) After a little searching through the lazyweb I found out that it s a bug reported on Ubuntu already. Apparently Compiz doesn t use the XDAMAGE extension properly. Somehow when using the graphics adapter s magic 3D powers X11 loses control of what sections are changed I m not an X11 guru. At least the client doesn t know when to redraw areas of the screen that have changed. The suggested workarounds were pretty ridiculous. Start another x11vnc session without compiz and connect to that. Very funny. I want to control my running desktop not create a new one. So I switched off compiz at work and voila VNC works again. Some eye-candy and shortcuts are gone but remote controlling the desktop is far more important while im on call. But I quickly missed Compiz. The missing eye-candy wasn t that bad. But metacity s window placement algorithm sucks as badly as back in 2002. As if I can hear it whisper: I have no idea where to put that stupid window. You tell me. I researched a little further and somehow the bug is known to Gnome, too. And other users suffering from the problem are pointed to the VirtualGL page of Wikipedia. Apparently the problem was fixed for several people until a regression was reported where things failed again. Seems like the vino developers removed the piece of code that detected the use of 3D window managers as a 2008 christmas present. Their reason: This should work fine nowadays. Hell, no, it doesn t! Somehow I downgraded vino to version 2.24.1-3 and ran the vino server manually (/usr/lib/vino/vino-server sm-disable) to see if it prints any error message. Indeed it does:
Composite detected, disabling XDamage extension.
This is a workaround while XDamage extension does not work correctly with 3d desktops.
Hopefully it will work at next vino release.
This time it worked well, too . I upgraded to the current version 2.28.0-1 (as in unstable by today) and suddenly my VNC connection had the same problems again. Aha! Let s try the Lenny version 2.22.2-1. Works, too. This has taken me hours to debug. Finally I know I can report a bug against the vino package. I wonder how long it would have taken a mortal to figure that out. :( [Sorry to my sponsorees who's package sponsoring requests were delayed again due to this mess.]

Martin F. Krafft: Voting denied, enter pirates

This coming Sunday, Germany is voting its parliament for the next four years. I wanted to participate remotely, but they didn t let me. The letter, inviting me to order the voting forms, was sent too late. Instead of ordering and waiting for the voting forms, hoping for them to arrive back in Germany before the municipal agency stops all their work at 16:59 on Friday afternoon, I filled in the fields in a way to give my mother full authority to submit my vote. On the document, it only said that she would need a notary procuration, which she has. When my mother tried to vote in my name, they refused her: the vote has to be confidential, and proxies may only vote for disabled people, or those who cannot read. I can understand half of the motivation behind this it guards against abuse but the bottom line is that I am now unable to cast a vote. Democracy stabbed by bureaucracy. Well, democracy to the limited degree that we have it in Germany. I do not mean to criticise at this time that voting in Germany means to place trust into people you d rather not trust, but once they get majority, they can do whatever they want for four years. The specific shortcoming in our democracy on which I would like to focus is the five percent clause , which states that a party needs to reach 5% of votes before they can enter the parliament; if those 5% are not reached, all votes for the party are basically lost. I suspect that this clause is what s keeping back a lot of people from electing Germany s pirate party, which focuses on the importance of freedom in today s media-centric and interconnected world. It seems doubtful that the Piratenpartei will reach the limit this round, but given all the attention they recently received, as well as their plain cause, I would not be surprised if they did. I think it is important for people to vote the party they want, without letting the 5% limit discourage them. Anything else would be undemocratic. Let s return to the pirates: it s paramount for the government to consider freedom and related issues, and to put clueful people on the job. The Piratenpartei is the only party that approaches the issue properly, all the others are just too happy to succomb to castration of personal freedom for such lofty goals as anti-terrorism, or the containment of child pornography. I don t like terrorists, and I think child pornography is among the most horrific ways to abuse defenceless humans in irrevocable ways that I can imagine. But I also don t like losing my deodorant to airport security, cameras tracking my every move, (diletantic) censorship on the Internet, and lack of control over my own data. Time to put an end to that, and to solve problems without castrating freedom. Benjamin Franklin is credited with saying: they that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. The Piratenpartei is probably not capable of leadership at this point, for they are young. But they can help the leading party maintain a bearing. That s what the Greens did, successfully albeit slowly, and I appreciate that the pirates are approaching the matter before it will be too late. The media have not missed a chance to try to stomp over them in recent months, but at the end of the day, after all the slander and some obvious mistakes on the side of the party, I still see a group of people prepared to fight for freedom with an open and honest approach. The run-ins with the media are hardly about content, it seems more that the Piratenpartei is simply unable to become part of the sensationalist mash which is our media-politician ensemble. And I think that s good. I appreciate a party communicating not via their PR office, but with new media, like blogs, or e-mail. I appreciate that their people maintain the personality of a human trying to do politics, not a politician trying to come across as human. If my municipality had let me vote, I would have selected the pirates. They may not make the 5% this year, but I would not let that be a reason against them. If one believes in democracy, then it s important to do it right.

25 September 2009

Julian Andres Klode: cupt and how to write package managers


cupt is a new package manager written in Perl by Eugene V. Lyubimkin, who previously contributed to APT. And more than all, the project makes no sense at all. First of all, there is a language issue. Implementing a package manager in Perl has some major drawbacks. One of the features of APT was it being written in a lower-level language (i.e. C++ which really is below Perl), making it possible to write applications like synaptic and python bindings which in turn lead to applications like gnome-app-install or Ubuntu s new Software Store. Furthermore, writing a package manager in Perl means that Distributions such as Emdebian might not be able to use it since they have excluded Perl due to its space requirements. This becomes even more important considering that cupt depends on even more perl libraries. This means that cupt will never be able to replace APT. Secondly, a package manager should not be designed specifically for one distribution. This is another major drawback of cupt and other package managers such as yum or zypper. The smart package manager, written in Python and funded by Canonical Ltd. is an example for a distribution-neutral package manager. Now let s take a look at package management in modern distributions. Usually we have two levels of package managers, the first being tools like dpkg and rpm which take care of installing and removing the packages and level-2 package managers implementing dependency resolvers and package retrieval from remote locations. Recently, distributions started to add a third layer named PackageKit, which shall provide distribution-independent package management user interfaces. The project was well-received by RPM-based distributions, but failed in Debian-based distributions due to not supporting debconf. Furthermore, adding a third layer just increases the possibility of problems. The right way to do package management is a distribution-independent level-2 package manager written in C. The smart project shows us that this is possible although itself fails to meet the lower-level language (C) requirement. That s why I decided to write a package manager in Vala, a GObject-based language which gets converted to C and then compiled. If successful, this project will be able to replace most of the current level-2 package managers and will also provide the same distribution-independence as provided by a level-3 package manager such as PackageKit. It is also easy to create binding for other programming language such as Python or Perl thus enabling application developers to choose the language they like most. The core of this project is a vendor-neutral library, temporarily called libapt (as the project is called APT2 for now). This library contains all the code which is not specific to a vendor i.e. file retrieval, dependency resolver, caches, etc and is then enhanced by several vendor-specific plugins, each implementing a PackageManager (interface to the distribution s level-1 package manager) and a Repository (well, repositories from which you can download packages) interface. We could even enhance the vendor-independent interface to include more details of a repository. Most repositories nowadays consist of 4 components: A meta index (Release files for Debian, repomd.xml for Fedora/openSUSE/etc.), a package index (e.g. Packages files for Debian, *primary.xml.gz on Fedora, etc. ), a source index (e.g Sources files in Debian) and a files index (e.g. Contents-*.gz for Debian). I took a look at the repository formats of Slackware, openSUSE and Fedora and it seems that this concept can be applied to all of them. So maybe all we need are distribution dependent parsers for those files. One of the most important issues with APT is its use of mmap() for the cache. Using mmap() makes it hard to grow the cache, which is sometimes needed. We see a lot of bug reports from people with too small cache sizes. We can circumvent this problem by utilizing an embedded database like SQLite for this, but we would probably loose some speed and it may be harder to maintain a flexible API. We should see what the best option is here, both ways are possible since Vala 0.7.6 includes my patches for adding mmap(), ftruncate(), mremap() and some other functions. An idea to circumvent the mmap() issue is gathering statistics about the relation between the number of repositories and the size of the cache and then using a value which is slightly above the average statistical value. The project is not very mature yet, it only includes basic library functions for downloading files and parsing configuration files, etc. You can find the (MIT licensed) code at http://git.debian.org/?p=users/jak/apt2.git. I also have some local code for repository management and multi-threaded file fetching, but it s just not ready to be merged yet. Posted in APT2, Debian

Julian Andres Klode: The APT2 project


I just started working on a replacement for APT written in Vala and called APT2 (I know, the name could be better). The main idea behind the project is to create a library for working with Debian repositories and packages, and on top of this library a few applications. This is different from APT because the project focuses on the library part, whereas APT is primarily focused on the application part. I chose Vala for this task because it has an easy syntax and allows me to use the features provided by GLib and Co. really easily. GLib provides many functions needed for APT like file access, unicode handling, hashsums creation. It also provides us with signals which are useful for implementing progress reporting. Another project, called cupt tries to provide an APT replacement in Perl. Apart from the fact that I don t like Perl, it also has the disadvantage of being too big to ship on embedded devices and is not useable from other languages such as Python or C/C++. Yet another project, smart provides a different package manager written in Python. While I like Python, it can t be used from languages such as Perl or C/C++ and is too large. APT2 will only require GLib, Gee and libarchive, which require about 2MB of space; about 1/10 of Perl s size. The core of APT2 is a library called libapt . At the moment, this library provides classes for reading tag files (TagFile&TagSection) and an incomplete configuration class based on libgee s HashMap (to ease coding, it will probably be changed to a special data structure like those in apt-pkg). The next step will be implementing a parser for configuration files (currently we only support in-memory objects) and writing the Acquire system. The planned Acquire system uses GModule to modularize the support for different URI schemes. Each module provides a worker class which implements one or more URI schemes. The first of these modules will provide a worker using GIO, which deals with local file access, samba shares, FTP, SFTP, WebDAV and various other protocols, including HTTP until a replacement has been written (since we don t want to force gvfs-backends). The workers communicate with the parent Acquire object using signals and can cause the whole acquisition to be aborted by emiting an abort signal (or similar). The whole project has just been started and may take some months until it becomes useful (if it becomes useful at all). The code can be found at http://git.debian.org/?p=users/jak/apt2.git and is licensed under the MIT license (see http://www.opensource.org/licenses/mit-license.php). It is a secondary project and will not affect my work on apt (not much yet) and python-apt (0.8 series). Posted in APT2, Debian

Gunnar Wolf: Honduras: .hn NIC attacked/intervened by the de-facto government authorities

I was requested to forward this information to as wide an audience as possible. Possibly two months ago the legality/legitimacy of the actions carried out by the Hondurean armed forces, which captured a democratically elected president and without a judicial order or trial process forced him out of the country, starting a de-facto government, was something questionable. Each day, however, it becomes clearer and clearer the Hondureans are suffering a represive military-backed system which cannot be expected to fulfill as a trustable entity to conduct fair, credible elections. I got this message from a Hondurean friend (of course, whose identity I am not divulging) denouncing the government's invasion of the .hn domain name registry, which is handled by the Sustainable Development Network (Red de Desarrollo Sustentable RDS-HN). The National Telecomunications Comission (Comisi n Nacional de Telecomunicaciones, CONATEL) demands all domain name registration under the .hn top-level domain (TLD) to be suspende, and all the lists and databases regarding said TLDs to be handed over, detailing the IP ranges and the responsibles. They did this under the argument that RDS-HN is an Internet Service Provider (which it is not Being a registrar means they are responsible for the well-keeping of public information and of handling a public good, the .hn TLD, not that they provide any kind of regulated service to individuals or organizations), with military personnel disguised as civilians (and who refused to identify themselves). If you are interested, please read further on the text I received straight from my Hondurean contacts (Spanish) (or its unaccurate but often helpful automated translation to English, done through Google Translate) Even though this information is normally accessible via WHOIS and similar services (this only states clearly nobody in CONATEL was able to do what I just did legally and anonymously from my personal workstation), they did it in such a fashion in order to scare the operators and the society. Honduras is going through a very hard process. Whatever happens there will likely impact on the future reactions to the most retrograd and powerful sectors of society in the rest of Latin America. We do our best (even if as non-Hondureans living outside Honduras it only means raising our voices) to avoid the risk of our region going back to the sad, cruel and bloody 1970s history. [update] My friend Mave, who works at NIC Chile, sent as a comment to this post LACTLD's official stand on this regard (Spanish. English version also available). LACTLD (Latin American and the Caribbean ccTLD's Organization) clearly backs RDS-HN and condemns the illegal government's actions.
AttachmentSize
Detailing the intervention attempt of RDS-HN by the de-facto government agents (Spanish)8.61 KB

Kumar Appaiah: Community at work - a success story

This is a positive event which happened to me recently in a Debian bug report, with regard to audio troubles. There are several positives, and I owe some gratitude to several people in the Debian (and ALSA) community, so I thought a blog post would be a good idea. It also shows the advantage in filing a well-written installation report to the Debian project. Here is the description of events.This was an extremely positive experience. While I agree that the number of people who would encounter this quirk may not be large, it makes me happy that at least some users who use Linux on this hardware will not have to face this issue anymore, irrespective of which distribution they use, as long as it carries an up-to-date ALSA.I wanted to share this experience with you, since this is a glowing example of how the "community" approach works perfectly for FOSS. Thanks to Elimar, for being a concerned maintainer, and to the ALSA upstream developers who were proactive in producing a fix to this problem. And, most of all, thanks to me, for filing an installation report; these things really work! (OK, tongue-in-cheek. But I did take the effort of filing it, so I deserve the credit, don't I?) :-)So, have you filed an installation report for your Debian installation? If not, I urge you to do so; I sincerely believe that filing good bugs and being proactive in helping developers zero in on the problem and fix is an important contribution to free software which any user can do. And the best way to start this process in Debian, is with an installation-report. :-)Tags: debian

24 September 2009

Eddy Petrișor: I'm voting Remus Cernea (Green Party) - Eu votez Remus Cernea (Partidul Verde)

Romania is closing really quickly to the presidential elections.

The options for voting seem to be non-existent and is basically about choosing the smallest evil and I was planning on not voting.

That was until I found out that a personality from the civic society, Remus Cernea has announced that he wants to candidate for Presidency.

He is a candidate from the Green Party and he is in the process of collecting support signatures. 200 000 - two hundred thousands - signatures to be able to candidate for presidency in a country of less than 20 millions.

Yes, Romania has one of the most undemocratic and tailored-for-the-current-in-power election laws. Here (in Romanian) is a summary made by Remus Cernea himself about the election laws in Romania.

A full report was sent by the Romanian Green Party to the European Green Party Committee and the European Greens have expressed their concern about the current state of the law in Romania.



Since I want to support him and since he has been actively ignored (ro) by mass media for the past month (since his announce), I decided to help collecting the signatures. This post is also about making the information flow to the people via the Internet.


So, if, after reading and informing yourselves about Remus Cernea and his civic actions:
you decide that you'd want him in the election, you can contact me (eddy.petrisor+remuscernea _at_ gmail.com) for details how to make this happen.

Christine Spang: FSF Women in Free Software Mini Summit

I was one of the participants invited to the FSF's Women in Free Software mini-summit which happened on the same day as Software Freedom Day this year. That was this past Saturday, September 19th. I was feeling hesitant going in, given the controversy over "why would the FSF run a closed event about such an important topic" and just a general burnt out feeling about the subject on my part. Honestly, I've tended to avoid being much of a feminist in the past and simply hope that people will stop bickering and leave me to my code, especially in the aftermath following well-publicised incidents. This is despite the fact that the thing that got me involved with free software originally was Debian Women, back in late 2004/early 2005. I needed a hook, and yet I haven't felt confident enough to reach out to others now that I'm in. I see now that there are valid reasons for having an event that is purposefully small. We all fit in a nice conference room at the FSF office, the atmosphere was very intimate, we got to learn everyone's names (and even remember them). I met some great local free software people who I hadn't met before. Food was take out from My Thai, an excellent vegan restaurant in Boston's Chinatown. While it sucks to be exclusive, in my opinion the small size really had an effect on what we got done and how we felt at the end. The official minutes from the meeting are online here, and there's a picture and brief blog post from Deborah here. The picture was taken using Cheese! I'm in the middle wearing my ever-popular Best Practical "my free software runs your company" t-shirt. There are only a few things I want to highlight myself.
  1. It was a great idea to invite someone who is involved in freedom movements but is not necessarily heavily involved in free software in particular. Hillary brought fresh insights and helped us draw parallels and come up with ideas that I don't think we would have thought of otherwise. It's easy to get used to parts of a community as being "normal" and I am so happy that we have allies who can show us where we've internalised or just have learned to ignore sexist parts of the community. Women in free software are already in free software and we need to learn to reach out to others who aren't in already.
  2. Cooperative power. There are groups of people that just aren't attracted by FOSS marketing that challenges you to "prove yourself the best" or similar. Just because someone doesn't like coding at 3am doesn't mean he doesn't like coding. If we want to succeed at our mission, we need to stop thinking win-lose and start thinking win-win. This applies to being more inclusive in general, not just for women. It also applies to valuing contributions from those who don't code. We need them too.
  3. There are times when I don't speak up and I should. There are times when I don't blog (or participate in discussions via other media) because I don't feel like dealing with potential community backlash. I am very careful about not stepping on peoples' toes, because I've seen other people get trampled on and am not particularly excited about experiencing that myself. While I'm trying to cultivate courage in myself, it was good to have a reminder that it's not just me.
The current plan is to hold a bigger event that is open to all in the spring. I am so excited about the momentum we are building! I think the biggest thing that I took out of this is that there are still things wrong with the community and that I shouldn't be afraid to speak up and be an activist. Watching people like Deborah and Hillary talk perfectly seriously about making very long-term plans and reaching parity was incredibly empowering. We can do so much better. The time to do it is now.

David Welton: LangPop.com Updates

It took me a while to get the September update of LangPop.com out. The reason: I had to redo the book statistics. Previously, I got those from Amazon's web service, which worked ok as far as those things go. However, recently, I ran into several problems:
  • They started requiring signatures on the requests, which meant I had to stop and fiddle with my code, which I did not appreciate. (If you have this problem and use Ruby, get the amazon-ecs Ruby gem)
  • Their results suffered from some kind of regression in any case: C, C# and C++ all showed up with the same number of results. They didn't use to show the same numbers. Not good from my point of view.
  • They were completely unresponsive to requests for information or help.
  • And for good measure, they hid their information about their products API on some other web site, with no links from their web services page, which makes it kind of annoying to look for it, and suspicious that they may be removing it at some point in the future.
All of these together made me decide it was time to switch data providers. Where to go? The answer was obvious: back to my home state of Oregon, to Powell's Books, a huge independent book seller. If you ever go to Portland, do yourself a favor and go to Powell's. It is a city block full of books on multiple floors. When I lived in Portland, I would go there before eating to give myself a reason to leave (although they do have a cafe...). And that's just the regular book store. They have another huge space a few blocks away dedicated entirely to technical books. Powell's is one of my favorite things in Oregon, and a great experience. Anyway, I decided that they're big enough, and do enough business online that I'd ask them about getting data, so I sent some email asking after what options were available. I got a very helpful answer from a real person, CJ Stritzel, who was kind enough to explain what the options were, and work with me to help find the best one for what I needed. Definitely better than "/dev/null", the Amazon message board! The API is still something they're working on, but it works well enough for what I need, and I'm quite happy with the situation. It has caused some changes in the rankings, though: they show Java books as being by far and away the most popular, which has pushed that language into the top spot in the rankings. We'll see over the coming months; I may rework the queries I'm using some as their API stabilizes and I experiment with different combinations of keywords. In any case, thanks Powell's, and enjoy the new statistics everyone.

Wouter Verhelst: RedHat is so medieval

So there's three systems. One of them runs CentOS 4, the other two run RedHat EL 4. One of the RedHats has been in use for quite a while, but the other two are freshly installed, and would need to have the same functionality as the first, but don't yet, mainly because they miss a number of packages. So we do this.
[wouter@working-host ~]$ rpm -qa --qf "% NAME "   sort -u > working-host
[wouter@CentOS ~]$ rpm -qa --qf "% NAME "   sort -u > CentOS
Next, we compare the two:
diff -u working-host CentOS   grep '^-'   tail -n +2   sed -e 's/^-//' > CentOS-missing
At this point, the file CentOS-missing contains some 403 lines. So how does one install packages on CentOS4?
[root@CentOS ~]# yum install $(cat CentOS-missing)
It starts thinking about it, then after several minutes comes back to me with "foo is already installed in an older version. I can't do what you're asking me"
[root@CentOS ~]# grep foo CentOS-missing
[root@CentOS ~]#
In other words, it decided that, to satisfy one of the dependencies of one of the packages I'm asking it to install, it needs to have some other package installed, too, so adds it to the set. Then figures out that it's already installed, at an older version than what it was trying to install. Then bombs. Okay Wouter, what were you thinking? Surely you can't expect a package manager to suddenly update packages when you ask it to install something else? That would mean upgrading and installing in the same command execution! Oh, the horror. So, okay, we run yum update first. Do you think that fixes it? Of course not! It now comes back with... a message saying that a newer version of what we're trying to install is already installed. Sigh. So I'm now running 'for i in $(cat CentOS-missing); yum -y install $i; done' on this host. Since it takes yum several minutes to resolve dependencies, that means it's going to take a while to have it install 403 packages. The not-yet-working RedHat 4 installation seems to have the exact same problem, except that up2date takes even longer to resolve dependencies, so we're not beyond the 'upgrade' step there yet. I'm sure it will come up with all kinds of other interesting failure modes. Luckily the RedHat-missing file is much smaller (83 packages). How do RedHat users not eat their keyboard in frustration? Note: yes, I'm aware of apt-rpm. Unfortunately, that is not supported on 64bit installations, because apt (in neither its dpkg nor its rpm variant) understands the notion of multiarch. At this time.

23 September 2009

Jon Dowland: win7 window management

Since I've started today writing about UI issues, I might as well continue. I stumbled across a satirical review of Windows7 on kuro5hin (who knew kuro5hin was still being updated?) which highlighted one or two new window management features:
You can drag a window to the top to maximize it, which is surprisingly useful. Dragging it to the side maximizes it to use half the screen, which is great for two Explorer windows for moving stuff around.
I've always found old-windows' right-click-on-taskbar menu items for auto-arranging windows very useful and wished that stock GNOME was as useable (you can get a bit closer with Devil's Pie. I might still one day resurrect my own window manager idea). This is probably no more discoverable than the old way, but useful none-the-less.

Next.

Previous.