Search Results: "xam"

2 May 2008

Simon Richter: .deb versus .msi

I've learnt a lot about building MSI files during the last days. In a way, it is very similar to building Debian packages -- you hand some magic tool a directory with files and a directory with meta-information. The main difference is that MSI files cannot call arbitrary commands in their installation sequence, which serves an useful purpose: as long as they make do with the standard actions provided with the base installer, it is guaranteed that you can undo all changes, and you can get a full overview what the installation will do, without running any untrusted code. If the standard actions aren't enough (for example, they don't cover driver installation), you can ship a DLL that can be loaded into the installer. You lose a bit of the audit functionality there, but that is not a problem in practice as these plug-ins are signed separately from the package, so for every genuine problem, there is already a plug-in that you can simply "merge" to your package.

30 April 2008

Uwe Hermann: Redirecting audio to a remote host using esddsp

There are situations where you might want to redirect some audio you're playing on your local computer to another computer's speakers, potentially in a different room, or even anywhere on the Internet. One of many possibilities to do that is to use the Enlightened Sound Daemon (EsoundD, or esd). It ships with a program called esddsp (apt-get install esound-clients) which can redirect various audio sources. First, you have to start the esd daemon on a console on the remote host (the one which should output the audio on some speaker, for example 192.168.0.xxx) e.g. like this:
  $ esd -public -nobeeps -tcp
You can do this as regular user (no need to be root) if you have the proper permissions. You also need to allow connections on port 16001 in your firewall settings. Then you can redirect audio to that daemon from another computer. In this example I'm redirecting some music using various players:
  $ esddsp -s 192.168.0.xxx:16001 mpg321 -o esd foo.mp3
  $ esddsp -s 192.168.0.xxx:16001 mplayer -ao esd foo.mp3
  $ esddsp -s 192.168.0.xxx:16001 ogg123 -d esd foo.ogg
This also works fine for videos, in which case you can redirect the audio (but not video):
  $ esddsp -s 192.168.0.xxx:16001 mplayer -ao esd foo.mp4
For the video player Miro, I've recently documented this in the Debian package's README.Debian file. Basically you have to edit ~/.xine/config and enable audio.driver:esd there, then start Miro with
  $ esddsp -s 192.168.0.xxx:16001 miro
Audio will be emitted on the remote host, video remains on your local PC. Some programs may also support esd natively, in which case esddsp is not required, e.g.
  $ ogg123 -d esd -o host:192.168.0.14:16001 foo.ogg

Adeodato Sim : Release work, Dudesconf and, oh my, with Minirok to Sevilla

These last two weeks most of my time has been sucked into getting Python 2.5 as default into testing. That’s done now. I made use of the block uploads thingie ftpmaster implemented for the release team to use. Basically, if your package could disrupt an “almost there” transition, the upload will be rejected. The blocks were in place for 5 days, which I think it’s acceptable. As long as we don’t end blocking stuff for very long, I think we should be fine. See the end of this post for more about this. Though it was quite a bit of work, I’m very glad I took care it myself, since now I really feel I’m 100% back to Debian, after the time I spent off for health reasons. Update: Oh, and I forgot to say: having control over britney has really really helped. Thanks a lot Joerg for that.
Dudesconf Tomorrow I’m leaving to Coru a for Dudesconf, which is a kind of Debconf-ES. I’m giving an introductory talk to Git, a semi-lighting talk about grep-dctrl (30 min.), and (gasp) a talk about Debian packaging with a VCS. We may have a Debian Quiz as well. I’m so looking forward to it, since many people who’ll attend are amongst my most loved ones, and I already missed last year’s since I wasn’t fully recovered yet. See you there!
Minirok and Sevilla One of the reasons I wasn’t fully back to release management during the past 5 months or so is because I spent as much time as I had doing development for Minirok. I don’t think I mentioned here before, but I was participating in a Free Software Contest for college students organized by the University of Sevilla, Spain. Such effort finally paid off, since Minirok was elected as one of the finalists. This means next week I’ll go to Sevilla, to make a presentation of the project, and who knows what more. ;-) I’m very excited.
Finally, more on blocking uploads This Python 2.5 transition was the first time the block uploads feature was used, and there were a couple bumps along the way. In particular, a couple packages were blocked, when they shouldn’t have been (libqt4-ruby and evolution-sharp), and one needed package was not blocked, though Rene Engelhard thankfully spotted it very quickly (mono). The problem is it’s not completely straightforward to generate a list of all the stuff that could possibly affect the transition. What I did was to make a run of britney on an arch that had all the needed bits in place, and block all the packages that migrated together as a result of the hint. This fails in two ways: The second problem, though, can be fixed by parsing the excuses list and blocking stuff that some bits of the transition depends on. Easy enough. Yet, there are more cases when things can go wrong, for example a shlibs-bumping upload of a package (say, sqlite3) linked against by some package still needing a couple of builds (say, qt4-x11). For that, blocking uploads is not an option, since that’d be an insane amounts of packages that, furthermore, can be uploaded if they don’t bump shlibs, so I guess we’ll have to ask in the next release update that shlibs-bumping uploads are coordinated in -release too, at least when close to finishing a transition.

Biella Coleman: The Future of the Internet Depends on its Past

A few weeks ago, NYU hosted an interesting event about the future of the Internet, appropriately tittled The Futures of the Internet, the video of which is now available here. One of the panelists was Jonathan Zittrain (who recently wrote an important new book bearing the same name as the event) and during the talk he provided a few ideas about how geeks and developers can help secure the Future of the Internet. While I agree with a lot in fact most of his assessments about the state and fate of the Internet as he lays out in his book and his talks, his characterization of geek/hacker/developer politics is not one of them. Basically, one of Zittrain’s claims is that developers are not doing enough to save the Future of the Internet and it is their rampant, Atlas-like libertarianism, which is, in part, to blame (first made 37:20 minutes into the video for those who want to listen to the actual comments). They have little-to-no political consciousness, are too cool to care about the fine print and they don’t care about the broader politics of the the Internet because they assume that they can just hack around any sort of barrier and impediment. While we can, without a doubt, identify a strain of libertarianism among hackers, it is by no means representative of all of geekdom and in fact, is becoming more and more a worn out 1990s stereotype/clich as time passes than an accurate representation of what is a far more variegated set of ethics and practices among hackers (and I will soon publish an article on this topic). It also completely fails to capture the ethical spirit as well as sociological, and political workings of one of the most important strains of hacking free and open source software which not only powers most of our (open) Internet but which in fact has provided a pretty hefty ethical backbone by which to conceptualize one of the ways we should think about the fight for the future of the Internet. Ok, time for a rant now :-) Geeks not only designed the Internet, an indisputably revolutionary medium, but also implemented, and continue to maintain it, and then in their copious spare time, also engage in fighting back the political, legal and corporate encroachment which threatens to limit the very revolutionary nature of the Internet (as Chris Kelty’s new book on Free Software argues). If these acts by geeks are not enough political action, then maybe the development of not just one, but multiple entirely open and free alternatives to the only two proprietary operating systems that exist today might be a political act that would satisfy? Many would agree that even simply using a free operating system is a political act. It would be better to claim that individuals, lawyers and other political actors are not doing enough to save the Future of the Internet, rather than imploring the already overtaxed geeks to set aside everything that they are already doing to do something even more. (end rant) It also seems that when it comes to political questions related to the Internet, net neutrality being the hot topic now, or fighting restrictive and problematic laws like the DMCA, one of the only groups of people (outside of lawyers and librarians) to actually understand and dissect the fine print (and geeks actually are pretty attuned to and like to dissect the fine legal print), to protest these unsavory laws, and to support the organizations who are doing something about it (like the EFF), are geeks and hackers. While many geeks are not necessarily keen on conceptualizing their labor in traditional political terms, or aligning their technical projects with a political affiliation, and yes would rather just be writing good code, they do fight for their productive freedom and this productive freedom just happens to relate to most questions and concerns related to an open, accessible, and tweakable Internet, built by the geeks, lest we forget What was perhaps most surprising was that he also seemed to think that geeks and developers have not turned to “apprenticeship,” nor policies and procedure to coordinate their development projects, unlike Wikipedia, which he considers a shining example that geeks should look towards as a beacon of policy that geeks should consider emulating in their projects (comments made answering my question). He clearly has not been hanging out with any Debian developers in the last 10 years nor has he gone through their New Maintainer Process ;-) In other words, he seems to think they are allergic to regulation due to their accentuated libertarianism, or are against structure because of their anarchism, neither which is remotely true. I think I found this characterization most ironic and problematic for before Wikipedia was even an entry on a Wiki, projects like Debian (and most other F/OSS projects) were transforming and changing to integrate normative procedures and policies that allowed a group of people to work together, scale, grow and deal with crises’. No, they don’t have the Wikipedia badge system, but that system is emblematic of Wikipedia’s own transformation into integrating its own normative procedures and policies for working together, not an example of an idealized policy system that other projects are too primitive to have evolved into yet. About one hour into the talk when questions opened up, I objected to his characterization, but given his answer back to me, I did not make much of a dent in his thinking. Another lawyer Tim Wu (who also wrote a wonderful book on the Internet) chimed in to give me some props and also made a good point that even if geeks are the only groups of people who would storm AT&T and know intimately about the importance of net neutrality, there is a lot of room for thinking about how to strengthen and improve the tactics and politics among geeks and developers so that we can ensure the type of open and generative Internet and set of technologies we value. As part of thinking and rethinking new strategies, it is as key to acknowledge and honor the past. In this regard, free software development has been pivotal both in terms of providing software (and making it is an important political act as is choosing to use free software over propriety software) and a set of important set of ideas that a lot of lawyers like Yochai Benkler and Lawrence Lessig have run with to make some important political claims of their own. So despite my rant above, which was a rant and thus exaggerates things to some degree, I do think there is much more that geeks and non-geeks can do, such as help translate these uber-geeky issues into less geeky terms (and actually this is already being done by some geeks as the work of Jelena Karanovic has shown, or translate the technical issues into new domains as the uber-geek Karl Fogel is doing with question copyright but first lets give credit where credit it due and recognize that labor is political

28 April 2008

Neil Williams: estron developments

I've got most of the build system / packaging stuff done for estron now. There's a basic Yelp manual pending completion of the Estron GNOME application, there are some refreshed screenshots, updated the website CSS and added an icon, updated Glade files using Gtk stock items for the buttons.

http://estron.alioth.debian.org/

It's grown into yet more packages from the one source:

ii estron 0.7.0-1 data-centric development interpreter for GNOME
ii estron-doc 0.7.0-1 data-centric development interpreter (help files)
ii estron-gnome 0.7.0-1 data-centric development environment for GNOME
ii libestron-dbg 0.7.0-1 estron debugging library
ii libestron-dbi-driver 0.7.0-1 DBI driver for estron
ii libestron-dev 0.7.0-1 estron development interface
ii libestron-doc 0.7.0-1 libestron development documentation
ii libestron-odbc-driver 0.7.0-1 ODBC driver for estron
ii libestron-pg-driver 0.7.0-1 PostgreSQL driver for estron
ii libestron0 0.7.0-1 data-centric interface support library

Only the docs are of any particular size - the interpreter is tiny. (estron-gnome is simply an estron file and companion glade file, run by the estron interpreter so it's Architecture: all like all estron applications.)

Lots to do still - upgrading the Gtk code, adding new features to the interpreter to fully support the development environment and finally putting some details into the user manual (estron-doc) - but at least the package builds easily and cleanly. Once I have a working estron-gnome setup, I'll put the POT file out for translation too (around 120 messages so far).

There isn't any need for debconf and things, this will use existing database configuration tools like mergeant - that needs work too because current estron example files have hardcoded database connection strings.

Stefano Zacchiroli: pts testing transitions

PTS bit of the day: show testing transitions Tiny teeny PTS bit of the day: the PTS now provides information about testing transitions. If your package is part of an ongoing transition its uploads to unstable will be blocked by DAK. The PTS informs you about that and points to extra information and upload alternatives. At the time of writing a couple of examples are: aubio and dballe. Though now that we have such powerful tools, Debian will be so fast at completing the python2.5 transition that you won't have time to see the new PTS messages ;-)

26 April 2008

Erich Schubert: BPM Tap Toy

Since I didn't find a similar tool for Linux, I've hacked together a tiny tool to do BPM counting. The usual stuff: tap any key with the music to get a BPM estimation. There is no music analysis done.Screenshot of BPM Tap ToyTo achieve better results, my tool uses an exponentially weighted average. It also computes a confidence interval. So if the song changes tempo (and quite some jazz songs do) or doesn't keep the tempo constantly (live music with human drummers often shows some tempo drift. It's just the drum computers that keep their speed very well...) - the tool can handle that quite well, while also not being overly sensitive to tapping errors (as you can see in the image).I figure there might be even better estimation tricks (e.g. doing some error detection and removing them), but this one was just very simple to implement. And by looking at the graph and the error interval it's quite easy to check that you didn't any recent mistakes and the result is okay.The Python code is about 158 lines SLOC, and a small UI description generated with Glade. It's built with PyGTK and uses Cairo for the plots, so it should be very platform independant and even run on Windows.Download will be available sometime when I get around to clean it up a little. There is for example still the code in there to compute plain averages, which can't handle songs with multiple tempos at all.[Update: it now also has a tiny 'flasher' icon that will show you the current tempo estimation. This is great for checking the quality of the result - if it keeps on flashing in sync with the music after you've stopped tapping, it's good. I was also pointed to GNU GTick which is a full featured metronome application and also has a tap button.]

Theodore Ts'o: Donald Knuth: I trust my family jewels only to Linux

Andrew Binstock interviewed Donald Knuth recently, and one of the more amusing tidbits was this:
I currently use Ubuntu Linux, on a standalone laptop it has no Internet connection. I occasionally carry flash memory drives between this machine and the Macs that I use for network surfing and graphics; but I trust my family jewels only to Linux.
More seriously, I found his comments about about multi-core computers to be very interesting:
I might as well flame a bit about my personal unhappiness with the current trend toward multicore architecture. To me, it looks more or less like the hardware designers have run out of ideas, and that they re trying to pass the blame for the future demise of Moore s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won t be surprised at all if the whole multithreading idea turns out to be a flop, worse than the “Itanium” approach that was supposed to be so terrific until it turned out that the wished-for compilers were basically impossible to write. Let me put it this way: During the past 50 years, I ve written well over a thousand programs, many of which have substantial size. I can t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading. Surely, for example, multiple processors are no help to TeX…. I know that important applications for parallelism exist rendering graphics, breaking codes, scanning images, simulating physical and biological processes, etc. But all these applications require dedicated code and special-purpose techniques, which will need to be changed substantially every few years.
This is a very interesting issue, because it raises the question of what next-generation CPU’s need to do in order to be successful. Given that it is no longer possible to just double the clock frequency every 18 months, should CPU architects just start doubling the number of cores every 18 months instead? Or should they try to concentrate a lot more computing power into an individual core, and optimize for a fast and dense interconnect between the CPU’s? The latter is much more difficult, and the advantage of doing the first is that it’s really easy for marketing types to use some cheesy benchmark such as SPECint to help sell the chip, but then people find out that it’s not very useful in real life. Why? Because programmers have proven that they have a huge amount of trouble writing programs that take advantage of these very large multicore computers. Ultimately, I suspect that we will need a radically different way of programming in order to take advantage of these systems, and perhaps a totally new programming language before we will be able to use them. Professor Knuth is highly dubious that the later approach will work, and while I hope he’s wrong (since I suspect the hardware designers are starting to run out of ideas, so it’s time software engineers started doing some innovating), he’s a pretty smart guy, and he may well be right. Of course, another question is whether what would we do with all of that computing power? Whatever happened to the predictions that computers would be able to support voice or visual recognition? And of course, what about the power and cooling issues for these super-high-powered chips? All I can say is, the next couple of years is going to be interesting, as we try to sort out all of these issues.

David Pashley: Violating Perl Module Namespaces

Perl doesn't enforce access to modules' namespaces. This would usually be considered a bad thing, but sometimes it allows us to work around problems in modules without changing their code. Here's a perfect example: I've been writing a script to talk to an XML-RPC endpoint, using Frontier::Client but for one of the requests, the script throws the following error:
wanted a data type, got  ex:i8'
Turning on debugging showed the response type was indeed ex:i8, which isn't one of the types that Frontier::Client supports.
<?xml version="1.0" encoding="UTF-8"?>
<methodResponse xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions">
  <params>
    <param>
      <value>
        <ex:i8>161</ex:i8>
      </value>
    </param>
  </params>
</methodResponse>
Searching through the code shows Frontier::Client is a wrapper around Frontier::RPC2 and the error message happens at the following section:
     elsif ($scalars $tag )  
       $expat-> 'rpc_text'  = "";
       push @  $expat-> 'rpc_state'   , 'cdata';
     else  
       Frontier::RPC2::die($expat, "wanted a data type, got \ $tag'\n");
    
So we can see that it's looking up the tag into a hash called %scalars to see if the type is a scalar type, otherwise throws the error we saw. Looking at the top, we can see this hash:
%scalars = (
    'base64' => 1,
    'boolean' => 1,
    'dateTime.iso8601' => 1,
    'double' => 1,
    'int' => 1,
    'i4' => 1,
    'string' => 1,
);
So, if we could add ex:i8 to this scalar, we could fix the problem. We could fix the module, but that would require every user of the script to patch their copy of the module. The alternative is to inject something into that hash across module boundaries, which we can do by just refering to the hash by it's complete name including the package name. We can use:
$Frontier::RPC2::scalars 'ex:i8'  = 1;
Now when we run the script, everything works. It's not nice and it's dependent on Frontier::RPC2 not changing. but it allows us to get on with our script.
Read Comments (1)

Craig Sanders: Keeping apt archives empty or not.

In this post, Steven Hanley wonders how to keep /var/cache/apt/archives empty on a machine that has a full mirror of debian on it. That’s actually the wrong question. The right question is ‘Why is /var/cache/apt/archives even relevant on a machine with a full mirror of debian?’ /etc/apt/sources.list has always supported file:/ URLs. for example, sources.list on my local debian mirror looks like this:
deb file:/home/ftp/debian unstable main contrib non-free
deb file:/home/ftp/debian-multimedia/ unstable main
With a file URL, apt-get doesn’t need to download any packages, and thus doesn’t need to cache anything in /var/cache/apt/archives. No need to keep it empty, it just is empty. In a related post, Andrew Pollock suggests adding DPkg::Post-Invoke “apt-get clean”; ; to /etc/apt/apt.conf. That would be fine if you ONLY ever ran “apt-get update ; apt-get dist-upgrade”. It’s not fine if you run “apt-get -d” to download packages first, then install a few packages with “apt-get install package”, and then run “apt-get dist-upgrade”. If you try that, then the DPKG::Post-Invoke rule will clean out /var/cache/apt/archives when you install the individual package(s). When you then run the dist-upgrade, apt-get will have to download all the packages again. Better just to get into the habit of running apt-get clean or autoclean occasionally when you want to reclaim some disk space. Syndicated from Craig Sanders' Errata: Miscellaneous Thoughts Keeping apt archives empty…or not.

25 April 2008

Ian Wienand: Conditional Operator Harmful ? yes : no

It seems existing versions of gcc don't warn when you use an assignment as a truth value as the first operand to the conditional operator. For example:
int fn(int i)
 
	return (i = 0x20) ? 0x40 : 0x80;
 
Presumably you meant ==, but unfortunately that does not warn with gcc (maybe it will one day). We can use the tree dumping mechanisms (previously mentioned here) to confirm our problem. For the following function gcc creates:
fn (i)
 
  int D.1541;
  int iftmp.0;
  i = 32;
  if (1)
     
      iftmp.0 = 64;
     
  else
     
      iftmp.0 = 128;
     
  D.1541 = iftmp.0;
  return D.1541;
 
As you can see, this isn't the result we wanted. This can be quite nasty if you use the conditional operator in a hidden fashion; for example behind an assert. A common idiom is:
#define ASSERT(x) (x) ? : fail()
/* programmer now subsitutes a == for */
ASSERT(x = y);
You'll currently get no warning you've slipped up and used the assert wrong. Moral of the story: a quick audit of your code might turn up some surprising uses! However, in this case, the Intel compiler picks this one up:
$ icc -c test.c
test.c(3): warning #187: use of "=" where "==" may have been intended
  	   return (i = 0x20) ? 0x40 : 0x80;
Although it's often not trivial to move to another compiler, as a rule I would recommend trying alternative compilers for your code as it's a great sanity check.

Bernhard R. Link: Some basics about make

Till today I thought make was a very simple concept, but looking at other people's debian/rules files I start to lose that faith. So let's begin with some basics (as I guess many reading this are maintainers for Debian packages, and you might need some of this knowledge): As you will already know, the most important part of a Makefile is a rule. Each rule is there to produce something and has prerequisites, i.e. things that have to be done before. So far so simple. When you think about this way, the first pitfall is already no pitfall anymore:
   build: patch build-indep
   build-indep: build-indep-stamp
   build-indep-stamp:
   	$(MAKE) doc
   	touch build-indep-stamp
   
There are two mistakes in this. First of all, the clean is only called on ./debian/rules build, but not on ./debian/rules build-indep. And then patch is only called by the build target. But what you really want is that the source is patched before you do something, so it is something do be done before build-indep-stamp. The pitfall with this error is that you will not see it most of the time. As make usually processes targets in the order it finds them, it usually runs patch first. Except when you have multiple processors and tell make to make sure of them (and even then there is a chance it might work as the command to run first is fast enough) or if someone trusting on what he learned about make does call ./debian/rules build-indep-stamp build. I guess a reason one sees this so often is the next pitfall. Most likely previous the following was tried:
   build-indep: build-indep-stamp
   build-indep-stamp: patch
   	$(MAKE) doc
   	touch build-indep-stamp
   patch: patch-stamp
   patch-stamp:
   	whatever
   	touch patch-stamp
   
The pitfall with this is some new concept. The problem is that the patch rule is phony. A rule is called phony when it does not produce the target it claims to do. The classical example is a clean target. You do not want that a clean target creates a file clean and the next time it is called says "already everything done". Targets get phony by telling make it is (via .PHONY:) or by just not producing the file it tells to produce or (suprising to many, it seems) by having a phony target as prerequisite. In the above example the patch target gets implicitly phony, as it does not produce a file called patch. Thus after having build the source and calling binary to create packages, this will most likely in some way depend on build-indep-stamp. But when make thus looks at build-indep-stamp to decide if it already there and even though it sees the file produces there with the touch command, it cannot determine if that is up to date. It depends on patch and there is no file called this, thus make must assume it is not up to date, thus build-indep-stamp has become effectively also phony, in the sense of having to be called every time it is depended on. (In case you have not noticed, the fix would have been to make build-indep-stamp depend on patch-stamp instead or to go without patch-stamp and make patch the file to be generated). Thus, you can put as many -stamp files as you want in a row, as long as you have a single phony prerequisite in them, it is all void.

Erich Schubert: Alsa and pluggable devices

Any ALSA experts around that can help me with the following setup?I want to create a virtual device that does the following: - send the audio data to my regular onboard sound devices - when I plug in some USB audio device, send it there as well OR just thereThe difficult part is the plugging part. There is an example in the Wiki how to use two cards as one. This works, but only if the USB audio device was plugged in before starting the audio application and it is not unplugged either.Obviously, switching the configuration files on plug events is another possibility, but again this requires applications to be restarted. :-(What I want to achieve is simple: when I plug in the USB device, I want to switch my music playback there (since it will be connected to my stereo). System events, voice calls etc. however should remain on the system audio.If you have any solution, please send me an email at erich AT debian DOT org.(And yes, I know about PulseAudio, and I use it. But only on top of ALSA)

24 April 2008

Joachim Breitner: Pausable IO actions for better GUI responsiveness

For a university seminar I m currently writing a GUI program to view fractals based on simple iterated function systems (only similaritudes allowed). It supports three different drawing algorithms and you can edit the IFS by dragging squares around on the screen. But this post is not about this program (I might present it later), but how Haskell allowed me to solve a problem very nicely:The first instances of the code had a problem that a lot of GUI programmers know: The drawing of the fractal took quite some time, and during that time the gtk main loop is blocked and the program becomes unresponsive. At first I avoided this problem by manually splitting the drawing function (e.g. by repeatedly increasing the resolution) and kept re-drawing it at higher resolutions in an idle handler, so at least I could interact whenever one resolution has finished drawing. It worked somewhat but it was not easily done for the other, not pixel based, algorithms and it was ugly.So I wanted a way to (a) pause the drawing at any convenient point, to resume it later, and (b) safely abort the drawing if what I m drawing has changed and I need to restart.A common solution to this would be to do the drawing in a separate thread, so (a) is actually not needed, but I did not know if I can safely do (b), and I have heard that threads cause problems with gtk.So I tried to dig deeper for the hidden treasures of advanced haskell programming: I need a monad transformer! I expected the infamous ContT monad transformer to help, but I couldn t figure out how, and I started to create my own monad transformer, called CoroutineT.I tried to figure out what an action of type (CoroutineT IO a) should do, and I came up with this type signature:
pausingAction :: IO (Either (CoroutineT IO a) a)
which means that after the pausingAction is done, it is Either paused (and I get back another pausingAction to run when I want it), or it is done (and I get the result). Note that I m writing IO here, but it can be any monad.The definition of the datatype and the monad instance came mostly from trying to make this type work (yay to haskell s type system, less thinking required), and looks like this:
data CoroutineT m a = CoroutineT  unCoroutineT :: (m (Either (CoroutineT m a) a))  

instance (Monad m) => Monad (CoroutineT m) where
return v = CoroutineT (return (Right v))
a >>= b = CoroutineT $ do
r <- unCoroutineT a
case r of
Left paused -> return $ Left (paused >>= b)
Right unpaused -> unCoroutineT (b unpaused)
This translates to english like this: A call to return is not paused. When an action is already paused, further actions should be run after the paused action is resumed. When an action is not paused, further actions can happen now.Like every well behaving monad transfomer, I also need a "runCoroutineT" function to start the coroutine. I probably could have used unCoroutineT directly, but for my use case (GUI drawing) I did not need a return value, so this function is more handy:
runCoroutineT :: Monad m => CoroutineT m () -> m (Maybe (CoroutineT m ()))
runCoroutineT a = either (Just) (const Nothing) liftM unCoroutineT a
Nothing surprising here, basically just turning the Either into a Maybe. So it becomes clear how to do (b): We can just throw away the resume action returned by runCoroutineT (if any). The more interesting thing is how we do (a): We need a pause action of type (Coroutine m ()). But how should it work? I did not really try to understand why it works, but by looking at the types, I came up with this:
pause :: Monad m => CoroutineT m ()
pause = CoroutineT (return (Left (CoroutineT (return (Right ())))))
Yes, it sounds like some dance step instructions (read the second line out aloud!), but it works somehow.So here is some example code: I have a pausable IO action that counts from one to ten, pausing after each number. I also have function that resumes an pausable action up to n times:
example n = keepGoingFor n $ do
        liftIO $ putStrLn "This is the coroutine"
        forM_ [1..10] $ \i -> do
            liftIO $ putStrLn $ "Counting to "++ show i ++" while you keep calling it"
            pause
  where --keepGoing :: Monad m => CoroutineT m () -> m ()
      keepGoingFor 0 _   = putStrLn "Here I just abort the run"
      keepGoingFor n cor = do
        resume <- runCoroutineT cor
        case resume of
            Just runAgain -> keepGoingFor (n-1) runAgain
            Nothing       -> putStrLn "Finally stopped"
And here is the output of two different runs:
*CouroutineT> example 5
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Here I just abort the run
*CouroutineT> example 14
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Counting to 6 while you keep calling it
Counting to 7 while you keep calling it
Counting to 8 while you keep calling it
Counting to 9 while you keep calling it
Counting to 10 while you keep calling it
Finally stopped
So it does really works fine, and it proved very useful in my GUI drawing problem. For that, I created this nice control structure which works like mapM_, but calls pause every n iterations, and therefore hides the pausing stuff almost completely:
pausingForM_ :: Monad m => Int -> [a] -> (a -> CoroutineT m ()) -> CoroutineT m ()
pausingForM_ period list action = pausing' 0 list
where pausing' _ [] = return ()
pausing' n (x:xs) = do action x
if n==period then pause >> pausing' 0 xs
else pausing' (n+1) xs
I have put the complete module (including instances omitted here) in the darcs repository that might later also contain the fractal drawing program.

Stefano Zacchiroli: unexpected kudos from TBL

Sir Tim Berners-Lee kudos the Debian packaging system I'm currently in Beijing to deliver a talk at the WWW2008 conference, and I just came back from the usual magnificent banquet of big conferences. Before the banquet it was scheduled a keynote talk by Sir Tim Berners-Lee. What is the least thing you expect to hear from Sir Tim mouth in such a talk? I for sure hadn't expected to hear kudos to the Debian packaging system! But in fact I did hear that and apparently I was not the only one (though probably I was the only one who got excited about it, attracting the attention of my table-mates). He was talking about various shortcoming to be addressed in the future of the Web, and one was the hackish way in which Javascript/AJAX applications are deployed on the web, with no encapsulation/packaging of libraries. Well, as an example to tend to he then mentioned the packaging system of our beloved distribution. It is really the time to say: we are just one step away from the total world (wide web) domination.

David Pashley: Violating Perl Module Namespaces

Perl doesn't enforce access to modules' namespaces. This would usually be considered a bad thing, but sometimes it allows us to work around problems in modules without changing their code. Here's a perfect example: I've been writing a script to talk to an XML-RPC endpoint, using Frontier::Client but for one of the requests, the script throws the following error:
wanted a data type, got  ex:i8'
Turning on debugging showed the response type was indeed ex:i8, which isn't one of the types that Frontier::Client supports.
<?xml version="1.0" encoding="UTF-8"?>
<methodResponse xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions">
  <params>
    <param>
      <value>
        <ex:i8>161</ex:i8>
      </value>
    </param>
  </params>
</methodResponse>
Searching through the code shows Frontier::Client is a wrapper around Frontier::RPC2 and the error message happens at the following section:
     elsif ($scalars $tag )  
       $expat-> 'rpc_text'  = "";
       push @  $expat-> 'rpc_state'   , 'cdata';
     else  
       Frontier::RPC2::die($expat, "wanted a data type, got \ $tag'\n");
    
So we can see that it's looking up the tag into a hash called %scalars to see if the type is a scalar type, otherwise throws the error we saw. Looking at the top, we can see this hash:
%scalars = (
    'base64' => 1,
    'boolean' => 1,
    'dateTime.iso8601' => 1,
    'double' => 1,
    'int' => 1,
    'i4' => 1,
    'string' => 1,
);
So, if we could add ex:i8 to this scalar, we could fix the problem. We could fix the module, but that would require every user of the script to patch their copy of the module. The alternative is to inject something into that hash across module boundaries, which we can do by just refering to the hash by it's complete name including the package name. We can use:
$Frontier::RPC2::scalars 'ex:i8'  = 1;
Now when we run the script, everything works. It's not nice and it's dependent on Frontier::RPC2 not changing. but it allows us to get on with our script.
Read Comments (1)

Theodore Ts'o: Organic vs. Non-organic Open Source

Brian Aker dropped by and replied to my previous essay by making the following comment:
I believe you are hitting the nail on the organic vs nonorganic open source. I do not believe we have a model for going from one to the other. Linux and Apache both have very different models for contribution but I don t believe either are really optimized at this point. Optimization to me would lead to a system of less priests and more inclusion.
I made an initial reply as comment, and then decided it was so long that I should promote it to a top-level post. I assume that when Brian talks about “organic open source” what he means is what I was calling an “open source development community”. Some googling turned up the following definition from Mozilla Firefox’s organic software page: “Our most well-known product, Firefox, is created by an international movement of thousands, only a small percentage of whom are actual employees.” This puts it in contrast with “non-organic” software, where all or nearly all of the developers are employed by one company. (And anyone who proves talented at adding features to that source base soon gets a job offer by that one company. :-) By that definition we can certainly see projects like Wine, Mysql, Ghostscript (at one time), and others as fitting into that model, and being quite successful. There’s nothing really wrong with the non-organic software model, although many of them have struggled to make enough money when competing with pure proprietary softare competitors, with MySQL perhaps being the exception which proves the rule. In most of these cases, though, the project started more as an organic open source, and then transitioned into the non-organic model when there was a desire to monetize the project and/or when the open source programmers decided that it would be nice if they could turn their avocation into a vocation, and let their hobby put food on the family table. Solaris, of course, is doing something else quite different, though. They are trying to make the transition from a proprietary customer/supplier relationship to trying to develop an Open Source community and what John s candidate statement pointed out is that they weren t really interested in creating an organic open source developer community at all, but they wanted the fruits of an open source community with plenty of application developers, end-users, etc., all participating in that community. We don t have a lot of precedent for projects who try to go in this direction, but I suspect they are skipping a step when they try to go to the end step without bothering to try to make themselves open to outside developers. And by continuing to act like a corporation, they end up shooting themselves in the foot. For example, the OpenSolaris license still prohibits people from publishing benchmarks or comparisons with other operating systems. Very common in closed-source operating systems and databases, but it discourages people from even trying to make things better, both within and outside of the Open Solaris core team. Instead, they respond to posts like David Miller s with Have you ever kissed a girl? . (Thanks, Simon, for that quote; I had seen it before, but not for a while, and it pretty well sums up the sheer arrogance of the Open Solaris development team.) So while Linux may not be completely optimized in terms of less priests and more inclusion, at least over 1200 developers contributed to 2.6.25 during its development cycle. Compared to that, Open Solaris is positively dominated by high priests and with a you may not touch the holy-of-holies attitude; heck, they won t even allow you to compare them to other religions without branding you a heretic and suing you for licensing violations!

Theodore Ts'o: Organic vs. Non-organic Open Source

Brian Aker dropped by and replied to my previous essay by making the following comment:
I believe you are hitting the nail on the organic vs nonorganic open source. I do not believe we have a model for going from one to the other. Linux and Apache both have very different models for contribution but I don t believe either are really optimized at this point. Optimization to me would lead to a system of less priests and more inclusion.
I made an initial reply as comment, and then decided it was so long that I should promote it to a top-level post. I assume that when Brian talks about “organic open source” what he means is what I was calling an “open source development community”. Some googling turned up the following definition from Mozilla Firefox’s organic software page: “Our most well-known product, Firefox, is created by an international movement of thousands, only a small percentage of whom are actual employees.” This puts it in contrast with “non-organic” software, where all or nearly all of the developers are employed by one company. (And anyone who proves talented at adding features to that source base soon gets a job offer by that one company. :-) By that definition we can certainly see projects like Wine, Mysql, Ghostscript (at one time), and others as fitting into that model, and being quite successful. There’s nothing really wrong with the non-organic software model, although many of them have struggled to make enough money when competing with pure proprietary softare competitors, with MySQL perhaps being the exception which proves the rule. In most of these cases, though, the project started more as an organic open source, and then transitioned into the non-organic model when there was a desire to monetize the project and/or when the open source programmers decided that it would be nice if they could turn their avocation into a vocation, and let their hobby put food on the family table. Solaris, of course, is doing something else quite different, though. They are trying to make the transition from a proprietary customer/supplier relationship to trying to develop an Open Source community and what John s candidate statement pointed out is that they weren t really interested in creating an organic open source developer community at all, but they wanted the fruits of an open source community with plenty of application developers, end-users, etc., all participating in that community. We don t have a lot of precedent for projects who try to go in this direction, but I suspect they are skipping a step when they try to go to the end step without bothering to try to make themselves open to outside developers. And by continuing to act like a corporation, they end up shooting themselves in the foot. For example, the OpenSolaris license still prohibits people from publishing benchmarks or comparisons with other operating systems. Very common in closed-source operating systems and databases, but it discourages people from even trying to make things better, both within and outside of the Open Solaris core team. Instead, they respond to posts like David Miller s with Have you ever kissed a girl? . (Thanks, Simon, for that quote; I had seen it before, but not for a while, and it pretty well sums up the sheer arrogance of the Open Solaris development team.) So while Linux may not be completely optimized in terms of less priests and more inclusion, at least over 1200 developers contributed to 2.6.25 during its development cycle. Compared to that, Open Solaris is positively dominated by high priests and with a you may not touch the holy-of-holies attitude; heck, they won t even allow you to compare them to other religions without branding you a heretic and suing you for licensing violations!

23 April 2008

Joey Hess: dh implementation

I've implemented last night's design, in debhelper 7.0.0. Currently fully supported are packages that use the autotools, or perl's MakeMaker, or python setup.py, or a simple Makefile. The other classes of packages will be added as bugs are filed about them. I've uploaded a few of my own packages converted to use debhelper 7 with minimised rules files, so there are a range of examples in incoming: Early adopters, have at it!

Russell Coker: Offensive Blog Posts

There has been ongoing debate in the Debian community for a number of years about what standards of behavior should be expected. Matthew Garrett sets a new low by making a joke about Jesus being molested as a child [1]. While I believe that debate and discussion about religion is a good thing, such comments about someone who is widely regarded as a God (part of the Holy Trinity) seems to provide no value and just needlessly offends people. I used to be a Christian, and while I have great disagreements with most Christians about issues of religion I still believe that Jesus (as described in the bible) was a good person and deserves some respect. I don’t believe that blasphemy should be illegal, but some minimum standards should be observed when discussing religion. Next there is the issue of child molesting, most people agree that there’s nothing amusing in that - so I hope that nothing more needs to be said about violating babies. Finally there is the issue of rape in general often being treated as a joke in the computer industry (I am not sure how prevalent this is in the wider community). One example that is from a Wired article: “We were raped by Microsoft. Bill Gates did it personally. Introducing Gates to the president of a small company is like introducing Mike Tyson to a virgin” [2]. I admit that finding examples of this on the web is not easy, part of it is due to such slang use being more common in spoken communication than in written communication, another is the vast number of slang terms that are used. A Google search for “male rape” [3] turns up some informative articles. One FAQ suggests that 3% of men will be raped as an adult [4] - I expect that some guys will decide that it’s not so funny when they realise that it could happen to them. For people who’s knowledge of the English language is not as good as that of Matthew and I, here is the dictionary definition of the word “violated” [5].

Next.

Previous.