Search Results: "ag"

9 April 2008

Steve McIntyre: Going to Debconf 8!

I'm going to DebConf8, edition 2008 of the annual Debian 
     developers meeting Looking forwards to seeing folks there, or on the flight over from Madrid... Now all I need to do is finish finding sponsors to pay for the conference itself!

Anthony Towns: Select and Python Generators

One of the loveliest things about Unix is the select() function (or its replacement, poll()), and the way it lets a single thread handle a host of concurrent tasks efficiently by just using file descriptors as work queues. Unfortunately, it can be a nuisance to use – you end up having to structure your program as a state machine around the select() invocation, rather than the actual procedure you want to have happen. You can avoid that by not using select() and instead just having a separate thread/process for every task you want to do – but that creates a bunch of tedious overhead for the OS (and admin) to worry about. But magically making state machines is what Python’s generators are all about; so for my little pet project that involves forking a bunch of subprocesses to do the interesting computational work my python program wants done, I thought I’d see if I could use that to make my code more obvious. What I want to achieve is to have a bunch of subprocesses accepting some setup data, then a bunch of two byte ids, terminated by two bytes of 0xFF, and for each of the two byte inputs to output a line of text giving the calculation result. For the time being at least, I want the IO to be asynchronous: so I’ll give it as many inputs as I can, rather than waiting for the result before sending the next input. So basically, I want to write something like:

def send_inputs(f, s, n):
	f.write(s) # write setup data
	for i in xrange(n):
		f.write(struct.pack("!H", i))
	f.write(struct.pack("!H", 0xFFFF))
def read_output(f):
	for line in f:
		if is_interesting(line):
			print line
Except of course, that doesn’t work directly because writing some data or reading a line can block, and when it does, I want it to be doing something else (reading instead of writing or vice-versa, or paying attention to another process). Generators are the way to do that in Python, with the “yield” keyword passing control flow and some information back somewhere else, so adopting the theory that: (a) I’ll only resume from a “yield” when it’s okay to write some more data, (b) if I “yield None” there’s probably no point coming back to me unless you’ve got some more data for me to read, and (c) I’ll provide a single parameter which is an iterator that will give me input when it’s available and None when it’s not, I can code the above as:

def send_inputs(_):
	# s, n declared in enclosing scope
	yield s
	for i in xrange(n):
		yield struct.pack("!H", i))
	yield struct.pack("!H", 0xFFFF)
def read_output(f):
	for line in f:
		if line is None: yield None; continue
		if is_interesting(line):
			print line
There’s a few complications there. For one, I could be yielding more data than can actually be written, so I might want to buffer there to avoid blocking. (I haven’t bothered; just as I haven’t worried about “print” possibly blocking) Likewise, I might only receive part of a line, or I might receive more than one line at once, and afaics a buffer there is unavoidable. If I were doing fixed size reads (instead of line at a time), that might be different. So far, the above seems pretty pleasant to me – those functions describe what I want to have happen in a nice procedural manner (almost as if they had a thread all to themselves) with the only extra bit the “None, None, continue” line, which I’m willing to accept in order not to use threads. Making that actually function does need a little grunging around, but happily we can hide that away in a module – so my API looks like:

p = subprocess.Popen(["./helper"], stdin=PIPE, stdout=PIPE, close_fds=True)
comm = communicate.Communication()
comm.add(send_inputs, p.stdin, None)
comm.add(read_output, None, p.stdout, communicate.ByLine())
comm.communicate()
The comm.add() function takes a generator function, an output fd (ie, the subprocess’s stdin), an input fd (the subprocess’s output), and an (optional) iterator. The generator gets created when communication starts, with the iterator passed as the argument. The iterator needs to have an “add” function (which gets given the bytes received), a “waiting” function, which returns True or False depending on whether it can provide any more input for the generator, and a “finish” function that gets called once EOF is hit on the input. (Actually, it doesn’t strictly need to be an iterator, though it’s convenient for the generator if it is) The generator functions once “executed” return an object with a next() method that’ll run the function you defined until the next “yield” (in which case next() will return the value yielded), or a “return” is hit (in which case the StopIteration exception is raised). So what we then want to do to have this all work then, is this: (a) do a select() on all the files we’ve been given; (b) for the ones we can read from, read them and add() to the corresponding iterators; (c) for the generators that don’t have an output file, or whose output file we can write to, invoke next() until either: they raise StopIteration, they yield a value for us to output, or they yield None and their iterator reports that it’s waiting. Add in some code to ensure that reads from the file descriptors don’t block, and you get:

def communicate(self):
    readable, writable = [], []
    for g,o,i,iter in self.coroutines:
        if i is not None:
            fcntl.fcntl(i, fcntl.F_SETFL, 
                        fcntl.fcntl(i, fcntl.F_GETFL)   os.O_NONBLOCK)
            readable.append(i)
        if o is not None:
            writable.append(o)
    
    while readable != [] or writable != []:
        read, write, exc = select.select(readable, writable, [])
        for g,o,i,iter in self.coroutines:
            if i in read:
                x = i.read()
                if x == "": # eof
                    iter.finish()
                    readable.remove(i)
                else:
                    iter.add(x)
            if o is None or o in write:
                x = None
                try:
                    while x is None and not iter.waiting():
                        x = g.next()
                    if x is not None:
                        o.write(x)
                except StopIteration:
                    if o is not None:
                        writable.remove(o)
    return
You can break it by: (a) yielding more than you can write without blocking (it’ll block rather than buffer, and you might get a deadlock), (b) yielding a value from a generator that doesn’t have a file associated with it (None.write(x) won’t work), (c) having generators that don’t actually yield, and (d) probably some other ways. And it would’ve been nice if I could have somehow moved the “yield None” into the iterator so that it was implicit in the “for line in f”, rather than explicit. But even so, I quite like it.

David Welton: Google App Engine as Ford?

A passing thought: Would it be fair to compare Google's App Engine to Ford's assembly line, in a roundabout sort of way? Ford's mass produced cars put cars within reach of many more people, and over time, put paid to the many custom car producers who didn't adapt. Google's App Engine is a way for people to easily scale their web applications in a way that wasn't possible until now. No fiddling with EC2 images, or hosting, or a bunch of other stuff that, for the time being will still drive people to want to control their own infrastructure, but the prospect of very cheap hosting and very easy scaling is potentially something that will change things significantly.

Adeodato Sim : giggle, a gitk in GTK+

By pure chance I read somebody mentioning giggle in #git. It’s a visualizer for git branches written in C using GTK+. I find the output a bit nicer than that of gitk, at least on some repos. As Mike Hommey points out, though, it’s quite slow on medium and big repositories. P.S.: It’s packaged for Debian.

Tore S. Bekkedal: No to OOXML demo

Today in Oslo, Wednesday April 9th, 12:00 AM, there will be a demo against the adoption of OOXML as an ISO standard. I’ve not got the time at the moment to elaborate on this, but adopting OOXML as an ISO standard is a really, really bad idea, mainly because it will weaken ISOs standing as a serious standards organization. I’ll whine at length about it in another post. http://blogs.freecode.no/isene/2008/04/07/demonstration-outside-iso-conference Be there, or B !

Adrian von Bidder: Mercurial

After my moderately successful attempts at learning git, I have now tackled Mercurial and found it easy to understand and use. I can't quite see where git is better, so I guess I'll stick with hg now. Output so far is a script I call hgdebuild, which I'll probably package when it's a bit less rough around the edges (yes, I know about hg-buildpackage, but I didn't really like it, and it appears to be orphaned at the moment anyway). Also, with Mercurial, I very easily could address the main problem I've always had with distributed revision control systems: With Subversion, it's easy to work on different computers, and backing up my home directory has become less critical (an important aspect on my Laptop). Both of these are not true when suddenly .hg in my working dir is the backing store.
$ cat ~/.hgrc
[hooks]
commit.push = [ -f .hg/dopush ] \
    && ping -w 1 -c 2 -q -i .2 mygitserver.example.com >/dev/null 2>/dev/null \
    && hg push \
      true
(Probably this problem has been solved before. Comments welcome.) Comments: From the person with madduck in his email address: Bzr checkouts, or a similar Git hook.

Kartik Mistry: Can you see anything like this in BLR?



Can you see anything like this in BLR?
Originally uploaded by aanjhan
No, I will not see :)

Adrian von Bidder: Vendor lock-in for children

Children should be trained at an early age that the only real OS comes from Redmond. Stupid gadgets reinventing the wheel and then using Windows-only drivers for their non-standard interfaces.

Michal Čihař: Similarity in project names

While looking at poll about Czech open source for first time, I was surprised that two of my projects got in - phpMyAdmin and Wammu. After looking again at that page I noticed there is a typo in phpMyAdmins name, it was written as phpMinAdmin. Then I looked even more carefully and even URL it is pointing to is something different. In this time I realized that it is something completely different with just very similar name. phpMinAdmin is some minimalistic clone of phpMyAdmin, but why they could not choose a bit more different name? PS: Anyway you can still vote for Wammu in Czech open source :-).

8 April 2008

Russell Coker: Olympics and Politics

The latest news related to the Beijing Olympic games is that Kevin Rudd (our Prime Minister) has said “It is absolutely clear that there are human rights abuses in Tibet. That’s clear-cut; we need to be upfront and absolutely straight about what’s going on,” - stating the obvious really. If that was all that happened then it wouldn’t be particularly interesting. The paper version of “The Age” quotes George Bush as seconding Kevin Rudd’s statement. I believe that this is quite significant. Bush is well known for being stupid, the fact that he is now following the example of someone else who is doing good things is a very positive thing for the world. When John Howard was our Prime Minister we had an idiot (Howard) following an even bigger idiot (Bush) and the result was not good. Kevan Gosper (an Australian IOC board member) said “They just take their hate out on whatever the issues are at the time, and that hate against the host country is being taken out on our torch“. People who are totally corrupt sometimes seem confused when other people are motivated by moral principles, maybe we have a culture clash between the corrupt IOC board and the attitudes of most people in the rest of the world (I can’t think of any other way to map Kevan’s statement to reality). Mr Rudd has confirmed that Chinese “security guards” (soldiers) will not be permitted to operate in Australia to protect the Olympic torch from protesters, but the “security guards” have been operating in the UK. I read about this on the web site of The Age, but I won’t link to them because they have a lot of broken links with the following explanation - I am not going to link to sites that are so transient in nature (linking to The Age would lower the quality of my blog). Incidentally does anyone know of a news service in Australia that has reliable pages which stay online? We could not find the page you requested. This is often because older content has been removed from our site. In most cases you can still find the item via our archive service, News Store, where you can buy articles for a small fee. It also has the following text which indicates that the most visible problem is probably transient, but the fact that they deliberately break links is unacceptable to me: If you reached this page from a link on our site, please contact the webmaster (choose Technical faults) and tell us the address of the faulty page and the address shown for this page. The MSN article about the meeting between Bush and Rudd also had an interesting quote from Nancy Pelosi (speaker of the house in the US congress) [1]. She said “As I said in India last week where I met with His Holiness the Dalai Lama, if freedom-loving people throughout the world do not speak out against China’s oppression in Tibet, we have lost our moral authority to speak out on behalf of human rights anywhere in the world” and that the IOC made a mistake in awarding the games to China. Pelosi is not known for being left-wing (the US Democratic party is centrist/right by the standards of most democracies) so it’s interesting to see her take a stand on this issue. Erich Schubert has written about this, he points out that the Olympics are not about sports [2]. Well of course the Olympics are about money, drugs, and sports-science! I’m not sure that I agree with Schubert and Pelosi, the Chinese Olympics has focussed a lot of international attention on what China is doing - this has to be good for human rights. The Olympic games are a white elephant, running them costs a huge amount of money and there is no evidence that they actually make money for the host country once the opportunity costs are taken into account. Maybe we should give the Olympics to Zimbabwe or the Sudan next?

Erich Schubert: Boycott Olympia!

Olympia HandcuffsIn my opinion, the protests in London and Paris were great.China wants to use Olympia for promoting itself. All this is just a huge image campaign. This is why we should remind people on how cruel that regime is and how little it cares for human rights.Human rights are more important than the olympic torch.Turning a chinese PR campaign into a PR disaster is good.This is not about sports. Sports is what you do at home, not what is on TV. When I go dancing with my friends, that is sports. When I sit in front of the TV, it is not. I will not be watching the Olympics. When IOC representatives talk about it all being "only about sports" they lie - it's all about money.This is not out of disrespect for the athletes, but I cannot tolerate China using the Olympics as image campaign for being an equal member of the international community. All the new stadiums and so on - they're trying really hard to give the impression of a new china - when they actually have not changed at all.We need to signal China that if they want to be treated equally, they need to adhere to the same standards as all others - respecting human rights is the bare minimum, free elections and democracy are the next important step then.P.S. Please also do not buy Olympia swag. It's reported to be manufactured at less than half of the minimum wage even in China! It's pure exploitation!Note that I'm talking about a consumers' boycott, not an athletes' boycott. I do think that the athletes go there for the competitions.P.P.S. I don't think the Olympic games are fair anyway - I believe that many athletes are using doping (just the detection methods are insufficient). Otherwise we would not see new world records each year. And I believe the IOC is corrupt and maybe actually criminal; all they actually care about are their revenue streams, not the sports either.P.P.P.S. This blog is representing my own opinion only. Therefore it does not have or need a comment function. Use your own blog if your opinion differs.

Gunnar Wolf: Some Rails plugins: Real FK

I have mostly shifted my main development language to Ruby, as most of the work I do tends to be pretty well covered with Rails. That's not new, however - What is new is that I feel I have got to the point where I actually have something to share. And it feels very good.
So my next couple of posts (three, actually) will be covering some plugins I've written. They are, of course, just sketched - but they work reliably enough to be deemed as useful by their author ;-)
Today I will start by blogging about RealFK. Of course, I will be copying from its documentation/description - I wrote it as well, so I'm not really plagiarizing my own site. I'm not suing self. You can of course also browse the RealFK SVN tree. References Rails is built upon a "dumb database" concept. Rails assumes the database will not force restrictions upon the user - i.e. that the model validators are all there is. But I disagree ;-) I was raised with PostgreSQL, not with MySQL, and I do believe the database can and should impose sane values - It should impose trust on the data..
So, if you want to create a relation field, properly acting as a foreign key and refering to another table, instead of doing it via the ActiveRecord::ConnectionAdapters::TableDefinition#column (or even ActiveRecord::ConnectionAdapters::TableDefinition#references) methods, declare them with add_refrence:
  def self.up
    create_table :proposals do  t 
      (...)
    end
    create_table :proposal_types do  t 
      (...)
    end
    create_table :proposal_statuses do  t 
      (...)
    end
    add_reference :proposals, :proposal_types
    add_reference :proposals, :proposal_statuses, :null => false, :default => 1
  end
The corresponding fields (proposal_type_id and proposal_status_id, respectively) will be created in proposals, and the RDBMS will impose a Foreign Key constraint. The references can be removed via remove_reference, i.e., for inclusion in down migrations:
  def self.down
    remove_reference :proposals, :proposal_types
    remove_reference :proposals, :proposal_statuses
    drop_table :proposal_statuses
    drop_table :proposal_types
    drop_table :proposals
  end
Of course, in this case the remove_reference call is not really needed - But if you are dropping a table that is referred to from a table that will remain existing, you will have to remove the foreign key constraints (that is, the referring columns). Join (HABTM) tables When you define a simple has_and_belongs_to_many (HABTM) relation in your model, you are expected to create an extra table representing this relation. This is a very simple table (i.e. it has only a row for each of the related table‘s IDs), and it carries foreign key constraints.
Please note that join tables are not supposed to carry any extra columns - If you need to add information to join tables, think twice and better use a has_many :elements, :through => :othermodel declaration, and create the othermodel table manually. To create a HABTM table representing that each person can attend many conferences and each conference can be attended by many people:
  def self.up
    create_table :people do  t 
      (...)
    end
    create_table :conferences do  t 
      (...)
    end
    create_habtm :people, :conferences
  end
  def self.down
    drop_habtm :people, :conferences
    drop_table :people
    drop_table :conferences
  end
The last call will create a conferences_people table (according to Rails' conventions, table names are sorted alphabetically to get the join table‘s name).
Note that in this case, in the down migration, the drop_habtm call must appear before the drop_table calls, as it carries foreign key constraints.

Lars Wirzenius: Remember to uncomment

A week ago or so I fiddled with my mirroring script. To make the fiddling go faster, I commented out the actual calls to debmirror, so I could concentrate on testing the other stuff I was adding. I forgot to uncomment. My mirror hasn't updated since. Oops. Moral of the story: when fiddling something, make sure you notice if forgot to re-enable stuff you disable temporarily. In this case, I should've used "echo" instead of ":" to comment things out. Had I done that, cron would've told me nothing was beinge executed.

Michael Prokop: Re: I am only mad on IRC

Martin, sorry for calling you Madduck on the debian.ch-mailinglist. Luckily I didn’t use that name during our meeting in Zurich last week. *pfuh* ;) To take part in the pdo-game with my own name (being Michael Prokop ;-)): as many of you might know I’m known as Mika in the community. The name was invented for me around 1999 (IIRC) by a greek flat collegue who liked to call me “Mika” - refering to Mika H kkinen (for whatever reason). From that on several people started to call me that way. I’m signing mails “to the community” with that nick since many years as well as using it in usenet and on freenode (IRC); on OFTC (another IRC network) I’ve to use mikap instead because mika was registered by another user already. Outside “the community” I’m usually known as Michael, good friends also use Michi and the family of my girlfriend uses Michl when the sister of my girlfriend (named Michaela and known as Michi as well) is present. Oh, and during grammar school I was AKA Proke.</secret> 8-) So calling me Mika is perfectly fine because it’s a kind of uniqueness for me. There might be just too many Michaels around and at least in Austria my nickname is everything but common. Quoting the marketing department: Great trademark! ;) Oh and my speech parser knows to handle Michael, Michi and Michl as well - though it strongly depends on the situation, place, environment,…. Oh but using Proke is like talking to /dev/null - you’ve never heard of that name, ok?

Adrian von Bidder: update-alternatives

Debian's “alternatives” system (/etc/alternatives) system doesn't really like having alternatives set to manual and then the corresponding packages being removed. Time to get rid of all those broken symlinks...
find -L * -type l    while read i; do update-alternatives --auto "$i"; done

Kartik Mistry: Tips: Travelling in BMTC


* Wake up early, catch bus in early morning. Avoid rush-hour traffic. * If you can’t wake up early, you will have crowded bus, go inside with hard try and look for person who has next bus stop to get seat. You need to study hard to become master in this - if person is moving his/her bag, keeping mobile/pen/paper in pocket, go nearby and wait he/she stand up and grab the place! * Make sure that you are not standing near door, crazy drivers shuts it suddenly and chance of injury is high. * Do not seat on seats reserved for ladies :P (either marked red or written on it). * Still, if you can’t find place, play ‘frozen-bubble’ or finish pending calls and wait for your stop! And, yes. Don’t assume that ’same bus number’ will reach you at ’same place’. The bus number on BMTC is volatile. And, please don’t ask ‘How to travel at evening’, I have left it upon your luck!

Erich Schubert: Licensing software

If you are writing free software, choose your license appropriately. This can make a big difference with respect to adoption of your software.I'll just show you an excerpt from a Debian changelog:
* DFSG version of Mono 1.9
  + Deleted the mcs/class/System.Web.Extensions/ directory as
    mcs/class/System.Web.Extensions/System.Web.Script.Serialization/JSON/*.cs
    is licensed under Creative Commons Attribution 2.5 which is not
    DFSG-free.
Debian policy doesn't allow us to include Creative Commons Attribution 2.5 in the main Debian archive, since that license doesn't meet the Debian Free Software Guidelines, which are part of our social contract. Therefore we must and will remove such code.debian-legal summary for "main" Creative Commons licensesCreative Commons is aware of these issues:
Creative Commons recommends and uses free and open source licenses for software.
And then they suggest to use GPL. While there is a link in the right side menu to the Creative Commons page on licensing Software, they IMHO don't make this obvious enough. Their license chooser should include a central link "if you intend to license software, read this".Creative Commons even includes "wrappers" for common licenses: CC GPL, CC LGPL, CC BSDSo if you like Creative Commons because of this pretty "commons deed" human-understandable version of the license they offer, just use these wrappers. The "legal code" links will actually take you to the GNU or Opensource.org license pages.I'm not aware of a good reason to use an opensource license other than GPL, LGPL or 3-clause BSD, depending on how you want to allow your opensource code to be used in combination with non-free software. If you are contributing to a bigger project, choosing the same license as the main project is although a very good idea.

Kartik Mistry: Late report: fossmeet@NITC, Calicut


* With uncertainty principle playing in full form with me, I decided to go to Fossmeet@NITC. But, came to know that there is no bus tickets available! What to do? I called up dear tuxmaniac aka Aanjhan and he suggested me to go by taxi. wow. Pretty expensive it will be, I thought. What ever, we decided to go with burning (hip) pockets and the journey was terrible. I did such long travel (7 hours) in taxi after long time. Anyway, we reached at NITC around 6 AM. The Keralian atmosphere was excellent and I felt similar humidity like Mumbai and that made me a bit sad. Ah. The event was already kicked off the day before and I attended Praveen’s talk on openid while figuring out that wifi is not working for us (proxy and they didn’t care to let us know password etc). So, we headed towards lab and enjoyed some Internet while seating in ‘Foss for school children’ workshop. Lunch was ok, but I enjoyed talking with different people and it was nice talk with Ani Peter about current localization work from Redhat. These guys simply rocks! Got my conference tshirt before ‘Medium’ size is out-of-stock. We were eagerly waiting for our dear Pradeepto, and voila, he came and started his talk! I was attending his talk first time (always missed due to some or other reasons..). Excellent and interactive. No slides. /me need to learn something from this fellow! We were 100% tired at evening. So, first we changed to better ‘room’. Thanks to NITC co-coordinators who did quick action for this (Why we requested for changing room? Answer: The previous room had only one power-supply plug! and we want to do hacking at night!). We then headed towards ‘Malabar Palace’. Excellent South Indian food and beer. The next day — I was suppose to give talk on ‘Contributing to Debian’ but by some magic, the talk was disappeared from schedule. Sad. Anyway, we (me, tuxmaniac and pradeepto) gave live talk on ‘How to contribute to free software’. We used SVN, Bugzilla, Wiki, SF.Net and other tools (IRC!) to show how students can use these in college projects and make it much better and how it can lead to Free Software development. I did svn commit for KDE Gujarati (Idea stoled from Rusty’s talk at Foss.in/2007). We then decided to leave the place. The return trip was amazing. Pookat lake and Mysore Palace. Amazing places. We had nice dinner and Kamat Hotel and I bought one wood penguin too (Total number of Penguin toys=4, Now). I learned that Calicut is also known as Kozikode :)

Kartik Mistry: 45 days


* I was hell tired from useless promotional calls and SMS from Vodaphone from last 30 days (I got my new number on 6th March), So, I tried their Do Not Disturb form and submitted it. Also, did SMS to toll-free number. And, I got this email after one confirmation email: “We would like to inform you that we have updated the number under DND option and the promotional SMS/calls will be stopped on your Vodafone number 9xxxxxxxxx within 45 days.” OMG, 45 Days!!!

Stefano Zacchiroli: land of lisp

land of lisp land of lisp is a really nice strip about the evils of side effects. It's made of 72 panels, ... it takes about 10 minutes to read, but is worth. As a bonus for Italian readers: the domain name hosting the strip rimes in between "lisp" and "desperate"; is there a hidden message?

Next.

Previous.