Search Results: "ron"

18 October 2009

Stefano Zacchiroli: RC bugs of the week - week 7

RCBW - week #7 RC bugs squashed this week by yours truly: After having whet my appetite last week with the Python 2.6 transition, this week I mostly focused on that. 5/7 of the bugs above have been NMUs to help getting closer to an upload of Python 2.6 to unstable. Next week I'll probably be on RCBW-vacation, as theoretically I'm supposed to be traveling. I say theoretically because, 2 days from the departure, I'm stuck in bed with a 38 Celsius fever. Great. Anyhow, see you on these pages in 2 weeks.

16 October 2009

Uwe Hermann: coreboot on the cover of the Linux Journal

coreboot on Linux Journal Nice coreboot news the Free Software x86 firmware ("BIOS") is featured on the cover of issue 186 of the Linux Journal. Anton Borisov's article Coreboot at Your Service! explains the basic ideas behind coreboot, how to build an image for your board, which payloads are available and how they are used, e.g. GRUB2, SeaBIOS if you need legacy BIOS callbacks (e.g. for booting Windows), Etherboot/GPXE, or more fun stuff such as space invaders or tint (a tetris clone) in your flash ROM chip... If you read the article and think the build process is a bit complicated and ugly, do not despair! We're currently in the process of converting the whole coreboot code base to use kconfig (the widely-known configuration tool used by the Linux kernel, busybox, and other projects), so in the very near future the whole process for building a coreboot image will work like this: 
  $ make menuconfig
  $ make
coreboot menuconfig Flashing the image can then be done using an EEPROM programmer and/or via the user-space utility flashrom (available for Linux, Mac OS X, FreeBSD, etc.)... It's nice to see that coreboot is getting more and more coverage in "mainstream" media and is growing both in number of deployments and in number of supported chipsets and boards. We are desperately in need of more developers though, there are just way too many chipsets, boards, and datasheets out there; we're happy about every patch and every new tester or developer who likes to mess with code that runs in the very first few (micro)seconds after power-on. If you think kernel hacking and related low-level development is nice, you might also be interested in writing code where there's no RAM yet (as coreboot has to initialize it), there's no serial port for debugging (coreboot has to initialize it), no PCI devices have been set up, most of your auxiliary hardware is not yet up (ethernet NIC, parallel port, audio, IDE, SATA, USB, you name it). It's a fun environment to work in and you'll learn a lot about PC hardware, even if you (so far) thought you knew everything there is to know. Feel free to join us on the mailing list or on IRC in #coreboot on Freenode.

Gerfried Fuchs: Signing Jokes in Contracts

It is great to see that the collaboration between Debian and Ubuntu is improving; and I don't say that just because it probably can't be much better within the Debian/Ubuntu Games Team, we have people from both distributions working inside the team and most of the packages don't carry Ubuntu specific patches (anymore) because of that. Actually, seeing that things do go pretty well in that area made me consider signing up to become a MOTU. To do that one has to sign the Ubuntu Code of Conduct (CoC) which is a fairly good document, actually. I wish there would be something similar within Debian that is considered binding, it would be able to reduce quite some tough and rough times, actually. It is about and explains to be considerate, be respectful, be collaborative, what to do when you disagree, when you are unsure and wants you to step down considerately. If these principles would be carried out amongst all the free software communities (and I really mean carried out and not just be there and grow old) I expect it would be much more welcoming for new people. And it's not too hard to do your part for it (... says the person who just recently had to excuse for her behavior). Anyway, there is this one part with the CoC that itches me. It's not that one has to sign it with their GnuPG key, but related to it. Making it a requirement to sign it gives the document a much more official character, actually gives it the feeling and impression of a contract and I expect it is meant to carry that feeling. Though, there is this one part in it that I consider off for such a document:
Nobody knows everything, and nobody is expected to be perfect in the Ubuntu community (except of course the SABDFL).
Given that the acronym SABDFL refers to Mark Shuttleworth it means that one has to expect him to be impeccable which I am sorry but cannot sign. I don't expect that from anyone else but myself, even Mark is only human and can make mistakes. Even though it's obvious that this is a tongue-in-cheek kind of joke which might be meant to make it clear that the Ubuntu community isn't just sterile having this in a document that is expected to get signed by contributors is just an extremely bad idea. Sorry, Ubuntu, as long as this joke is part of the CoC I can't sign it with clean conscience, no matter how much I would sign the rest of it a thousands' time. It really makes me wonder how many others actually did read the "contract" carefully that they are signing but on the other hand simply didn't care. There is too many site signup stuff out there that noone reads neither.

Russ Allbery: On vacation

It is that time of year again, in which I disappear for a while to get my head together, engage in intensive relaxation, and catch up on my reading. Don't expect to see much of me on-line for the next three weeks. I was going to post a book review tonight, since otherwise it's unlikely that I'll post any in the month of October, but I just ran out of steam. The steam that I had went to talking to friends, where I won't get much chance for a while. Since I lost all of last week sick with gastroenteritis, I think I'm going to give myself a free pass for this month on the review front. Regular reviews will resume once I get back from vacation. (Contrary to how it may appear, I have no intention of letting them slowly trickle off to nothing, nor have I stopped reading. I have three books finished so far this month and waiting for me to review. I just badly need a vacation and then better time allocation.) What with illness and with all of the work stress lately, I didn't do as good of a job this year of shutting everything down and getting everything into a good state before vacation. Apologies for the loose ends left sitting around, although I think everything will survive. I shall return refreshed, well-read, and with many more book reviews and photographs in hand. See you all in about three weeks!

Gunnar Wolf: Getting closer to the fifth EDUSOL Encounter

This is the fifth year we hold an EDUSOL, and we are closing in on it. EDUSOL is an online encounter whose topics are Education and Free Software Actually, this year we are widening our scope, and we will include Free Culture as well as a base area. Now, besides those three general areas, each year we have had a base topic around which we invite the speakers to talk about (although it is a lax requirement). This year, the base topic is social networks No, not in the Twitter/Facebook/blah sense, but as a wider phenomenon, studying interaction between people, the forming of communities. And for our particular areas, the forming of knowledge-based communities. Anyway I agreed with the organizers to provide the English translation of the participation invitation. I will skip the call for papers, as we are basically at the proposal deadline (October 17), but if you have anything you want to propose, please tell us so! Leaving that aside... Please excuse the quality of my translation, it is late and I'm tired. We will work on it :) Videoconference reception EDUSOL spans several participation categories. The closest category to a traditional, face to face conference. Each year, we invite a group of speakers to talk about a topic related to our main discoursive line. Among the speakers that have confirmed so far, we have: We invite you to be active participants in the videoconference cycle. You can invite your social or user group to be part of the Encounter. There are three ways to do it:
  1. Using the videoconference facilities in your University network
  2. Connecting from a personal computer with broadband access, by using Ekiga or any other H.323 client
  3. Following the .ogg stream and participating back via IRC
In any case (specially in the first two, which require more coordination), please contact us. For further information, visit http://edusol.info/es/e2009/convocatorias/videoconfrencias November 6: Free Education and Free Culture day We invite social and user groups to host talks regarding Free Education and Free Culture. This is not a call to promote Free Software, as there are many other spaces devoted to it. We should start with the idea that freedom resides in us, not in the software. Some of the topics our community recommends are: Further details at http://edusol.info/es/e2009/convocatorias/dialternativo We need some help in this area to generate contents with slides, making it easier for proposed scripts for the talks. If you want to collaborate, please write to our academic support list, http://lists.edusol.info/listinfo.cgi/soporte-academico-edusol.info Want to collaborate? Further questions or comments? We are holding on-line meetings for this group of topics on Thursday 22:00 GMT-5, in the #edusol channel in OFTC (irc.debian.org); you can enter the IRC channel using the Web client at http://irc.bine.org.mx or http://edusol.info/irc About EDUSOL EDUSOL started as a proposal seeking to provide a space so that people interested in education could discuss and analize the good and bad points about Free Culture and Free Software, with no geographic restrictions. Year after year, freedom-loving educators of all Latin America and Spain gather for our annual party. EDUSOL's core language is Spanish, although participation in English is allowed and encouraged (although understanding Spanish will be a strong aid). We invite you to participate and contribute in this collective effort using and commenting on our blogs, or adding your personal blog to our planet: http://edusol.info/es/node/add/blog
http://edusol.info/es/node/add/feedapi-node You can follow us by identi.ca or RSS: http://identi.ca/group/edusol
http://edusol.info/rss.xml

15 October 2009

Anand Kumria: It is easy to not hear when your fingers are in your ears.

I read this post by Steve Waddington of Exetel where he says that customers never seem to follow-up with him when he asks them about their poor customer experiences. I just had to laugh. I had actually just filled a complaint with the TIO about Exetel. I have emailed Steve Waddington a number of times, generally outlining problems, but I am yet to ever receive a response. Generally if you want to hear feedback from your customers, you have to take your fingers out of your ears. Here is my (most recent) Exetel story. To set the scene, I am synchronising a large backup of data. Think around 100Gb. In reality a small amount. Towards the end of September, we receive an email indicating that we had used a large amount of service. Furthermore that we had 72 hours to pay an interim fee of AUD$100. The email was sent in the early hours of a Friday morning. Which effectively turned it into a single business day. Heaven help you if you had decided to take a long weekend. At any rate, the fee was paid by that afternoon and a name and receipt were obtained. Everything should have be hunky-dorey. Come Monday morning - the Internet service had mysteriously failed. After some remote, expensive AU to UK phones calls and diagnostics I determined that the problem was that Exetel's system had disconnected the service anyway. I emailed them to see what the problem was, and contact was made from Australia. Around the middle of the day, service was restored. A complaint was filed with them (reference: #1691636), that day, which was summarily responded too. I left things for a while since I was busy. But on the 13th Oct night, I decided to file a complaint with the TIO (reference: 09/251856). Here is the full text:
Describe your complaint On 25 Sep, roughly 07:30am, an email was received from Exetel that they required an interim payment as our usage had exceeded AUD$100. The email indicated that if payment was not received within 72 hours - the service may be suspended. At roughly 3:15pm, on 25 Sep - i.e. the same day. A payment of AUD$100 was made and we were issued receipt number 346650646 by an employee who indicated that there name was Rukshani. On 28 Sep at roughly 8am, the Internet ceased to function. After some investigation we determined that Exetel's automated systems believed that the payment had not occurred. #1: I sent an email at 11am on 28 Sep to their billing department, along with their senior management, with payment and service details asking why this occurred. #2: A complaint was sent by the payee at 11:30am on 28 Sep indicating that we were unhappy with the service -- having been customers of long standing (over 4 years) -- being disconnected for over 4 hours despite paying is annoying. Particular when this was not through fault of our own. Nor a technical fault. How did the service provider respond to your complaint #1: On 30 Sep at 12:45 an email was received from James Lowe, who said "Your service is un block now [sic]". No explanation was provided for the outage and non-application of the payment made. #2: 2 responses were received to this email. #2a: At ~11:50am on 28 Sep we received an email from Larry Kaeto who said that the service was unavailable for 2hrs 31mins, not the 4hrs as we had claimed. Additionally he indicated that we had a residental service which is not to be relied upon for business/mission critical requirements and suggested we purchase a more expensive service which has "business grade type support". #2b: At ~09:40am on 29 Sep we received a further email from Larry Kaeto who claimed that no payment had been received by Exetel on 25 Sep. How would you like the service provider resolve your complaint Exetel appears to pride itself on its automated systems. (c.f. http://johnl.blogs.exetel.com.au/index.php?/archives/2783-Automating-Key-Systems-Is-A-Never-Ending-Task.html) However, in this case, the automated system issued a demand for money. The money was paid, but it suspended access to the service anyway. Service should not have been disrupted for 1 minute nor 150 minutes nor 240 minutes. I believe the charge of AUD$100 and the monthly charge should be credited back to us for the hassle.
I've put in the headings as they are on the TIO complaint form, and made the URL a link otherwise it is exactly as I submitted it which is also why the text is not very lengthy, as the TIO restrict how much you can submit. This morning, at 09:10 I received an email from Exetel acknowledging receipt of the TIO complaint and asking Please detail in a point by point manner your concerns. This email was from Larry Kaeto. It also contained a copy of the most recent LNS logs. I replied with a full copy of the complaint as I submitted it to the TIO (reference: #1770435) and also indicated I would speak with the TIO to confirm what they had sent to Exetel (Exetel reference: #1771491). By 12:56, Larry Kaeto was able to responsd, quoted here and edited to occlude privacy information:
Exetel acknowledge payment 25/9/09 was manually processed 25/9/09 but the record to remove the timer to suspend in 72 hrs was not removed. I did not see the manual payment of the 25/9/09. Sorry for my ''incompetence'' for saying payment was not made. The duration of suspension was 2 hours and 28 minutes (or thereabouts) as the facts show below (our session logs and your usage time logged in and out) and nto 4 hours as is claimed. As gesture of goodwill, Exetel will provide a refund of $100 Exetel has made a commercial decision to end the relationship with you. See e-mail sent separately titled ''Ticket #1771491: XXXXXXXX5X - Notice of Termination - XXXXX XXXXXX ''
So, as you can see from John Linton's post, if there is a problem with taking payment your service is summarily disconnected. And, as you can see here, even if they take payment, but their automated system fails, if you complain about it, your service is summarily disconnected. It is much easier to listen to your customers when you take your fingers out of your ears.

Enrico Zini: Building a web-based API with Turbogears2

Building a web-based API with Turbogears2 I am using TurboGears2 to export a python API over the web. Every API method is wrapper by a controller method that validates the parameters and returns the results encoded in JSON. The basic idea is this: 
@expose("json")
def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
    # Call API
    res = self.engine.list_colours(filter, productID, maxResults)
    # Return result
    return res
To validate the parameters we can use forms, it's their job after all: 
class ListColoursForm(TableForm):
    fields = [
            # One field per parameter
            twf.TextField("filter", help_text="Please enter the string to use as a filter"),
            twf.TextField("productID", help_text="Please enter the product ID"),
            twf.TextField("maxResults", validator=twfv.Int(min=0), default=200, size=5, help_text="Please enter the maximum number of results"),
    ]
list_colours_form=ListColoursForm()
#...
    @expose("json")
    @validate(list_colours_form, error_handler=list_colours_validation_error)
    def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
        # Parameter validation is done by the form
        # Call API
        res = self.engine.list_colours(filter, productID, maxResults)
        # Return result
        return res
All straightforward so far. However, this means that we need two exposed methods for every API call: one for the API call and one error handler. For every API call, we have to type the name several times, which is error prone and risks to get things mixed up. We can however have a single error handler for all methonds: 
def get_method():
    '''
    The method name is the first url component after the controller name that
    does not start with 'test'
    '''
    found_controller = False
    for name in pylons.c.url.split("/"):
        if not found_controller and name == "controllername":
            found_controller = True
            continue
        if name.startswith("test"):
            continue
        if found_controller:
            return name
    return None
class ValidatorDispatcher:
    '''
    Validate using the right form according to the value of the "method" field
    '''
    def validate(self, args, state):
        method = args.get("method", None)
    # Extract the method from the URL if it is missing
        if method is None:
            method = get_method()
            args["method"] = method
        return forms[method].validate(args, state)
validator_dispatcher = ValidatorDispatcher()
This validator will try to find the method name, either as a form field or by parsing the URL. It will then use the method name to find the form to use for validation, and pass control to the validate method of that form. We then need to add an extra "method" field to our forms, and arrange the forms inside a dictionary: 
class ListColoursForm(TableForm):
    fields = [
            # One hidden field to have a place for the method name
            twf.HiddenField("method")
            # One field per parameter
            twf.TextField("filter", help_text="Please enter the string to use as a filter"),
    #...
forms["list_colours"] = ListColoursForm()
And now our methods become much nicer to write: 
    @expose("json")
    def api_validation_error(self, **kw):
        pylons.response.status = "400 Error"
        return dict(form_errors=pylons.c.form_errors)
    @expose("json")
    @validate(validator_dispatcher, error_handler=api_validation_error)
    def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
        # Parameter validation is done by the form
        # Call API
        res = self.engine.list_colours(filter, productID, maxResults)
        # Return result
        return res
api_validation_error is interesting: it returns a proper HTTP error status, and a JSON body with the details of the error, taken straight from the form validators. It took me a while to find out that the form errors are in pylons.c.form_errors (and for reference, the form values are in pylons.c.form_values). pylons.response is a WebOb Response that we can play with. So now our client side is able to call the API methods, and get a proper error if it calls them wrong. But now that we have the forms ready, it doesn't take much to display them in web pages as well: 
def _describe(self, method):
    "Return a dict describing an API method"
    ldesc = getattr(self.engine, method).__doc__.strip()
    sdesc = ldesc.split("\n")[0]
    return dict(name=method, sdesc = sdesc, ldesc = ldesc)
@expose("myappserver.templates.myappapi")
def index(self):
    '''
    Show an index of exported API methods
    '''
    methods = dict()
    for m in forms.keys():
        methods[m] = self._describe(m)
    return dict(methods=methods)
@expose('myappserver.templates.testform')
def testform(self, method, **kw):
    '''
    Show a form with the parameters of an API method
    '''
    kw["method"] = method
    return dict(method=method, action="/myapp/test/"+method, value=kw, info=self._describe(method), form=forms[method])
@expose(content_type="text/plain")
@validate(validator_dispatcher, error_handler=testform)
def test(self, method, **kw):
    '''
    Run an API method and show its prettyprinted result
    '''
    res = getattr(self, str(method))(**kw)
    return pprint.pformat(res)
In a few lines, we have all we need: an index of the API methods (including their documentation taken from the docstrings!), and for each method a form to invoke it and a page to see the results. Make the forms children of AjaxForm, and you can even see the results together with the form.

Gustavo Noronha Silva: Moving back to Belo!

So, next Saturday Luciana and I are moving back from Rio de Janeiro to Belo Horizonte after about 6 months here. It was an interesting ride. We didn t really have a good time finding a good place to live at - Rio is very expensive and it looks like demand is so high anywhere close to Luciana s work place that the only reasonable solution was to live very far away, and even there we didn t really have much success, although for other reasons I may talk about later. Belo is how the foreign people who live there usually refer to Belo Horizonte. I think that is probably because it s a bit too hard for them to say Horizonte =D (if it helps you, you could use the Mineirish Belzonte ). I have borrowed this svg file kindly licensed in the FDL by Raphael Lorenzeto de Abreu, and generated a simple png image that you can see in this post with Belo Horizonte, and some other cities we often talk about at Collabora =): A straight line between Recife and Belo Horizonte has about 1640km. There are 340km between BH and Rio, and 973km between BH and Florian polis. A bit about Belo Horizonte, for those friends of mine to whom I always have a hard time explaining where it s located, and what kind of city it is: Belo Hozizonte is a nice, beautiful city, with a quite big metropolitan area. About 6 million people live in greater Belo Horizonte. It is considered one of the three most influential cities in Brazil, along with S o Paulo and Rio de Janeiro. It s surrounded by mountains, which are considered one of the defining geographical features of the Minas Gerais state, of which it is the capital. Here s a picture that is available under the public domain in Wikipedia: Speaking of Minas Gerais, it is, according to Wikipedia, the fourth in size among the 26 States of Brazil, being just a bit bigger than mainland-France. Minas Gerais is one of the main earth transports hubs of the country (having the largest number of federal roads), and is famous for its tasty food, the beauty of its people, its cheeses, the calm, introspective nature of its people, its interesting way of speaking portuguese, and its alcoholic beverages. Now, going away from Rio to Belo Horizonte has a single big disadvantage associated with it: we don t get to have the Sea a few minutes away, anymore. Our solution to that is, of course, going to one of the numerous bars (the legend goes around that BH has the largest absolute number of bars of all cities in the world), and drinking instead of swimming! Quem n o tem mar, vai pro bar!

Romain Beauxis: Ocaml-cry

Coming along with the next release of liquidsoap, we have just released a new module for ocaml, namely ocaml-cry. Ocaml-cry is a native implementation of the protocols used for sending source data to icecast and shoutcast servers. The main difference with the famous libshout implementation is about flexibility. This module allows to set any source header, which in particular allows to use any custom format which is compatible with the servers. However, contrary to libshout, ocaml-cry does not handle the synchronisation of the audio data sent to the servers. Hence, the task of controling the rate of the data sent is up to the programmer. You can find more informations on this module there: http://www.rastageeks.org/cry.html Debian packages are available there [1]: http://packages.debian.org/libcry-ocaml-dev
[1] Many thanks to the ftp-masters for dealing with the NEW package so fast !

14 October 2009

Patrick Schoenfeld: PHP and the great "===" operator

Lets suppose you've got an array with numeric indices:
[0] => 'bla'
[1] => 'blub'
Now you want to do something if the element 'bla' is found in that array.
Well, you know that PHP has a function array_search, which returns the key(s) of the found values. Lets say, you write something like this:
if(array_search($array, 'bla')) do_something

Would you expect that do_something would actually do something?
If yes: You are wrong.
If no: Great, you've understand some parts of PHP - insanitygoodness..

Actually, if 'bla' wouldn't have the index 0 it would work, because 1, 2, 3, 4, etc. is TRUE. But unfortunately PHP has some sort of implicit casting, which makes 0 behave like a FALSE, depending on the context. So following this, the if() works for all elements except 0.

You might be tempted to write
if(array_search... != FALSE)
But this wouldn't help you, because 0 would still evaluate to FALSE, leading to if(FALSE != FALSE) which is (hopefully obviously) never true.

A PHP beginner (or even an intermediate, if he never stumbled across this case) might ask:
Whats the solution for this dilemma?

Luckily the PHP documentation is great. It tells you about this. And additionally PHP has got this great operator (===), which causes people to ask "WTF?" when they hear about it for the first time. Additionaly to comparing the values to each other, they also check the type of the variable. This leads to the wanted result because 0 is an integer, while FALSE is Boolean. So the solution for our problem looks like this:
if (array_search($array, 'bar') !== FALSE)
Isn't this great?

Robert McQueen: Telepathy Q&A from the Boston GNOME Summit

The first Telepathy session session on Saturday evening at the Boston GNOME Summit was very much of a Q&A where myself and Will answered various technical and roadmap issues from a handful of developers and downstream distributors. It showed me that there s a fair amount of roadmap information we should do better at communicating outside of the Telepathy project, so in the hope its useful to others, read on MC5 Ted Gould was wondering why Mission Control 4 had an API for getting/setting presence for all of your accounts whereas MC5 does not. MC5 has a per-account management of desired/current presence which is more powerful, but loses the convenience of setting presence in one place. Strictly speaking, doing things like deciding which presence to fall back on (a common example being if you have asked for invisible but the connection doesn t support it) is a UI-level policy which MC should not take care of, but in practice there aren t many different policies which make sense, and the key thing is that MC should tell the presence UI when the desired presence isn t available so it could make a per-account choice if that was preferable. As a related side point, Telepathy should implement more of the invisibility mechanisms in XMPP so it s more reliably available, and then we could more meaningfully tell users which presence values were available before connecting, allowing signing on as invisible. Since MC5 gained support for gnome-keyring, its not possible to initialise Empathy s account manager object without MC5 prompting the user for their keyring password un-necessarily (especially if the client is Ubuntu s session presence applet and the user isn t using Empathy in the current session but has some accounts configured). Currently the accounts D-Bus API requires that all properties including the password are presented to the client to edit. A short-term fix might be to tweak the spec so that accounts don t have to provide their password property unless it s explicitly queried, but this might break the ABI of tp-glib. Ultimately, passwords being stored and passed around in this way should go away when we write an authentication interface which will pass authentication challenges up to the Telepathy client to deal with, enabling a unified interface for OAuth/Google/etc web token, Kerberos or SIP intermediate proxy authentication, and answering password requests from the keyring lazily or user on demand. Stability & Security Jonathan Blandford was concerned about the churn level of Telepathy, from the perspective of distributions with long-term support commitments, and how well compatibility will be maintained. Generally the D-Bus API and the tp-glib library APIs are maintained to the GNOME standards of making only additive changes and leaving all existing methods/signals working even if they are deprecated and superseded by newer interfaces. A lot of new interfaces have been added over the past year or so, many of which replace existing functionality with a more flexible or more efficient interface. However, over the next 4-6 months we hope to finalise the new world interfaces (such as multi-person media calls, roster, authentication, certificate verification, more accurate offline protocol information, chat room property/role management), and make a D-Bus API break to remove the duplicated cruft. Telepathy-glib would undergo an ABI revision in this case to also remove those symbols, possibly synchronised with a move from dbus-glib to GVariant/etc, but in many cases clients which only use modern interfaces and client helper classes should not need much more than a rebuild. Relatedly there was a query about the security of Telepathy, and how much it had been through the mill on security issues compared to Pidgin. In the case of closed IM protocols (except MSN where we have our own implementation) then we re-use the code from libpurple, so the same risks apply, although the architecture of Telepathy means its possible to subject the backend processes to more stringent lockdowns using SElinux or other security isolation such as UIDs, limiting the impact of compromises. Other network code in Telepathy is based on more widely-used libraries with a less chequered security history thus far. OTR The next topic was about support for OTR in Telepathy. Architecturally, it s hard for us to support the same kind of message-mangling plugins as Pidgin allows because there is no one point in Telepathy that messages go through. There are multiple backends depending on the protocol, multiple UIs can be involved in saving (eg a headless logger) or displaying messages (consider GNOME Shell integration which previews messages before passing conversations on to Empathy), and the only other centralised component (Mission Control 5) does not act as an intermediary for messages. Historically, we ve always claimed OTR to be less appealing than native protocol-level end-to-end encryption support, such as the proposals for Jingle + peer to peer XMPP + TLS which are favoured by the XMPP community, mostly because if people can switch to an unofficial 3rd party client to get encryption, they could switch to a decent protocol too, and because protocol-level support can encrypt other traffic like SRTP call set-up, presence, etc. However, there is an existing deployed OTR user base, including the likes of Adium users on the Mac, who might often end up using end to end encryption without being aware of it, who we would be doing a disservice by Telepathy not supporting OTR conversations with these people. This is a compelling argument which was also made to me by representatives from the EFF, and the only one to date which actually held some merit with me compared to just implementing XMPP E2E encryption. Later in the summit we went on to discuss how we might achieve it in Telepathy, and how our planned work towards XMPP encryption could also help. Tubes We also had a bit of discussion about Tubes, such as how the handlers are invoked. Since the introduction of MC5, clients can register interest in certain channel types (tubes or any other) by implementing the client interface and making filters for the channels they are interested in. MC5 will first invoke all matching observers for all channels (incoming and outgoing) until all of them have responded or timed out (eg to let a logger daemon hook up signal callbacks before channel handling proceeds), all matching approvers for incoming channels until one of them replies (eg to notify the user of the new channel before launching the full UI), and then sending it to the handler with the most specific filter (eg Tomboy could register for file transfers with the right MIME type and receive those in favour to Empathy whose filter has no type on it). Tubes can be shared with chat rooms, either as a stream tube where one member shares a socket for others to connect to (allowing re-sharing an existing service implementation), or a D-Bus tube where every member s application is one endpoint on a simulated D-Bus bus, and Telepathy provides a mapping between the D-Bus names and the members of the room. In terms of Tube applications, now we ve got working A/V calling in Empathy, as well as desktop sharing, and an R&D project on multi-user calls, our next priority is on performance and Tube-enabling some more apps such as collaborative editing (Gobby, AbiWord, GEdit, Tomboy ?). There was a question about whether Tube handlers can be installed on demand when one of your contacts initiates that application with you. It d be possible to simulate this by finding out (eg from the archive) which handlers are available, and dynamically registering a handler for all of those channel types, so that MC5 will advertise those capabilities, but also register as an approver. When an incoming channel actually arrives at the approval stage, prompt the user to install the required application and then tell MC5 to invoke it as the handler. Colin Walters asked about how Telepathy did NAT traversal. Currently, Telepathy makes use of libnice to do ICE (like STUN between every possible pair of addresses both parties have, works in over 90% of cases) for the UDP packets involved in calls signalled over XMPP, either the Google Talk variant which can benefit from Google s relay servers if one or other party has a Google account, so is more reliable, or the latest IETF draft which can theoretically use TURN relays but its not really hooked up in Telepathy and few people have access to them. XMPP file transfers and one-to-one tube connections use TCP which is great if you have IPv6, but otherwise impossible to NAT traverse reliably, so often ends up using strictly rate-limited SOCKS5 -ish XMPP proxies, or worse, in-band base64 in the XML stream. We hope to incorporate (and standardise in XMPP) a reliability layer which will allow us to use Jingle and ICE-UDP for file transfers and tubes too, allowing peer to peer connections and higher-bandwidth relays to enhance throughput significantly. Future Ted Gould had some good questions about the future of Telepathy
Should Empathy just disappear on the desktop as things like presence applets or GNOME Shell take over parts of its function? Maybe, yes. In some ways its goal is just to bring Telepathy to end users and the desktop so that its worth other things integrating into Telepathy, but Telepathy allows us to do a lot better than a conventional IM client. Maemo and Sugar on the OLPC use Telepathy but totally integrates it into the device experience rather than having any single distinct IM client, and although Moblin uses Empathy currently it has its own presence chooser and people panel, and may go on to replace other parts of the user experience too. GNOME Shell looks set to move in this direction too and use Telepathy to integrate communications with the desktop workflow. Should Telepathy take care of talking to social networking sites such as Facebook, Twitter, etc? There s no hard and fast rule Telepathy only makes sense for real-time communications, so it s good for exposing and integrating the Facebook chat, but pretty lame for dealing with wall posts, event invitations and the like. Similarly on the N900, Telepathy is used for the parts of the cellular stack that overlap with real-time communications like calling and SMS, but there is no sense pushing unrelated stuff like configuration messages through it. For Twitter, the main question is whether you actually want tweets to appear in the same UI, logging and notification framework as other messages. Probably not anything but the 1-to-1 tweets, meaning something like Moblin s Mojito probably makes more sense for that. Later in the summit I took a look at Google Latitude APIs, which seem like something which Telepathy can expose via its contact location interface, but probably not usefully until we have support for metacontacts in the desktop. Can/will Telepathy support IAX2? It can, although we d have to do a local demultiplexer for the RTP streams involved in separate calls. It s not been a priority of ours so far, but we can help people get started (or Collabora can chat to people who have a commercial need for it). Similarly nobody has looked at implementing iChat-compatible calling because our primary interest lies with open protocols, but if people were interested we could give pointers its probably just SIP and RTP after you dig through a layer of obfuscation or two.
If you want to know more about Telepathy feel free to comment with some follow-up questions, talk to us in #telepathy on Freenode, or post to the mailing list.

Robert McQueen: Boston GNOME Summit 2009

I spent this weekend in Boston for the annual GNOME summit. I really enjoyed it this year, although there were fewer attendees than previously it felt very focussed and productive. There s some cool stuff going on, and it s always great to catch up with all of the usual free software suspects in Boston. Some highlights from the weekend: I was really impressed by Jason Clinton and others summaries of the sessions, which I think are really valuable for the people who couldn t make it to the summit. He asked me to take some notes about the first Telepathy session on Saturday evening while he was taking notes about the Outreach session. Rather than lumber him with my deranged scratchings from Tomboy, I ll blog them separately.

Christian Perrier: Virtualization (take 2)

Thanks to everybody who went through the pain of my blog (which doesn't allow comments because I don't want to bother with blog spam) and still cooked up a private reply to me. It's very much appreciated. So, Patrick, Dan, Dirk and No l, you just won a free beer for your wise advice.....except the two marathoners among you guys who shouldn't drink beer (though it would help you slowing down enough to be able to run along with me). In short, all this can be summarized to "Christian, you're going the wrong way with Xen, use KVM". One of the advices was using VirtualBox OSE, which is certainly a wise advice as well...but not what I want to experiment (I already use VB for the few D-I work I'm still doing and that would not be a new experiment for me). I'll go your proposed way and will let you know. Still, I'd love being able to figure out Xen stuff as this is what we're more and more using at ONERA.

13 October 2009

Steve Kemp: There's no such thing as a wrong war

Once upon a time I wrote a blog compiler, a simple tool that would read in a bunch of text files and output a blog. This blog would contain little hierarchies for tags, historical archives, etc. It would also have a number of RSS feeds too. Every now and again somebody will compare it to ikiwiki and I'll ignore that comparison entirely, because the two tools do different things in completely different fashions. But I was interested to see Joey talk about performance tweaks recently as I have a blog which has about 900 pages, and which takes just over 2 minutes to build from start to finish. (Not this one!) I've been pondering performance for a while as I know my current approach is not suited to high speed. Currently the compiler reads in every entry and builds a giant data structure in memory which is walked in different fashions to generate and output pages. The speed issue comes about because storing the data structure entirely in memory is insane, and because sometimes a single entry will be read from disk multiple times. I've made some changes over the past few evenings such that a single blog entry will be read no more than once from disk (and perhaps zero times if Memcached is in use :) but that doesn't solve the problem of the memory usage. So last night I made a quick hack - using my introduction to SQLite as inspiration I wrote a minimal reimplementation of chronicle which does things differently: This is a significantly faster approach than the previous one - with a "make steve" job taking only 18 seconds, down from just over 2 minutes 5 seconds. ("make steve" uses rsync to pull in comments on entries, rebuilds the blog, then uses rsync to push the generated output into its live location.) ObFilm: If...

Russell Coker: New Servers a non-virtual Cloud

NewServers.com [1] provides an interesting service. They have a cloud computing system that is roughly comparable to Amazon EC2, but for which all servers are physical machines (blade servers with real disks). This means that you get the option of changing between servers and starting more servers at will, but they are all physical systems so you know that your system is not going to go slow because someone else is running a batch job. New Servers also has a bandwidth limit of 3GB per hour with $0.10 per GB if you transfer more than that. Most people should find that 3GB/hour is enough for a single server. This compares to EC2 where you pay $0.10 per GB to receive data and $0.17 to transmit it. If you actually need to transmit 2100GB per month then the data transfer fees from EC2 would be greater than the costs of renting a server from New Servers. When running Linux the EC2 hourly charges are (where 1ECU is provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor):


NAMECostDesc
Small$0.101.7G, 160G, 32bit, 1ECU, 1core
Large$0.207.5G, 850G, 64bit, 4ECU, 2core
Extra Large$0.4015G, 1690G, 64bit, 8ECU, 4core
High CPU Medium$0.201.7G, 350G, 32bit, 5ECU, 2core
High CPU Extra Large$0.807G, 1690G, 64bit, 20ECU, 5core
The New Servers charges are:


NAMECostDesc
Small$0.111G, 36G, 32bit, Xeon 2.8GHz
Medium$0.172G, 2*73G, 32bit, 2*Xeon 3.2GHz
Large$0.254G, 250G, 64bit, E5405 Quad Core 2Ghz
Jumbo$0.388G, 2*500G, 64bit, 2 x E5405 Quad Core 2Ghz
Fast$0.534G, 2*300G, 64bit, E5450 Quad Core 3Ghz
The New Servers prices seem quite competitive with the Amazon prices. One down-side to New Servers is that you have to manage your own RAID, the cheaper servers have only a single disk (bad luck if it fails). The better ones have two disks and you could setup your own RAID. Of course the upside of this is that if you want a fast server from New Servers and you don t need redundancy then you have the option of RAID-0 for better performance. Also I don t think that there is anything stopping you from running Xen on a New Servers system. So you could have a bunch of Xen images and a varying pool of Dom0s to run them on. If you were to choose the Jumbo option with 8G of RAM and share it among some friends with everyone getting a 512M or 1G DomU then the cost per user would be a little better than Slicehost or Linode while giving better management options. One problem I sometimes have with virtual servers for my clients is that the disk IO performance is poorer than I expect. When running the server that hosts my blog (which is shared with some friends) I know the performance requirements of all DomUs and can diagnose problems quickly. I can deal with a limit on the hardware capacity, I can deal with trading off my needs with the needs of my friends. But having a server just go slow, not knowing why, and having the hosting company say I can move you to a different physical server (which may be better or worse) doesn t make me happy. I first heard about New Servers from Tom Fifield s LUV talk about using EC2 as a cluster for high energy physics [2]. According to the detailed analysis Tom presented using EC2 systems on demand can compete well with the costs of buying Dell servers and managing them yourself, EC2 wins if you have to pay Japanese prices for electricity but if you get cheap electricity then Dell may win. Of course a major factor is the amount of time that the servers are used, a cluster that is used for short periods of time with long breaks in between will have a higher cost per used CPU hour and thus make EC2 a better option.

11 October 2009

Jeff Bailey: Karmic review

Angie has my old laptop that some might remember from a couple Ubuntu developer summits. Adam Conrad used to refer to it as the surfboard: a lovely 15" widescreen laptop. It's my test machine for Ubuntu updates and so far I'm loving Karmic. This machine has been continually upgraded since either Hoary or Breezy without a reinstall.

Jaunty was miserable. Bugs in the Intel drivers meant that DVD playing stopped working, sounds would occasionally lock up, and things were just painfully slow. Karmic to the rescue. I did the update at beta time like I usually do, and without any further effort:

* Flash just worked.
* DVD playing in totem just worked.
* Boot time was slightly quicker
* nethack-gnome continues to work (despite being removed from the archive a few releases ago)
* Suspend/resume continues to just work.
* Atheros wireless continues to just work with the free kernel drivers.

I then added the dev channel of Google Chrome, which imported all the Firefox bookmarks and started to remove some optional things that Angie doesn't use anymore and don't run well on the now 5 year-old laptop. I took out openoffice, a pile of -dev packages that were leftover from when I used to build glibc on this thing (This laptop had the Hurd running on it at one point... *g*), and then discovered the System->Admin->Package Cleanup (Or at least, it's called something like that, the laptop is in French) where I could start to remove unused packages. I did a few rounds of that, some apt-get autoremove. I took universe and multiverse out of the packages list and did some more removals, making sure to leave in things like deborphan and the gstreamer packages that are in universe and multiverse. Lastly, I did a round of deborphan to take out some pieces that were just hanging out.

As a final step I've updated to grub2. I don't have a separate /boot partition on this machine, so I haven't been brave enough to convert it to ext4 yet.

The boottime at this point still feels a little long for what this laptop does. While at Canonical, I played with some appliance configs and had from kernel load to running firefox in a dozen seconds or some. This laptop doesn't have anything I'd see as unusual for daemons - there's no mysqld, sshd, or anything unneeded like that. But to have everything Just Work(tm) at this stage is amazing.

Nice job. =)

update: Angie's asked me to figure out why the screen saver kicks in after 30 seconds instead of 10 minutes like it's set to. But still, if this is the worst that goes wrong with this release, then it's still a wild success already at Beta.

C.J. Adams-Collier: A quick update I m a grad student!

Hey all! I m sorry I haven t been very active with my packages recently. I all-of-a-sudden started grad school and have been swamped with studying. I also started a contract and have been busy trying to learn a new codebase while contributing something other than snark. I promise I ll get back to packaging IronRuby and IronPython on Mono for Debian as soon as things start settling down. Getting an A in the class is higher priority, though, sorry Don t worry. I haven t forgotten about you ;) Cheers, C.J. PS, I am implementing a Perl library to exercise my understanding of the class. You can follow along at the search.cpan.org page for Lingua::HPSG or by cloning the git repo: $ git clone git://karma.colliertech.org/colliertech/langparser
<script>//<script language="javascript" src="http://reddit.com/button.js?t=1"></script>

Sebastian Rittau: VirtualBox on i386 with amd64 Kernel

I have recently started to use an amd64 kernel on my i386 Debian unstable system. Unfortunately, VirtualBox OSE does not work with that setup. When I try to start a virtual machine, it fails with an oblique error message:
RTR3Init failed with rc=-1912 (rc=-1912) The VirtualBox kernel modules do not fit to this version of VirtualBox. The installation of VirtualBox was apparently not successful. Executing

'/etc/init.d/vboxdrv setup'

should fix that problem. Make sure that you don't mix the OSE version and the PUEL version of VirtualBox.
Debian bug #456391 explains the problem. In that report Michael Meskes alludes to running VirtualBox in an amd64 chroot jail, so I tried this myself. It works flawlessly, once I got it setup. Here is what I did (as root): 
robinson:~# mkdir /srv/amd64
robinson:~# cdebootstrap --arch amd64 sid /srv/amd64 http://ftp.debian.org/debian/
[...]
robinson:~# chroot /srv/amd64
robinson:/# apt-get update
[...]
robinson:/# apt-get upgrade
[...]
robinson:/# apt-get install virtualbox-ose # add more packages here if needed
[...]
robinson:/# adduser --uid 1000 --no-create-home --disabled-password --disabled-login srittau
[...]
robinson:/#
These commands install the base system and create a user account. Now I created a script called /usr/local/bin/amd64.sh: 
#!/bin/sh
CHROOT=/srv/amd64
if test ! -e $CHROOT/dev/.udev; then
    mount -t none /dev $CHROOT/dev/ -o bind
fi
if test  ls $CHROOT/proc   wc -l  = "0"; then
    mount -t proc none $CHROOT/proc
fi
if test  ls $CHROOT/sys   wc -l  = "0"; then
    mount -t sysfs none $CHROOT/sys
fi
if test  ls $CHROOT/home   wc -l  = "0"; then
    mount --bind /home $CHROOT/home
fi
chroot $CHROOT sh -c "su - srittau"
Running sudo amd64.sh will now enter the chroot environment as user srittau where I can start virtualbox normally.

Martin Michlmayr: Upgrading SheevaPlug's U-Boot to version 3.4.19

Marvell made a new version of U-Boot for the SheevaPlug available last month. This version, 3.4.19, integrates SD card support and improves support for FAT32 USB partitions (although USB support in general is still a bit flaky). This page describes how to install this new version of U-Boot on your SheevaPlug. First of all, download the SheevaPlug U-Boot 3.4.19. Unpack the zip file and copy the file u-boot-3.4.19/u-boot-rd88f6281Sheevaplug_400db_nand.bin to your TFTP server. I suggest you call the file u-boot.bin. Then start your SheevaPlug and type the following: 
setenv serverip 192.168.1.2 # IP of your TFTP server
setenv ipaddr 192.168.1.200
bubt u-boot.bin
The upgrade process will ask whether you want to change the environment: 
**Warning**
If U-Boot Endiannes is going to change (LE->BE or BE->LE),
Then Env parameters should be overridden..
Override Env parameters? (y/n) n
Make sure to answer "n" here. We also need to make a small change to the u-boot configuration because this version of u-boot no longer accepts the console parameter we have been using. Thanks to Tom Rinehart for pointing this out. Instead of console, we'll therefore add a new bootargs_console and update bootcmd to make use of it: 
setenv bootargs_console console=ttyS0,115200
printenv bootcmd
You'll see that bootcmd contains $(console). You have to set a new bootcmd and change the console parameter to $(bootargs_console). In my case, I'd type: 
setenv bootcmd 'setenv bootargs $(bootargs_console); run bootcmd_usb; bootm 0x400000 0x0800000; reset'
Make sure you don't forget the quotes! Finally, save the environment and restart your machine: 
saveenv
reset

David Pashley: mod_proxy or mod_jk

This entry was originally posted in slightly different form to Server Fault There are several ways to run Tomcat applications. You can either run tomcat direcly on port 80, or you can put a webserver in front of tomcat and proxy connections to it. I would highly recommend using Apache as a front end. The main reason for this suggestion is that Apache is more flexible than tomcat. Apache has many modules that would require you to code support yourself in Tomcat. For example, while Tomcat can do gzip compression, it's a single switch; enabled or disabled. Sadly you can not compress CSS or javascript for Internet Explorer 6. This is easy to support in Apache, but impossible to do in Tomcat. Things like caching are also easier to do in Apache. Having decided to use Apache to front Tomcat, you need to decide how to connect them. There are several choices: mod_proxy ( more accurately, mod_proxy_http in Apache 2.2, but I'll refer to this as mod_proxy), mod_jk and mod_jk2. Mod_jk2 is not under active development and should not be used. This leaves us with mod_proxy or mod_jk. Both methods forward requests from apache to tomcat. mod_proxy uses the HTTP that we all know an love. mod_jk uses a binary protocol AJP. The main advantages of mod_jk are: A slight disadvantage is that AJP is based on fixed sized chunks, and can break with long headers, particularly request URLs with long list of parameters, but you should rarely be in a position of having 8K of URL parameters. (It would suggest you were doing it wrong. :) ) It used to be the case that mod_jk provided basic load balancing between two tomcats, which mod_proxy couldn't do, but with the new mod_proxy_balancer in Apache 2.2, this is no longer a reason to choose between them. The position is slightly complicated by the existence of mod_proxy_ajp. Between them, mod_jk is the more mature of the two, but mod_proxy_ajp works in the same framework as the other mod_proxy modules. I have not yet used mod_proxy_ajp, but would consider doing so in the future, as mod_proxy_ajp is part of Apche and mod_jk involves additional configuration outside of Apache. Given a choice, I would prefer a AJP based connector, mostly due to my second stated advantage, more than the performance aspect. Of course, if your application vendor doesn't support anything other than mod_proxy_http, that does tie your hands somewhat. You could use an alternative webserver like lighttpd, which does have an AJP module. Sadly, my prefered lightweight HTTP server, nginx, does not support AJP and is unlike ever to do so, due to the design of its proxying system.
Read Comments (3)

Next.

Previous.