Search Results: "ag"

16 October 2009

Russell Coker: TPG Lies

Shortly before 9AM this morning I discovered that the IP address for my mail server was not being routed, according to my logs the problem started shortly after midnight. It s on a TPG ADSL connection, there is one IP address for the PPPOE link and 6 addresses in a /29 routed to it one of the addresses in the /29 is for my mail server. It wasn t until 3PM that I was able to visit the server to sort the problem out. It turned out that the main IP address was working but the /29 wasn t being routed to it. So TPG had somehow dropped the route from my routing table. I pinged all the addresses from a 3G broadband connection on my EeePC while running tcpdump on the server, and no packets for the /29 came through but the IP address for the PPP link worked fine. I was even able to ssh in to the server once I knew the IP address of the ppp0 device for future use I need to keep ALL IP addresses of all network gear on my EeePC not just the ones used for providing services. So I phoned the helpdesk, and naturally they asked me inane questions. My patience extended to telling them the broadcast address etc that was being used on the Ethernet device (actually a bridge for Xen but I wasn t going to confuse them). The system been power-cycled before I got there in the hope that it might fix the problem so I could honestly answer the question have you rebooted it (usually I lie rebooting systems to fix network problems is a Windows thing). But my patience started to run out when they asked me to check my DNS settings, I explained very clearly that my problem was that IP packets couldn t get through and that I wasn t using DNS and demanded that they fix it. I didn t get anyone technical to look at the problem until I firmly demanded that the help-desk operator test the routing by pinging my systems. The help-desk people don t have Internet access so that actually testing the connection required escalating the issue. It seems that the algorithm used for help-desk people is to just repeatedly tell people to check various things on their own system, and that continues until the customer s patience runs out. Either the customer goes away or makes requests firmly enough to get something done about it. So their technician did some tests and proclaimed that there was no problem. While said tests were being done things started working, so obviously their procedure is to fix problems and then blame it on the customer. It is not plausible to believe that a problem in their network which had persisted for more than 15 hours would accidentally disappear during the 5 minute window that the technician was investigating the problem. In the discussion that followed the help-desk operator tried to trick me into admitting that it was my fault. They claimed that because I had used multiple IP addresses I must have reconfigured my system and had therefore fixed a problem on my end, my response was I HAVE A HEAP OF MACHINES HERE RUNNING ALL THE TIME, I USE WHICHEVER ONE I FEEL LIKE, I CHANGED NOTHING . I didn t mention that the machines in question are DomUs on the same Xen server, someone who doesn t understand how ping works or what routing is wouldn t have been able to cope with that. I stated clearly several times that I don t like being lied to. Either the help-desk operator was lying to me or their technician was lying to them. In either case they were not going to trick me I know more about how the Internet works than they do. TPG was unable to give me any assurance that such problems won t happen again. The only thing I can be sure of is that when they lie they will stick to their story regardless of whether it works.

Gerfried Fuchs: Signing Jokes in Contracts

It is great to see that the collaboration between Debian and Ubuntu is improving; and I don't say that just because it probably can't be much better within the Debian/Ubuntu Games Team, we have people from both distributions working inside the team and most of the packages don't carry Ubuntu specific patches (anymore) because of that. Actually, seeing that things do go pretty well in that area made me consider signing up to become a MOTU. To do that one has to sign the Ubuntu Code of Conduct (CoC) which is a fairly good document, actually. I wish there would be something similar within Debian that is considered binding, it would be able to reduce quite some tough and rough times, actually. It is about and explains to be considerate, be respectful, be collaborative, what to do when you disagree, when you are unsure and wants you to step down considerately. If these principles would be carried out amongst all the free software communities (and I really mean carried out and not just be there and grow old) I expect it would be much more welcoming for new people. And it's not too hard to do your part for it (... says the person who just recently had to excuse for her behavior). Anyway, there is this one part with the CoC that itches me. It's not that one has to sign it with their GnuPG key, but related to it. Making it a requirement to sign it gives the document a much more official character, actually gives it the feeling and impression of a contract and I expect it is meant to carry that feeling. Though, there is this one part in it that I consider off for such a document:
Nobody knows everything, and nobody is expected to be perfect in the Ubuntu community (except of course the SABDFL).
Given that the acronym SABDFL refers to Mark Shuttleworth it means that one has to expect him to be impeccable which I am sorry but cannot sign. I don't expect that from anyone else but myself, even Mark is only human and can make mistakes. Even though it's obvious that this is a tongue-in-cheek kind of joke which might be meant to make it clear that the Ubuntu community isn't just sterile having this in a document that is expected to get signed by contributors is just an extremely bad idea. Sorry, Ubuntu, as long as this joke is part of the CoC I can't sign it with clean conscience, no matter how much I would sign the rest of it a thousands' time. It really makes me wonder how many others actually did read the "contract" carefully that they are signing but on the other hand simply didn't care. There is too many site signup stuff out there that noone reads neither.

Russ Allbery: On vacation

It is that time of year again, in which I disappear for a while to get my head together, engage in intensive relaxation, and catch up on my reading. Don't expect to see much of me on-line for the next three weeks. I was going to post a book review tonight, since otherwise it's unlikely that I'll post any in the month of October, but I just ran out of steam. The steam that I had went to talking to friends, where I won't get much chance for a while. Since I lost all of last week sick with gastroenteritis, I think I'm going to give myself a free pass for this month on the review front. Regular reviews will resume once I get back from vacation. (Contrary to how it may appear, I have no intention of letting them slowly trickle off to nothing, nor have I stopped reading. I have three books finished so far this month and waiting for me to review. I just badly need a vacation and then better time allocation.) What with illness and with all of the work stress lately, I didn't do as good of a job this year of shutting everything down and getting everything into a good state before vacation. Apologies for the loose ends left sitting around, although I think everything will survive. I shall return refreshed, well-read, and with many more book reviews and photographs in hand. See you all in about three weeks!

Gunnar Wolf: Getting closer to the fifth EDUSOL Encounter

This is the fifth year we hold an EDUSOL, and we are closing in on it. EDUSOL is an online encounter whose topics are Education and Free Software Actually, this year we are widening our scope, and we will include Free Culture as well as a base area. Now, besides those three general areas, each year we have had a base topic around which we invite the speakers to talk about (although it is a lax requirement). This year, the base topic is social networks No, not in the Twitter/Facebook/blah sense, but as a wider phenomenon, studying interaction between people, the forming of communities. And for our particular areas, the forming of knowledge-based communities. Anyway I agreed with the organizers to provide the English translation of the participation invitation. I will skip the call for papers, as we are basically at the proposal deadline (October 17), but if you have anything you want to propose, please tell us so! Leaving that aside... Please excuse the quality of my translation, it is late and I'm tired. We will work on it :) Videoconference reception EDUSOL spans several participation categories. The closest category to a traditional, face to face conference. Each year, we invite a group of speakers to talk about a topic related to our main discoursive line. Among the speakers that have confirmed so far, we have: We invite you to be active participants in the videoconference cycle. You can invite your social or user group to be part of the Encounter. There are three ways to do it:
  1. Using the videoconference facilities in your University network
  2. Connecting from a personal computer with broadband access, by using Ekiga or any other H.323 client
  3. Following the .ogg stream and participating back via IRC
In any case (specially in the first two, which require more coordination), please contact us. For further information, visit http://edusol.info/es/e2009/convocatorias/videoconfrencias November 6: Free Education and Free Culture day We invite social and user groups to host talks regarding Free Education and Free Culture. This is not a call to promote Free Software, as there are many other spaces devoted to it. We should start with the idea that freedom resides in us, not in the software. Some of the topics our community recommends are: Further details at http://edusol.info/es/e2009/convocatorias/dialternativo We need some help in this area to generate contents with slides, making it easier for proposed scripts for the talks. If you want to collaborate, please write to our academic support list, http://lists.edusol.info/listinfo.cgi/soporte-academico-edusol.info Want to collaborate? Further questions or comments? We are holding on-line meetings for this group of topics on Thursday 22:00 GMT-5, in the #edusol channel in OFTC (irc.debian.org); you can enter the IRC channel using the Web client at http://irc.bine.org.mx or http://edusol.info/irc About EDUSOL EDUSOL started as a proposal seeking to provide a space so that people interested in education could discuss and analize the good and bad points about Free Culture and Free Software, with no geographic restrictions. Year after year, freedom-loving educators of all Latin America and Spain gather for our annual party. EDUSOL's core language is Spanish, although participation in English is allowed and encouraged (although understanding Spanish will be a strong aid). We invite you to participate and contribute in this collective effort using and commenting on our blogs, or adding your personal blog to our planet: http://edusol.info/es/node/add/blog
http://edusol.info/es/node/add/feedapi-node You can follow us by identi.ca or RSS: http://identi.ca/group/edusol
http://edusol.info/rss.xml

15 October 2009

David Moreno: Debian Workshop 1 at New York City

So Debian-NYC, mainly the DebConf10 local team, organized a workshop for people interested on Debian packaging. The nice people of drop.io offered their handy location in Brooklyn. So this is the first attempt to bring new comers to Debian aiming DebConf10 next summer. Micah and dkg are the ones leading the workshop so far. Pictures!

Anand Kumria: It is easy to not hear when your fingers are in your ears.

I read this post by Steve Waddington of Exetel where he says that customers never seem to follow-up with him when he asks them about their poor customer experiences. I just had to laugh. I had actually just filled a complaint with the TIO about Exetel. I have emailed Steve Waddington a number of times, generally outlining problems, but I am yet to ever receive a response. Generally if you want to hear feedback from your customers, you have to take your fingers out of your ears. Here is my (most recent) Exetel story. To set the scene, I am synchronising a large backup of data. Think around 100Gb. In reality a small amount. Towards the end of September, we receive an email indicating that we had used a large amount of service. Furthermore that we had 72 hours to pay an interim fee of AUD$100. The email was sent in the early hours of a Friday morning. Which effectively turned it into a single business day. Heaven help you if you had decided to take a long weekend. At any rate, the fee was paid by that afternoon and a name and receipt were obtained. Everything should have be hunky-dorey. Come Monday morning - the Internet service had mysteriously failed. After some remote, expensive AU to UK phones calls and diagnostics I determined that the problem was that Exetel's system had disconnected the service anyway. I emailed them to see what the problem was, and contact was made from Australia. Around the middle of the day, service was restored. A complaint was filed with them (reference: #1691636), that day, which was summarily responded too. I left things for a while since I was busy. But on the 13th Oct night, I decided to file a complaint with the TIO (reference: 09/251856). Here is the full text:
Describe your complaint On 25 Sep, roughly 07:30am, an email was received from Exetel that they required an interim payment as our usage had exceeded AUD$100. The email indicated that if payment was not received within 72 hours - the service may be suspended. At roughly 3:15pm, on 25 Sep - i.e. the same day. A payment of AUD$100 was made and we were issued receipt number 346650646 by an employee who indicated that there name was Rukshani. On 28 Sep at roughly 8am, the Internet ceased to function. After some investigation we determined that Exetel's automated systems believed that the payment had not occurred. #1: I sent an email at 11am on 28 Sep to their billing department, along with their senior management, with payment and service details asking why this occurred. #2: A complaint was sent by the payee at 11:30am on 28 Sep indicating that we were unhappy with the service -- having been customers of long standing (over 4 years) -- being disconnected for over 4 hours despite paying is annoying. Particular when this was not through fault of our own. Nor a technical fault. How did the service provider respond to your complaint #1: On 30 Sep at 12:45 an email was received from James Lowe, who said "Your service is un block now [sic]". No explanation was provided for the outage and non-application of the payment made. #2: 2 responses were received to this email. #2a: At ~11:50am on 28 Sep we received an email from Larry Kaeto who said that the service was unavailable for 2hrs 31mins, not the 4hrs as we had claimed. Additionally he indicated that we had a residental service which is not to be relied upon for business/mission critical requirements and suggested we purchase a more expensive service which has "business grade type support". #2b: At ~09:40am on 29 Sep we received a further email from Larry Kaeto who claimed that no payment had been received by Exetel on 25 Sep. How would you like the service provider resolve your complaint Exetel appears to pride itself on its automated systems. (c.f. http://johnl.blogs.exetel.com.au/index.php?/archives/2783-Automating-Key-Systems-Is-A-Never-Ending-Task.html) However, in this case, the automated system issued a demand for money. The money was paid, but it suspended access to the service anyway. Service should not have been disrupted for 1 minute nor 150 minutes nor 240 minutes. I believe the charge of AUD$100 and the monthly charge should be credited back to us for the hassle.
I've put in the headings as they are on the TIO complaint form, and made the URL a link otherwise it is exactly as I submitted it which is also why the text is not very lengthy, as the TIO restrict how much you can submit. This morning, at 09:10 I received an email from Exetel acknowledging receipt of the TIO complaint and asking Please detail in a point by point manner your concerns. This email was from Larry Kaeto. It also contained a copy of the most recent LNS logs. I replied with a full copy of the complaint as I submitted it to the TIO (reference: #1770435) and also indicated I would speak with the TIO to confirm what they had sent to Exetel (Exetel reference: #1771491). By 12:56, Larry Kaeto was able to responsd, quoted here and edited to occlude privacy information:
Exetel acknowledge payment 25/9/09 was manually processed 25/9/09 but the record to remove the timer to suspend in 72 hrs was not removed. I did not see the manual payment of the 25/9/09. Sorry for my ''incompetence'' for saying payment was not made. The duration of suspension was 2 hours and 28 minutes (or thereabouts) as the facts show below (our session logs and your usage time logged in and out) and nto 4 hours as is claimed. As gesture of goodwill, Exetel will provide a refund of $100 Exetel has made a commercial decision to end the relationship with you. See e-mail sent separately titled ''Ticket #1771491: XXXXXXXX5X - Notice of Termination - XXXXX XXXXXX ''
So, as you can see from John Linton's post, if there is a problem with taking payment your service is summarily disconnected. And, as you can see here, even if they take payment, but their automated system fails, if you complain about it, your service is summarily disconnected. It is much easier to listen to your customers when you take your fingers out of your ears.

Enrico Zini: Building a web-based API with Turbogears2

Building a web-based API with Turbogears2 I am using TurboGears2 to export a python API over the web. Every API method is wrapper by a controller method that validates the parameters and returns the results encoded in JSON. The basic idea is this:
@expose("json")
def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
    # Call API
    res = self.engine.list_colours(filter, productID, maxResults)
    # Return result
    return res
To validate the parameters we can use forms, it's their job after all:
class ListColoursForm(TableForm):
    fields = [
            # One field per parameter
            twf.TextField("filter", help_text="Please enter the string to use as a filter"),
            twf.TextField("productID", help_text="Please enter the product ID"),
            twf.TextField("maxResults", validator=twfv.Int(min=0), default=200, size=5, help_text="Please enter the maximum number of results"),
    ]
list_colours_form=ListColoursForm()
#...
    @expose("json")
    @validate(list_colours_form, error_handler=list_colours_validation_error)
    def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
        # Parameter validation is done by the form
        # Call API
        res = self.engine.list_colours(filter, productID, maxResults)
        # Return result
        return res
All straightforward so far. However, this means that we need two exposed methods for every API call: one for the API call and one error handler. For every API call, we have to type the name several times, which is error prone and risks to get things mixed up. We can however have a single error handler for all methonds:
def get_method():
    '''
    The method name is the first url component after the controller name that
    does not start with 'test'
    '''
    found_controller = False
    for name in pylons.c.url.split("/"):
        if not found_controller and name == "controllername":
            found_controller = True
            continue
        if name.startswith("test"):
            continue
        if found_controller:
            return name
    return None
class ValidatorDispatcher:
    '''
    Validate using the right form according to the value of the "method" field
    '''
    def validate(self, args, state):
        method = args.get("method", None)
    # Extract the method from the URL if it is missing
        if method is None:
            method = get_method()
            args["method"] = method
        return forms[method].validate(args, state)
validator_dispatcher = ValidatorDispatcher()
This validator will try to find the method name, either as a form field or by parsing the URL. It will then use the method name to find the form to use for validation, and pass control to the validate method of that form. We then need to add an extra "method" field to our forms, and arrange the forms inside a dictionary:
class ListColoursForm(TableForm):
    fields = [
            # One hidden field to have a place for the method name
            twf.HiddenField("method")
            # One field per parameter
            twf.TextField("filter", help_text="Please enter the string to use as a filter"),
    #...
forms["list_colours"] = ListColoursForm()
And now our methods become much nicer to write:
    @expose("json")
    def api_validation_error(self, **kw):
        pylons.response.status = "400 Error"
        return dict(form_errors=pylons.c.form_errors)
    @expose("json")
    @validate(validator_dispatcher, error_handler=api_validation_error)
    def list_colours(self, filter=None, productID=None, maxResults=100, **kw):
        # Parameter validation is done by the form
        # Call API
        res = self.engine.list_colours(filter, productID, maxResults)
        # Return result
        return res
api_validation_error is interesting: it returns a proper HTTP error status, and a JSON body with the details of the error, taken straight from the form validators. It took me a while to find out that the form errors are in pylons.c.form_errors (and for reference, the form values are in pylons.c.form_values). pylons.response is a WebOb Response that we can play with. So now our client side is able to call the API methods, and get a proper error if it calls them wrong. But now that we have the forms ready, it doesn't take much to display them in web pages as well:
def _describe(self, method):
    "Return a dict describing an API method"
    ldesc = getattr(self.engine, method).__doc__.strip()
    sdesc = ldesc.split("\n")[0]
    return dict(name=method, sdesc = sdesc, ldesc = ldesc)
@expose("myappserver.templates.myappapi")
def index(self):
    '''
    Show an index of exported API methods
    '''
    methods = dict()
    for m in forms.keys():
        methods[m] = self._describe(m)
    return dict(methods=methods)
@expose('myappserver.templates.testform')
def testform(self, method, **kw):
    '''
    Show a form with the parameters of an API method
    '''
    kw["method"] = method
    return dict(method=method, action="/myapp/test/"+method, value=kw, info=self._describe(method), form=forms[method])
@expose(content_type="text/plain")
@validate(validator_dispatcher, error_handler=testform)
def test(self, method, **kw):
    '''
    Run an API method and show its prettyprinted result
    '''
    res = getattr(self, str(method))(**kw)
    return pprint.pformat(res)
In a few lines, we have all we need: an index of the API methods (including their documentation taken from the docstrings!), and for each method a form to invoke it and a page to see the results. Make the forms children of AjaxForm, and you can even see the results together with the form.

Gustavo Noronha Silva: Moving back to Belo!

So, next Saturday Luciana and I are moving back from Rio de Janeiro to Belo Horizonte after about 6 months here. It was an interesting ride. We didn t really have a good time finding a good place to live at - Rio is very expensive and it looks like demand is so high anywhere close to Luciana s work place that the only reasonable solution was to live very far away, and even there we didn t really have much success, although for other reasons I may talk about later. Belo is how the foreign people who live there usually refer to Belo Horizonte. I think that is probably because it s a bit too hard for them to say Horizonte =D (if it helps you, you could use the Mineirish Belzonte ). I have borrowed this svg file kindly licensed in the FDL by Raphael Lorenzeto de Abreu, and generated a simple png image that you can see in this post with Belo Horizonte, and some other cities we often talk about at Collabora =): A straight line between Recife and Belo Horizonte has about 1640km. There are 340km between BH and Rio, and 973km between BH and Florian polis. A bit about Belo Horizonte, for those friends of mine to whom I always have a hard time explaining where it s located, and what kind of city it is: Belo Hozizonte is a nice, beautiful city, with a quite big metropolitan area. About 6 million people live in greater Belo Horizonte. It is considered one of the three most influential cities in Brazil, along with S o Paulo and Rio de Janeiro. It s surrounded by mountains, which are considered one of the defining geographical features of the Minas Gerais state, of which it is the capital. Here s a picture that is available under the public domain in Wikipedia: Speaking of Minas Gerais, it is, according to Wikipedia, the fourth in size among the 26 States of Brazil, being just a bit bigger than mainland-France. Minas Gerais is one of the main earth transports hubs of the country (having the largest number of federal roads), and is famous for its tasty food, the beauty of its people, its cheeses, the calm, introspective nature of its people, its interesting way of speaking portuguese, and its alcoholic beverages. Now, going away from Rio to Belo Horizonte has a single big disadvantage associated with it: we don t get to have the Sea a few minutes away, anymore. Our solution to that is, of course, going to one of the numerous bars (the legend goes around that BH has the largest absolute number of bars of all cities in the world), and drinking instead of swimming! Quem n o tem mar, vai pro bar!

Romain Beauxis: Ocaml-cry

Coming along with the next release of liquidsoap, we have just released a new module for ocaml, namely ocaml-cry. Ocaml-cry is a native implementation of the protocols used for sending source data to icecast and shoutcast servers. The main difference with the famous libshout implementation is about flexibility. This module allows to set any source header, which in particular allows to use any custom format which is compatible with the servers. However, contrary to libshout, ocaml-cry does not handle the synchronisation of the audio data sent to the servers. Hence, the task of controling the rate of the data sent is up to the programmer. You can find more informations on this module there: http://www.rastageeks.org/cry.html Debian packages are available there [1]: http://packages.debian.org/libcry-ocaml-dev

[1] Many thanks to the ftp-masters for dealing with the NEW package so fast !

14 October 2009

Patrick Schoenfeld: PHP and the great "===" operator

Lets suppose you've got an array with numeric indices:
[0] => 'bla'
[1] => 'blub'
Now you want to do something if the element 'bla' is found in that array.
Well, you know that PHP has a function array_search, which returns the key(s) of the found values. Lets say, you write something like this:
if(array_search($array, 'bla')) do_something

Would you expect that do_something would actually do something?
If yes: You are wrong.
If no: Great, you've understand some parts of PHP - insanitygoodness..

Actually, if 'bla' wouldn't have the index 0 it would work, because 1, 2, 3, 4, etc. is TRUE. But unfortunately PHP has some sort of implicit casting, which makes 0 behave like a FALSE, depending on the context. So following this, the if() works for all elements except 0.

You might be tempted to write
if(array_search... != FALSE)
But this wouldn't help you, because 0 would still evaluate to FALSE, leading to if(FALSE != FALSE) which is (hopefully obviously) never true.

A PHP beginner (or even an intermediate, if he never stumbled across this case) might ask:
Whats the solution for this dilemma?

Luckily the PHP documentation is great. It tells you about this. And additionally PHP has got this great operator (===), which causes people to ask "WTF?" when they hear about it for the first time. Additionaly to comparing the values to each other, they also check the type of the variable. This leads to the wanted result because 0 is an integer, while FALSE is Boolean. So the solution for our problem looks like this:
if (array_search($array, 'bar') !== FALSE)
Isn't this great?

Robert McQueen: Telepathy Q&A from the Boston GNOME Summit

The first Telepathy session session on Saturday evening at the Boston GNOME Summit was very much of a Q&A where myself and Will answered various technical and roadmap issues from a handful of developers and downstream distributors. It showed me that there s a fair amount of roadmap information we should do better at communicating outside of the Telepathy project, so in the hope its useful to others, read on MC5 Ted Gould was wondering why Mission Control 4 had an API for getting/setting presence for all of your accounts whereas MC5 does not. MC5 has a per-account management of desired/current presence which is more powerful, but loses the convenience of setting presence in one place. Strictly speaking, doing things like deciding which presence to fall back on (a common example being if you have asked for invisible but the connection doesn t support it) is a UI-level policy which MC should not take care of, but in practice there aren t many different policies which make sense, and the key thing is that MC should tell the presence UI when the desired presence isn t available so it could make a per-account choice if that was preferable. As a related side point, Telepathy should implement more of the invisibility mechanisms in XMPP so it s more reliably available, and then we could more meaningfully tell users which presence values were available before connecting, allowing signing on as invisible. Since MC5 gained support for gnome-keyring, its not possible to initialise Empathy s account manager object without MC5 prompting the user for their keyring password un-necessarily (especially if the client is Ubuntu s session presence applet and the user isn t using Empathy in the current session but has some accounts configured). Currently the accounts D-Bus API requires that all properties including the password are presented to the client to edit. A short-term fix might be to tweak the spec so that accounts don t have to provide their password property unless it s explicitly queried, but this might break the ABI of tp-glib. Ultimately, passwords being stored and passed around in this way should go away when we write an authentication interface which will pass authentication challenges up to the Telepathy client to deal with, enabling a unified interface for OAuth/Google/etc web token, Kerberos or SIP intermediate proxy authentication, and answering password requests from the keyring lazily or user on demand. Stability & Security Jonathan Blandford was concerned about the churn level of Telepathy, from the perspective of distributions with long-term support commitments, and how well compatibility will be maintained. Generally the D-Bus API and the tp-glib library APIs are maintained to the GNOME standards of making only additive changes and leaving all existing methods/signals working even if they are deprecated and superseded by newer interfaces. A lot of new interfaces have been added over the past year or so, many of which replace existing functionality with a more flexible or more efficient interface. However, over the next 4-6 months we hope to finalise the new world interfaces (such as multi-person media calls, roster, authentication, certificate verification, more accurate offline protocol information, chat room property/role management), and make a D-Bus API break to remove the duplicated cruft. Telepathy-glib would undergo an ABI revision in this case to also remove those symbols, possibly synchronised with a move from dbus-glib to GVariant/etc, but in many cases clients which only use modern interfaces and client helper classes should not need much more than a rebuild. Relatedly there was a query about the security of Telepathy, and how much it had been through the mill on security issues compared to Pidgin. In the case of closed IM protocols (except MSN where we have our own implementation) then we re-use the code from libpurple, so the same risks apply, although the architecture of Telepathy means its possible to subject the backend processes to more stringent lockdowns using SElinux or other security isolation such as UIDs, limiting the impact of compromises. Other network code in Telepathy is based on more widely-used libraries with a less chequered security history thus far. OTR The next topic was about support for OTR in Telepathy. Architecturally, it s hard for us to support the same kind of message-mangling plugins as Pidgin allows because there is no one point in Telepathy that messages go through. There are multiple backends depending on the protocol, multiple UIs can be involved in saving (eg a headless logger) or displaying messages (consider GNOME Shell integration which previews messages before passing conversations on to Empathy), and the only other centralised component (Mission Control 5) does not act as an intermediary for messages. Historically, we ve always claimed OTR to be less appealing than native protocol-level end-to-end encryption support, such as the proposals for Jingle + peer to peer XMPP + TLS which are favoured by the XMPP community, mostly because if people can switch to an unofficial 3rd party client to get encryption, they could switch to a decent protocol too, and because protocol-level support can encrypt other traffic like SRTP call set-up, presence, etc. However, there is an existing deployed OTR user base, including the likes of Adium users on the Mac, who might often end up using end to end encryption without being aware of it, who we would be doing a disservice by Telepathy not supporting OTR conversations with these people. This is a compelling argument which was also made to me by representatives from the EFF, and the only one to date which actually held some merit with me compared to just implementing XMPP E2E encryption. Later in the summit we went on to discuss how we might achieve it in Telepathy, and how our planned work towards XMPP encryption could also help. Tubes We also had a bit of discussion about Tubes, such as how the handlers are invoked. Since the introduction of MC5, clients can register interest in certain channel types (tubes or any other) by implementing the client interface and making filters for the channels they are interested in. MC5 will first invoke all matching observers for all channels (incoming and outgoing) until all of them have responded or timed out (eg to let a logger daemon hook up signal callbacks before channel handling proceeds), all matching approvers for incoming channels until one of them replies (eg to notify the user of the new channel before launching the full UI), and then sending it to the handler with the most specific filter (eg Tomboy could register for file transfers with the right MIME type and receive those in favour to Empathy whose filter has no type on it). Tubes can be shared with chat rooms, either as a stream tube where one member shares a socket for others to connect to (allowing re-sharing an existing service implementation), or a D-Bus tube where every member s application is one endpoint on a simulated D-Bus bus, and Telepathy provides a mapping between the D-Bus names and the members of the room. In terms of Tube applications, now we ve got working A/V calling in Empathy, as well as desktop sharing, and an R&D project on multi-user calls, our next priority is on performance and Tube-enabling some more apps such as collaborative editing (Gobby, AbiWord, GEdit, Tomboy ?). There was a question about whether Tube handlers can be installed on demand when one of your contacts initiates that application with you. It d be possible to simulate this by finding out (eg from the archive) which handlers are available, and dynamically registering a handler for all of those channel types, so that MC5 will advertise those capabilities, but also register as an approver. When an incoming channel actually arrives at the approval stage, prompt the user to install the required application and then tell MC5 to invoke it as the handler. Colin Walters asked about how Telepathy did NAT traversal. Currently, Telepathy makes use of libnice to do ICE (like STUN between every possible pair of addresses both parties have, works in over 90% of cases) for the UDP packets involved in calls signalled over XMPP, either the Google Talk variant which can benefit from Google s relay servers if one or other party has a Google account, so is more reliable, or the latest IETF draft which can theoretically use TURN relays but its not really hooked up in Telepathy and few people have access to them. XMPP file transfers and one-to-one tube connections use TCP which is great if you have IPv6, but otherwise impossible to NAT traverse reliably, so often ends up using strictly rate-limited SOCKS5 -ish XMPP proxies, or worse, in-band base64 in the XML stream. We hope to incorporate (and standardise in XMPP) a reliability layer which will allow us to use Jingle and ICE-UDP for file transfers and tubes too, allowing peer to peer connections and higher-bandwidth relays to enhance throughput significantly. Future Ted Gould had some good questions about the future of Telepathy
Should Empathy just disappear on the desktop as things like presence applets or GNOME Shell take over parts of its function? Maybe, yes. In some ways its goal is just to bring Telepathy to end users and the desktop so that its worth other things integrating into Telepathy, but Telepathy allows us to do a lot better than a conventional IM client. Maemo and Sugar on the OLPC use Telepathy but totally integrates it into the device experience rather than having any single distinct IM client, and although Moblin uses Empathy currently it has its own presence chooser and people panel, and may go on to replace other parts of the user experience too. GNOME Shell looks set to move in this direction too and use Telepathy to integrate communications with the desktop workflow. Should Telepathy take care of talking to social networking sites such as Facebook, Twitter, etc? There s no hard and fast rule Telepathy only makes sense for real-time communications, so it s good for exposing and integrating the Facebook chat, but pretty lame for dealing with wall posts, event invitations and the like. Similarly on the N900, Telepathy is used for the parts of the cellular stack that overlap with real-time communications like calling and SMS, but there is no sense pushing unrelated stuff like configuration messages through it. For Twitter, the main question is whether you actually want tweets to appear in the same UI, logging and notification framework as other messages. Probably not anything but the 1-to-1 tweets, meaning something like Moblin s Mojito probably makes more sense for that. Later in the summit I took a look at Google Latitude APIs, which seem like something which Telepathy can expose via its contact location interface, but probably not usefully until we have support for metacontacts in the desktop. Can/will Telepathy support IAX2? It can, although we d have to do a local demultiplexer for the RTP streams involved in separate calls. It s not been a priority of ours so far, but we can help people get started (or Collabora can chat to people who have a commercial need for it). Similarly nobody has looked at implementing iChat-compatible calling because our primary interest lies with open protocols, but if people were interested we could give pointers its probably just SIP and RTP after you dig through a layer of obfuscation or two.
If you want to know more about Telepathy feel free to comment with some follow-up questions, talk to us in #telepathy on Freenode, or post to the mailing list.

Robert McQueen: Boston GNOME Summit 2009

I spent this weekend in Boston for the annual GNOME summit. I really enjoyed it this year, although there were fewer attendees than previously it felt very focussed and productive. There s some cool stuff going on, and it s always great to catch up with all of the usual free software suspects in Boston. Some highlights from the weekend: I was really impressed by Jason Clinton and others summaries of the sessions, which I think are really valuable for the people who couldn t make it to the summit. He asked me to take some notes about the first Telepathy session on Saturday evening while he was taking notes about the Outreach session. Rather than lumber him with my deranged scratchings from Tomboy, I ll blog them separately.

Dirk Eddelbuettel: RPostgreSQL 0.1-5

A new version of RPostgreSQL just went out to CRAN and should propagate via the mirrors over the next few days. This is a maintenance and bug fix release that addresses four of the seven issue at the Google Code page. Two more are really enhancement requests for which we encourage patches as we are unlikely to have to write them, and the last issue I have been unable to replicate. This release has benefitted greatly from the generous help of Neil Tiffin who now looks after all things OS X and keeps a good eye on the Google Code site (where I had, ahem. overlooked the existence of these open issue tickets for a few months), Joe Conway who we can count on for hard-core PostgreSQL issues as well as Jeff Horner for general R, DBI and database smarts. Some more details (including the full ChangeLog) are at the local RPostgreSQL page, the RPostgreSQL package page at CRAN and of course the Google Code repository for RPostgreSQL.

13 October 2009

Eddy Petrișor: OpenStreetMap, Debian competes with it

There is no secret that many Debian contributors share their time between various projects, while Debian is one of those projects.

I am no exception: Debian has been competing lately with OpenStreetMap (along with other projects) over my time and, sadly for Debian and fortunately for OSM, OSM has been winning a lot in the last year, since I bought my Nokia E71 phone which has a built-in GPS.

Still, this work on OSM has its own satisfactions, such as managing to map alone (not completely) my home town, Caracal, managing to import municipality data for the entire country, participating in the first mapping party in Romania, exactly one month ago.



Recently I received several invitations to publicly present or advocate these open projects I am involved in and this is really exciting and a huge reward for me.

Is really great to see things moving without me trying to push them, for a change!

Steve Kemp: There's no such thing as a wrong war

Once upon a time I wrote a blog compiler, a simple tool that would read in a bunch of text files and output a blog. This blog would contain little hierarchies for tags, historical archives, etc. It would also have a number of RSS feeds too. Every now and again somebody will compare it to ikiwiki and I'll ignore that comparison entirely, because the two tools do different things in completely different fashions. But I was interested to see Joey talk about performance tweaks recently as I have a blog which has about 900 pages, and which takes just over 2 minutes to build from start to finish. (Not this one!) I've been pondering performance for a while as I know my current approach is not suited to high speed. Currently the compiler reads in every entry and builds a giant data structure in memory which is walked in different fashions to generate and output pages. The speed issue comes about because storing the data structure entirely in memory is insane, and because sometimes a single entry will be read from disk multiple times. I've made some changes over the past few evenings such that a single blog entry will be read no more than once from disk (and perhaps zero times if Memcached is in use :) but that doesn't solve the problem of the memory usage. So last night I made a quick hack - using my introduction to SQLite as inspiration I wrote a minimal reimplementation of chronicle which does things differently: This is a significantly faster approach than the previous one - with a "make steve" job taking only 18 seconds, down from just over 2 minutes 5 seconds. ("make steve" uses rsync to pull in comments on entries, rebuilds the blog, then uses rsync to push the generated output into its live location.) ObFilm: If...

Gunnar Wolf: Linux ranks high on stability One more time

Linux ranks high on stability   One more time
Even at the most physical level. This is a cup that won't topple over when you accidentally kick the desk! Gunnar's Viking of Approval certifies it. I bought this Tux Mug (Mugx) in Colombia, from CeramiGeek. It feels a bit strange to drink penguin brain, but all in all, it is a great geek present ;-) Thanks a lot to Andr s Restrepo and his girlfriend for coming up with this product! :- I expect to get quite a bit of joy out of it. [update] CeramiGeek's site says they sell the mug for $20,000 Of course, that's Colombian pesos. Slightly over US$10. I don't know whether they ship outside Colombia, and am completely unaffiliated to them. But I surely wish them success!

Stefano Zacchiroli: TeX gadget of the day: detexify

decompile TeX symbols Here is the best TeX gadget ever: Detexify. It is basically a decompiler for TeX symbols: you draw them using the mouse and you get back the corresponding TeX markup. It (claims to) supports all of the comprehensive LaTeX symbol list, so that you can avoid browsing through it again and again in the future (and keeping various printed copies of it: at home, in the office, ...). I've used in the past some other drawing-based search engines, but I've always found them no more than toys. This is the first service that I find very useful based on similar technology. Also, source code is available from GitHub (which is unfortunately down at the time of writing, so I can't check the license ...). Note that the web interface relies on some HTML5 elements (e.g. canvas), so you'll need recent browsers such as Firefox/Iceweasel 3.5 to use it. (Thanks to Roberto Di Cosmo that pointed me to the service.)

Sha Liu: Let s stay in touch on LinkedIn


LinkedIn Sandyleo26 s, I d like to add you to my professional network on LinkedIn. - Sha Confirm that you know Sha Liu Every day, millions of professionals like Sha Liu use LinkedIn to connect with colleagues, find experts, and explore opportunities. 2009, LinkedIn Corporation

Russell Coker: New Servers a non-virtual Cloud

NewServers.com [1] provides an interesting service. They have a cloud computing system that is roughly comparable to Amazon EC2, but for which all servers are physical machines (blade servers with real disks). This means that you get the option of changing between servers and starting more servers at will, but they are all physical systems so you know that your system is not going to go slow because someone else is running a batch job. New Servers also has a bandwidth limit of 3GB per hour with $0.10 per GB if you transfer more than that. Most people should find that 3GB/hour is enough for a single server. This compares to EC2 where you pay $0.10 per GB to receive data and $0.17 to transmit it. If you actually need to transmit 2100GB per month then the data transfer fees from EC2 would be greater than the costs of renting a server from New Servers. When running Linux the EC2 hourly charges are (where 1ECU is provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor):


NAMECostDesc
Small$0.101.7G, 160G, 32bit, 1ECU, 1core
Large$0.207.5G, 850G, 64bit, 4ECU, 2core
Extra Large$0.4015G, 1690G, 64bit, 8ECU, 4core
High CPU Medium$0.201.7G, 350G, 32bit, 5ECU, 2core
High CPU Extra Large$0.807G, 1690G, 64bit, 20ECU, 5core
The New Servers charges are:


NAMECostDesc
Small$0.111G, 36G, 32bit, Xeon 2.8GHz
Medium$0.172G, 2*73G, 32bit, 2*Xeon 3.2GHz
Large$0.254G, 250G, 64bit, E5405 Quad Core 2Ghz
Jumbo$0.388G, 2*500G, 64bit, 2 x E5405 Quad Core 2Ghz
Fast$0.534G, 2*300G, 64bit, E5450 Quad Core 3Ghz
The New Servers prices seem quite competitive with the Amazon prices. One down-side to New Servers is that you have to manage your own RAID, the cheaper servers have only a single disk (bad luck if it fails). The better ones have two disks and you could setup your own RAID. Of course the upside of this is that if you want a fast server from New Servers and you don t need redundancy then you have the option of RAID-0 for better performance. Also I don t think that there is anything stopping you from running Xen on a New Servers system. So you could have a bunch of Xen images and a varying pool of Dom0s to run them on. If you were to choose the Jumbo option with 8G of RAM and share it among some friends with everyone getting a 512M or 1G DomU then the cost per user would be a little better than Slicehost or Linode while giving better management options. One problem I sometimes have with virtual servers for my clients is that the disk IO performance is poorer than I expect. When running the server that hosts my blog (which is shared with some friends) I know the performance requirements of all DomUs and can diagnose problems quickly. I can deal with a limit on the hardware capacity, I can deal with trading off my needs with the needs of my friends. But having a server just go slow, not knowing why, and having the hosting company say I can move you to a different physical server (which may be better or worse) doesn t make me happy. I first heard about New Servers from Tom Fifield s LUV talk about using EC2 as a cluster for high energy physics [2]. According to the detailed analysis Tom presented using EC2 systems on demand can compete well with the costs of buying Dell servers and managing them yourself, EC2 wins if you have to pay Japanese prices for electricity but if you get cheap electricity then Dell may win. Of course a major factor is the amount of time that the servers are used, a cluster that is used for short periods of time with long breaks in between will have a higher cost per used CPU hour and thus make EC2 a better option.

Jonathan McDowell: What to wear?

I tend to dress quite casually - unless there's a good reason to do otherwise I'll be in jeans and a t-shirt, or something similar. I'm comfortable in it and I take the view point that in general people shouldn't be making assumptions based on what I'm wearing. Of course they do, and while this can be infuriating at times it can also be amusing. The man at the RBoS who witnessed me signing the personal DD guarantee form for Black Cat obviously couldn't reconcile my appearance with what I was there to do, for example.

I'd never thought about this from the other side until a few weeks ago. I spoke to someone who explained how difficult it had become to conduct business meetings with other companies with the increased level of business casual. In fact on occasion different branches of the same company that he'd be meeting with would have different dress codes, so he'd go to something involving casual dress in the morning and something with full business suit attire in the afternoon and end up feeling over or under dressed.

This isn't likely to make me change my own behaviour in the immediate future (I don't have to interact with external parties as part of my current job). It did help me realise that there was another explanation for awkward behaviour when I'm my usual scruffy self that wasn't just about judging on appearances though.

(All of the above blatantly obvious once you think about it, but it took that conversation for me to do so.)

Next.

Previous.