Search Results: "Sandro Tosi"

11 April 2011

Sandro Tosi: Python: group a list in sub-lists of n items

A long list, and you want to process its items n at a time; easy, but how to split that list in sublists of n elements (except the last one, of course)?

I looked a bit into the stdlib but it doesn't seem to exist anything I could use (oh, did I say I'm still on 2.4?) so I directed my research to Google, and found a nice recipe at ActiveState, but it has the problem it discards the last list, if it has less than n items.

Searching again, I got more lucky with this article: it's a generator of tuples from a list, splitting every n elements and optionally return the last semi-full tuple. I slightly modified it to obtain:

def group_iter(iterator, n=2):
""" Given an iterator, it returns sub-lists made of n items
(except the last that can have len < n)
inspired by http://countergram.com/python-group-iterator-list-function"""
accumulator = []
for item in iterator:
accumulator.append(item)
if len(accumulator) == n: # tested as fast as separate counter
yield accumulator
accumulator = [] # tested faster than accumulator[:] = []
# and tested as fast as re-using one list object
if len(accumulator) != 0:
yield accumulator

How would you have done it?

2 March 2011

Sandro Tosi: MySQL master/slave chain

Have you ever needed to create a MySQL databases replication chain like A->B->C where B is slave of A and master of C? Me neither, until yesterday.

Since it took us about an afternoon to make it works (along with our DBAs, so we're not alone ;)) let's share some knowledge.

A very brief recap of how MySQL replication works:

  1. slave I/O thread connects to the master, gets the new information from the binlog files and stores them in the relay log;
  2. slave SQL thread reads the relay log and applies the changes to the slave database, without changing the slave binlog files.
That said, B replicates correctly from A, but C is unable to replicate from B because B doesn't change its binlog files with updates coming from A, because there's no changes done directly on B.

In order to make the chain works, you need to add the parameter log-slave-updates on B configuration: that will reply the changes from relay log to binlog, and so C will see the changes it needs to correctly replicate.

PS: mysqldump --master-data (executed on the slave server against the master) would help you set up the correct information for replication.

27 February 2011

Sandro Tosi: FTWCA Shani

As you probably know, Shani is our pet hedgehog. We finally managed to setup a blog just for her (sorry, it's in italian only), and also we uploaded tons of her photos.

You don't remember her? Here's a cute reminder:

21 January 2011

Sandro Tosi: EuroPython 2011 @ Florence, IT - it's coming!

Just when I was looking for a contact email to ask news about EuroPython 2011 dates... I noticed they are already there!! June 19 to 26 !

Book your flights, reserve your vacations, hope to see you all there!

PS: this year, PyCon Italia joins EuroPython in a single conference - the bigger the funnier

29 December 2010

Sandro Tosi: Re: Converting date to epoch

Alexander,did you even consider that I might need to convert a date to epoch in a python script? :)

28 December 2010

Sandro Tosi: Convert a date to epoch

it seems like an easy quest, ain't it? well, it took me far too long to get it done, so let just write it down, so that maybe I won't forget in 2 seconds:
>>> import time
>>> str = '2009-03-04'
>>> format = '%Y-%m-%d'
>>> time.mktime(time.strptime(str, format))
1236121200.0
>>> int(_)
1236121200
and here you have your epoch (if you just need seconds, use int()).

11 December 2010

Sandro Tosi: Welcome Shani!!

It's been two weeks since our "family" has a new element: an african hedgehog. Her name is Shani, that means wonder in swahili.



she is so cute, isn't she?

2 December 2010

Sandro Tosi: bts-link has a new home!

Since (already) several weeks, bts-link has a new home: busoni.debian.org! That's one of the machines where BTS is running, and it helps to let bts-link complete its job in about 30 mins each time. Thanks goes to Don Armstrong and Peter Palfrader for their support in the migration (from merkel).

Once again, let me encourage you to contact the bts-link developers mailing-list to request your project to be added to the tool in case it's not (we usually do some checks and adds from time to time, but if you notify us, you'll get it faster :) ).

5 November 2010

Sandro Tosi: "multi-strace"

Thanks to a colleague, we hereby present the multi-strace:
strace $(pgrep httpd awk ' print "-p " $1 ' tr '\n' ' ')
(it's left as exercise to the reader to understand what it does) if you want to save the output and the play with it, just add '-o <file>'.

Welcome, new handy tool!

22 October 2010

Sandro Tosi: Scribe (& thrift) for Debian Squeeze

Some time ago, I asked what are the common solutions for services remote logging. It was suggested scribe, an application log server developed at Facebook and later released as an open source project, and we finally managed to get this ready for Debian squeeze.

Scribe requires a couple of other tools to work: thrift, a framework for services development (born at Facebook too), and thrift-fb303, a piece of thrift used as a bassline layer for Facebook services.

Searching the web for some work-in-progress of a Debian packaging, I found the wonderful work done by SimpleGeo; since I have some changes to do (and also take the occasion to play a bit with github), I forked their repos in these ones: scribe, thrift and thrift-fb303.

I've prepared a repository with those 3 packages, if you would like to try scribe, give it a look. They are for squeeze because, well, after all I need it for work and since it requires Python 2.6, using squeeze was the easiest choice (backporting 2.6 to lenny could be a mess, even if there are packages out there) and so I could also see how the upgrade from lenny would be.

Let's now dive into the doc, and some examples too, to find out the several configuration options and decide what's best for us.

Sandro Tosi: URL shortcuts in Chromium

If you like the Firefox feature to create special bookmarks that are actual templates for specific urls, well, here's how I got the same for Chromium.

That's actually how WordReference is suggesting, so it's defining new search engines: right click on the URL bar, Edit seach engine, and then Add; now you can define what you like, f.e. :


and so now typing "pts package<package>" in the URL bar you'll get to the package PTS page.</package>

That's one of the core points I had to get fixed in order to migrate from FF to chromium, one step closer then.

20 October 2010

Sandro Tosi: Feeling welcome: Python, you're doing it right!

It was for a very long time (I think back from PyconIT 2) that I wanted to contribute to python. About a month and a half ago I started reading all the documentation I could about development processes & workflows, mercurial stuff (I don't have commit right, so I prefer working on a DVCS than on SVN), bugs management and so.

After all of this, less than month ago I started contributing very small patches; then when looking for "something to do" I found bugs that could be closed, in particular because they were already been fixed, and then last night R. David Murray proposed to give me "tracker privs", that moments after were granted \o/

What can I say? Thank you! that's something making me feel like I'm doing something useful to the language I love, and even if the privs are "not that much", it's a sign of trust, which really makes me happy!

As of now, I feel like I'm between "better be safe than sorry" and "it's easier to ask forgiveness than permission" mood, but we'll see how it goes.

Python, you're making me feel welcome, and so, once again, thank you!

17 September 2010

Sandro Tosi: HTTP requests specifying the Host header (in Python)

Since it took me some time to find this solution, I think it might be worth to share it.

When you have a web server listening on a single IP address but serving several domains, it's quite common to run:
$ curl -H "Host: domain" http://ip_address/path/to/the/page.html
if you need to view that domain directly pointing to the web server (so avoiding any balancers or network "magics" you might have in place).

Well, the question is: how to do that in Python? I find my answer in the httplib module:
import httplib
conn = httplib.HTTPConnection("ip_address")
conn.putrequest("GET", "/", skip_host=True)
conn.putheader("Host", "domain.ext")
conn.endheaders()
res = conn.getresponse()
print res.read()
HTH

28 June 2010

Carl Chenet: Debian maintainer

As some of you may (or more certainly may not) have noticed reading the Debian Project News, I recently became a Debian maintainer. A hugh thanks to Sandro Tosi (morph) and Piotr O arowski (piotr) for their careful reviews of my work. And thanks to Raphael Hertzog (buxy) to pull the trigger for the beginning of my involvment in Debian. I can t wait to attend Debconf 2010. See you there ! Follow me on Identi.ca: http://identi.ca/carlchenet

20 April 2010

Sandro Tosi: My nerd score is...


I am nerdier than 99% of all people. Are you a nerd? Click here to take the Nerd Test, get nerdy images and jokes, and talk on the nerd forum!


oh my...

14 April 2010

Sandro Tosi: select count(bugs) in reportbug where status is open = 91

well, the actual select (against UDD) would be
udd=> select source, all_bugs from bugs_count where source = 'reportbug';
source all_bugs
----------+----------
reportbug 91
but the result is the same: less than 100 bugs against reportbug! Yay!

As you can see from the bugs graph, the Christmas and Easter holidays helped quite a lot to squash those bugs. When I first step into the reportbug world, the bugs count was around 250, and I first imagined in my mind to brake the 200 barrier, than the 150 and finally the less-than-100 one: while the first 2 limits where the hardest one (new to the codebase, other interest) the last one is the one I'm most proud of.
They seem quite nice-to-have stuff but recently I've tried to concentrate on reply fast to new report, fix as many reported crashes as possible and implement some of the long-awaited features.

Additionally, sometimes "No, sorry" is the proper reply to a bug and then it can be closed: it's hypocrite to keep bugs opened while internally you think it's a wontfix but you're not closing them because of a fear of possibly upset the reporters.

There will be more in the future, that's for sure, but I think this is the reportbug we'll see in squeeze, modulo some new crashes or very interesting feature we want it, so expect minor releases from now until squeeze+1

26 March 2010

Sandro Tosi: I'm going to Pycon Italia 4

Yep, I'll be there this year too (sadly I can't say it's my fourth time, since I missed Pycon1).

I'm not that excited about the proposed talks, hey we don't need that much of Django :) I hope the invited speakers will surprise me, I'm confident the organization will do that.

See ya there!

17 March 2010

Sandro Tosi: Check Nagios from the desktop: nagstamon

I just discovered nagstamon and all the team fallen in love with it!

I tried to use Nagios Checker, the Firefox plugin to notify of any Nagios alert, but that doesn't play nicely with several opened windows (alerts are multiplied for the number of windows, since it seems everyone does the checks, not just one) and it tends to slow down Firefox, that's already quite slow per se :)

The upstream author provides a Debian package, so promptly I wrote to him asking if he can consider maintain the package in Debian, with me as mentor/co-maintainer and so (it's in Python so I can that :) ); let's see how it goes.

Give it a try, it's really simple and awesome!

6 March 2010

Sandro Tosi: Project Euler - Problem 14

Inspired by S. Lott and his blog post (and by the pure genius that xkcd is giving us these days, today included) I gave a look to Project Euler problem 14, that's about the Collatz conjecture.

The straightforward recursive solution:
def collatz(n):
if n == 1:
return 1
if n % 2 == 0:
return 1 + collatz(n/2)
else:
return 1 + collatz(3*n + 1)
very rapidly converges to... an error:
RuntimeError: maximum recursion depth exceeded
So I've recoded it a bit using a cache dictionary to store all the intermediate values in it:
cache =  1: 1 

def collatz(n, res):
nn = n

while True:
if n in cache:
if nn != n:
cache[nn] = cache[n]
return cache[nn]

if n % 2 == 0:
cache[n] = 1 + collatz(n/2, cache)
else:
cache[n] = 1 + collatz(3*n + 1, cache)
Now loping for all the numbers lower than 1 million we find the solution to the problem: the number with the longest path is 837799 with a path length of 524. The code runs in about 4secs.

Since I got that cache dict around to play with, I graphed with Matplotlib (strange, ha ;) ) the frequencies of each path length:
There are few short paths, several long paths but rare to occur, and most of the paths have length between 100 and 200 (more or less).

UPDATE: I've removed length variable, not needed (left from a previous version of the code) and fixed the path length for 837799, wrongly reported.

Sandro Tosi: Tune google-code-prettify for Blogger dark theme

Following up my post about using google-code-prettify to syntax highlight code on Blogger, I want to share what I did to tune it for a dark theme.

Big warning: I'm completely ignorant of any web development, html/css/js are kinda arab (no offense intended) for me, so I'm quite sure there are more elegant solutions to this, but at least it works, and seems to have broken nothing :)

So, the default CSS has some bad values for tokens colors, in particular for the keywords and tags (that are in dark blue) and for the plain text (that is in black, so completely hidden).

What I did was to simply override those values, using this table as reference; so now the code to add to Blogger "Minima Black" template (the one I'm using) is:
<link href='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css'
rel='stylesheet' type='text/css'/>
<style type='text/css'>
.pln color: #C0C0C0;
.kwd color: #4169E1;
.tag color: #4169E1;
</style>
<script src='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js'
type='text/javascript'> </script>

Next.

Previous.