Search Results: "ag"

2 October 2009

Ingo Juergensmann: Buildd.Net: source code available via SVN

As I already wrote some time ago I can't spend not that much time anymore on Buildd.Net how it deserves. Therefor I put up a SVN and Trac online. Here it is:


If you have questions or would like to contribute to the repository, please contact me.

Julien Danjou: Courier to dovecot migration

This week, I've managed to migrate from courier-imap to dovecot at work. I always had a good experience with dovecot, and I still have one. Dovecot performances are very good in comparison with courier. With that switch, we dropped the CPU usage of the server from 25 % to 10 %, and it's damn faster now. I have no idea why, but I think that it's better written looking at the code, and also that its usage of index files helps a lot. We got no problem getting things work with public folders either, so the switch was almost painless. The only problem we had is that Dovecot is too smart for some MUA. Consequently, we hit an 8 years old Mutt bug #969, which I also reported to the Debian BTS as #549204 with a not-well-tested-but-seems-to-work patch. Thanks to Claws mail, we also found a bug in dovecot 1.2.5, which should be fixed soon. Dovecot upstream is very responsive and that's always something nice to know when you use a free software.

Neil Williams: Cold calls and LinuxExpo

Normally I, like most people, ignore cold calls but someone asking about opinions on software caught my interest - doesn't happen often. Turned out to be a study using contact data gleaned from a now dead Linux magazine which, in turn, had been gleaned from the attendance information submitted to a now all-but dead conference: LinuxExpo (the data itself was hidden in the data stream behind those fancy barcodes on the admission badges).

Simple questions about which languages I used, which development environments I used (questioner hadn't heard of either anjuta or geany but then as she also asked me to spell Debian that was probably to be expected) as well as questions about magazines and other sources of technical information. (She had heard of LinuxFormat and El Reg so that wasn't too bad.) More complex questions about why I chose a particular tool which I couldn't really answer in just a few words seeing as Martin has done an entire thesis on the answers to that question!

Got a more interesting response when asked for my opinion on various vendors - 1 to 5 where 1 is rubbish and 5 is superb. Adobe: 1 - got an oooh, why's that? (umm, no source code and better free software alternatives). RedHat: 4 got a slightly surprised tone but Microsoft: 1 got a "Really? why?" - to which the only answer had to be "release all the source code". :-)

Interesting that RedHat was mentioned, along with Google and Sun/Oracle but neither Ubuntu nor Debian.

Gradually found that the inspiration for the (expensive) methodology came from companies trying to get me to use proprietary demos which then turned into subscription-model tools. No go area for me, all my tools will need to remain free software or I'll just find something else. (Got another "why?" to that - with the inevitable "I need the source code to fix the tools" answer.) Also asked "what are your hobbies?" - to which I could only say "You've spent the last 5 minutes asking about my hobby, my paid work is not in IT".

Despite the call ending not 5 minutes ago, I've no memory of who was doing the survey. Still, she seemed impressed that I've been subscribed to LinuxFormat since about issue 3. :-)

1 October 2009

Gunnar Wolf: Strange scanning on my server?

Humm... Has anybody else seen a pattern like this? I am getting a flurry of root login attempts at my main server at the University since yesterday 7:30AM (GMT-5). Now, from the machines I run in the 132.248.0.0/16 network (UNAM), only two listen to the world with ssh at port 22 And yes, it is a very large network, but I am only getting this pattern on one of them (they are on different subnets, quite far apart). They are all attempting to log in as root, with a frequency that varies wildly, but is consistently over three times a minute right now. This is a sample of what I get in my logs: [update] Logs omitted from blog post, as it is too wide and breaks displays for most users. You can download the log file instead. Anyway This comes from all over the world, and all the attempts are made as root (no attempts from unprivileged users). Of course, I have PermitRootLogin to no in /etc/ssh/sshd_config, but I want to understand this as much as possible. Initially it struck me that most of the attempts appeared to come from Europe (quite atypical for the usual botnet distribution), so I passed my logs through:
  1. #!/usr/bin/perl
  2. use Geo::IP;
  3. use IO::File;
  4. use strict;
  5. my ($geoip, $fh, %by_ip, %by_ctry);
  6. $fh = IO::File->new('/tmp/sshd_log');
  7. $geoip=Geo::IP->new(GEOIP_STANDARD);
  8. while (my $lin = <$fh>) next unless $lin =~ /rhost=(\S+)/; $by_ip $1 ++ ;
  9. print " Incidence by IP:\n", "Num Ctry IP\n", ('='x60),"\n";
  10. for my $ip ( sort $by_ip $a <=> $by_ip $b keys %by_ip)
  11. my $ctry = ($ip =~ /^[\d\.]+$/) ?
  12. $geoip->country_code_by_addr($ip) :
  13. $geoip->country_code_by_name($ip);
  14. $by_ctry $ctry ++;
  15. printf "%3d %3s %s\n", $by_ip $ip , $ctry, $ip;
  16. print " Incidence by country:\n", "Num Country\n", "============\n";
  17. map printf "%3d %s\n", $by_ctry $_ , $_
  18. sort $by_ctry $b <=> $by_ctry $a
  19. keys(%by_ctry);
The top countries (where the number of attempts 5) are:
  1. 104 CN
  2. 78 US
  3. 58 BR
  4. 49 DE
  5. 43 PL
  6. 20 ES
  7. 20 IN
  8. 19 RU
  9. 17 CO
  10. 17 UA
  11. 16 IT
  12. 13 AR
  13. 12 ZA
  14. 10 CA
  15. 10 CH
  16. 8 GB
  17. 8 AT
  18. 8 JP
  19. 8 FR
  20. 7 KR
  21. 7 HK
  22. 7 PE
  23. 7 ID
  24. 6 PT
  25. 5 CZ
  26. 5 AU
  27. 5 BE
  28. 5 SE
  29. 5 RO
  30. 5 MX
I am attaching to this post the relevant log (filtering out all the information I could regarding legitimate users) as well as the full output. In case somebody has seen this kind of wormish botnetish behaviour lately please comment. [Update] I have tried getting some data regarding the attacking machines, running a simple nmap -O -vv against a random sample (five machines, I hope I am not being too agressive in anybody's eyes). They all seem to be running some flavor of Linux (according to the OS fingerprinting), but the list of open ports varies wildly I have seen the following:
  1. Not shown: 979 closed ports
  2. PORT STATE SERVICE
  3. 21/tcp open ftp
  4. 22/tcp open ssh
  5. 23/tcp open telnet
  6. 111/tcp open rpcbind
  7. 135/tcp filtered msrpc
  8. 139/tcp filtered netbios-ssn
  9. 445/tcp filtered microsoft-ds
  10. 593/tcp filtered http-rpc-epmap
  11. 992/tcp open telnets
  12. 1025/tcp filtered NFS-or-IIS
  13. 1080/tcp filtered socks
  14. 1433/tcp filtered ms-sql-s
  15. 1434/tcp filtered ms-sql-m
  16. 2049/tcp open nfs
  17. 4242/tcp filtered unknown
  18. 4444/tcp filtered krb524
  19. 6346/tcp filtered gnutella
  20. 6881/tcp filtered bittorrent-tracker
  21. 8888/tcp filtered sun-answerbook
  22. 10000/tcp open snet-sensor-mgmt
  23. 45100/tcp filtered unknown
  24. Device type: general purpose WAP PBX
  25. Running (JUST GUESSING) : Linux 2.6.X 2.4.X (96%), ( )
  26. Not shown: 993 filtered ports
  27. PORT STATE SERVICE
  28. 22/tcp open ssh
  29. 25/tcp open smtp
  30. 80/tcp open http
  31. 443/tcp open https
  32. 444/tcp open snpp
  33. 3389/tcp open ms-term-serv
  34. 4125/tcp closed rww
  35. Device type: general purpose phone WAP router
  36. Running (JUST GUESSING) : Linux 2.6.X (91%), ( )
  37. Not shown: 994 filtered ports
  38. PORT STATE SERVICE
  39. 22/tcp open ssh
  40. 25/tcp closed smtp
  41. 53/tcp closed domain
  42. 80/tcp open http
  43. 113/tcp closed auth
  44. 443/tcp closed https
  45. Device type: general purpose
  46. Running (JUST GUESSING) : Linux 2.6.X (90%)
  47. OS fingerprint not ideal because: Didn't receive UDP response. Please try again with -sSU
  48. Aggressive OS guesses: Linux 2.6.15 - 2.6.26 (90%), Linux 2.6.23 (89%), ( )
  49. Not shown: 982 closed ports
  50. PORT STATE SERVICE
  51. 21/tcp open ftp
  52. 22/tcp open ssh
  53. 37/tcp open time
  54. 80/tcp open http
  55. 113/tcp open auth
  56. 135/tcp filtered msrpc
  57. 139/tcp filtered netbios-ssn
  58. 445/tcp filtered microsoft-ds
  59. 1025/tcp filtered NFS-or-IIS
  60. 1080/tcp filtered socks
  61. 1433/tcp filtered ms-sql-s
  62. 1434/tcp filtered ms-sql-m
  63. 4242/tcp filtered unknown
  64. 4444/tcp filtered krb524
  65. 6346/tcp filtered gnutella
  66. 6881/tcp filtered bittorrent-tracker
  67. 8888/tcp filtered sun-answerbook
  68. 45100/tcp filtered unknown
  69. Device type: general purpose WAP broadband router
  70. Running (JUST GUESSING) : Linux 2.6.X 2.4.X (95%), ( )
  71. Not shown: 994 filtered ports
  72. PORT STATE SERVICE
  73. 22/tcp open ssh
  74. 25/tcp open smtp
  75. 53/tcp open domain
  76. 80/tcp open http
  77. 110/tcp open pop3
  78. 3389/tcp open ms-term-serv
  79. Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
  80. Device type: firewall general purpose
  81. Running: Linux 2.6.X
  82. OS details: Smoothwall firewall (Linux 2.6.16.53), Linux 2.6.13 - 2.6.24, Linux 2.6.16
Of course, it strikes me that several among said machines seem to be Linuxes, but (appear to) run Microsoft services. Oh, and they also have P2P clients.
AttachmentSize
Results of parsing the logs21.58 KB
Relevant portion of the logs1.05 MB

Leo 'costela' Antunes: On the origins of perceived gender disparity

I was gonna write this as a comment to Daniel Kahn Gillmor s post on sexism in the FOSS community, but it got a bit too big and I noticed I couldn t log in with OpenID to comment, so here it goes. Do you think that the significant under-representation of women is a problem? I don t think it s a problem in itself, but it is a symptom which can point to a series of problems, with different levels of importance, depending largely on the cultural background involved. That is to say: the reason for this gender disparity may have significantly different roots depending on the place, some of which need to be addressed in completely different ways, if at all. Off the top of my head I d name a few:
  1. Cultural favoring of men as intellectually superior in any given field, filtering women out of participation in said field.
  2. Cultural/religious taboo of women as professionals in general.
  3. Cultural/religious taboo of learning for women.
  4. Cultural favoring of women as superior in some fields, draining other fields of prospective women professionals.
  5. Cultural characterization of certain fields as non-feminine, scaring women away on other grounds than technical capacity.
  6. Cultural expectations regarding pregnancy and marriage, coupled with age expectations for entry level jobs, barring entry to certain fields.
And since I m speaking theoretically, we could also imagine a maternalist culture in which women aren t attracted to IT because it s beneath their level , preferring instead careers in politics or whatever is seen as a superior career in this imaginary culture.
This is obviously bogus, but I feel the need to make at least one Devil s Advocate argument pointing out that women don t necessarily need help achieving equality everywhere. With this in mind, I d say we can start addressing the problems where they exist and for what they are. Seems like a clearer starting point, IMHO. Disclaimer: I m not really active in this endeavor, partly because I ve never believed in any particular large-scale plan to change the scenario. I m more of a small-scale guy: changing the world one person at a time (which is ironically contradicted by the sheer existence of this post, but anyway ).

Daniel Kahn Gillmor: sexist behavior in the free software community

So not even 3 months out from RMS's sexist Gran Canaria virgins remarks, we have another powerful leader in the Free Software Community making sexist remarks in a talk to developers (this time, it's Mark Shuttleworth). It's a shame that these two people have said stupid things that hurt their causes and their communities by perpetuating an unfriendly environment for women. And it's a bigger shame that neither leader appears to care enough about their community to issue a sincere public apology for their screwup (if i'm wrong about this, please point me to the apology i've looked). These guys are in a situation which is nowhere near as hard as writing good software or managing complex technical projects: if you make a stupid mistake, own up to it, apologize, and try not to make similar mistakes in the future. Perhaps worst of all, are the remarkable number of unreasonably fucked-up comments on the blog posts discussing these unfortunate events. If you're in the habit of defending remarks like those made by RMS and Shuttleworth on the 'net, please take a minute and ask yourself a few questions: Please read the Geek Feminism wiki and blog. Even if you don't agree with everything on those sites (hey, it's a wiki! and a blog! you don't have to agree with everything!), people are at least trying to address the problem of sexism in our community there. Engage constructively and don't hide or ignore problems! UPDATE: I've asked the site admin to remove some trollish comments. If you want to use the comments on this blog to announce something like "i'd like for free software to be a boys-only club", that's fine, i'd like to know who you are so i can adjust my perception of you accordingly. Anything advocating or excusing rape and/or murder is way over the line and is not welcome here. i'm sure you can find some other place to troll. Tags: community, conduct, feminism, free software, sexism

Gerfried Fuchs: Times Are Changing

Since I returned from this year's debconf quite a lot things have changed in my life. One thing I knew way before debconf already, I switched jobs. I was working for a pretty long time (and also through great times) for Silver Server, for well over 6 years. Let's see what I can do in and for my new job. I'm still only pretty short there yet and I being to understand what's going on (and what could need improvements in some workflows) but it's too early to see that more precisely. On a small side-note, the job change also required me to switch my mobile phone number because the old one was a company's number. I now have a private one that I don't fear of losing anymore. If you subtract the number 3933309527644 from my old one you have my new one. On the other hand you can always ask me for it in case you had my old one and wonder. ;) Besides my mobile phone number other smaller things have changed over time too. Most of you noticed that I'm neither using my @ist.org nor my @debian.org address anymore because they did carry the wrong nick. And given that everyone was switching their gnupg keys I did so too. Like before I again created separate keys for private usage and for Debian usage. I'm not really following the procedure of sending out signing requests to people who have signed my old key(s), personally I don't follow them myself so I don't expect others to do. They are though cross-signed with the old ones just to be on the save side to keep the WoT not losing connections. Also I finally got around to set up my own ejabberd server and did create my a new jabber ID that I also plan to keep. It is the same as my private mailaddress as you can see linked above in the key for private usage. ;) One of the reasons for the server is by the way also a testbed for the packaging of ejabberd which will get moved into team maintenance with the next upload which will be the next upstream version, 2.1. But the biggest and most important change in my life is that I will have to rethink my spare time spending quite a lot because there will be something demanding a lot of time starting in about march next year. An extremely fortunate change and almost as happy one even though it means that I won't be able to attend next year's debconf in NY, but I hope the people whom I told that I will be there aren't too disappointed by this announcement. Sorry! :)

Ingo Juergensmann: Debian Unstable - nomen est omen

Oh well... in the last days my machine crashed during the nightly backup. When I restarted it the next morning I realized two problems: the LVM is not recognized anymore and my X crashes on startup.
With the help of Tino Keitel (scorpi) I fixed the first issue by installing an older version of the lvm2 package (namely: lvm2_2.02.39-7_i386.deb) and libdevmapper1.02.1 (namely: libdevmapper1.02.1_1.02.37-1_i386.deb). Tino reported this in Bug #546817.

But there's still the Xorg problem left. Xorg.0.log reports:

CODE:
(II) RADEON(0): RADEONRestoreMemMapRegisters() :
(II) RADEON(0): MC_FB_LOCATION : 0xefffe800 0xefffe800
(II) RADEON(0): MC_AGP_LOCATION : 0xf87ff800
restore common
restore crtc1
restore pll1
finished PLL1
set RMX
set FP1
enable FP1
disable primary dac
disable TV
(II) RADEON(0): RandR 1.2 enabled, ignore the following RandR disabled message.

Backtrace:
0: X(xorg_backtrace+0x3b) [0x81314bb]
1: X(xf86SigHandler+0x51) [0x80c57d1]
2: [0xb7f32400]
3: X(xf86DiDGAInit+0x2f) [0x80f08cf]
4: X(xf86CrtcScreenInit+0x110) [0x80ed9e0]
5: /usr/lib/xorg/modules/drivers//radeon_drv.so [0xb7a2a114]
6: X(AddScreen+0x19d) [0x80712cd]
7: X(InitOutput+0x206) [0x80adfa6]
8: X(main+0x1db) [0x80719bb]
9: /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5) [0xb7bb37a5]
10: X [0x8071051]

Fatal server error:
Caught signal 11. Server aborting


Maybe this is Bug #546586? I've already detached the second display, but it still crashes. Pointers and solutions appreciated!

UPDATE: thanks for the points! X is now working again!

Eddy Petri&#537;or: Some Debian work in a long time

I finally managed to do some Debian work in a long time, I managed to package Wormux 0.8.5 for Debian Sid (aka Unstable).

This new version delivers one new binary package, wormux-servers, which contains the stand alone (aka headless) game server and the index server. Unfortunately, the information on setting these up resumes to have a couple of configuration files which are placed in /usr/share/wormux/examples/ .

Also, there's a bug that affects network games when using the construction tool, but fortunately, upstream already has some fixes, including one for this bug, and as soon as 1:0.8.5-1 enters squeeze, I'll prepare a patched version (including some other fixes present in upstream).

Until then, please enjoy Wormux.

Martin F. Krafft: Thesis submitted to examiners

At 14:45 CEST, today, 1 October 2009, I submitted my thesis to the print shop. Sarah has kindly agreed to pick it up tomorrow and drop it off at Brian s office, from where it will go to the three examiners. On 30 October at 9:30 IST, I will stand to publicly defend my work at the University of Limerick. I expect to receive a list of changes to be made from the examiners, hopefully minor, such that there won t be much in the way before I submit the final version to the University Board by 3 December. Once they give their green light, I will make it available on my research website, along with the data. I expect this to happen in January 2010. I have many many people to thank and will do so in due time. Right now I just want to thank Brian Fitzgerald and Kieran Conboy for their flexibility. I was supposed to have the thesis done by the end of August to given them a month to read it over, but by then, I only had the first six (of eight) chapters done. We hence spent all of September sending back and forth PDFs and paper comments, working iteratively so that today, I could put the finishing touch on the work and keep the schedule. The next thing on my agenda is the second edition of my book, which we are targetting for Debian squeeze . If you would like to be the first to receive more details, sign up to the announcement mailing list.

John Goerzen: MoinMoin as a Personal Wiki, Zen To Done, And A Bit of Ikiwiki

Since I last evaluated and complained about wikis last year, I ve been using moinmoin for two sites: a public one and a personal one. The personal site has notes on various projects, and my task lists. I ve been starting out with the Zen To Done (ebook, PDF, paper) idea. It sounds great, by the way; a nice improvement on the better-known GTD. My To Do Page Anyhow, in MoinMoin, I have a ToDos page. At the top are links to pages with different tasks: personal, work, yard, etc. Below that, are the three big rocks (as ZTD calls them) for the day: three main goals for the day. I edit that section every day. The Calendar And below that, I use MoinMoin s excellent MonthCalendar macro. I have three calendars in a row: this month, next month, and last month. Each day on the calendar is a link to a wiki page; for instance, ToDos/Calendar/2009-10-01. The day has a red background if the wiki page exists, and white otherwise. So when I need to do something on or by a specific day, I click on the link, click my TaskTemplate, and create a simple wiki page. When I complete all the tasks for that day, I delete that day s wiki page (and can note what I did as the log message if I like). Very slick. The Task Lists My task pages are similar. They look like this:
= Personal =
<<NewPage(TaskTemplate,Create new task,@SELF)>> <<Navigation(children,1)>>
<<BR>>
So, my personal task page has a heading, then it has an input form with a text box and a button that says Create new task. Type something in and that becomes the name for a wiki page, and takes you do the editor to describe it. Below the button is a list of all the sub-pages under the Personal page, which represent the tasks. When a task is done, I delete the page and off the list it goes. I can move items from one list to another by renaming the page. It works very, very nicely. Collecting Part of both ZTD and GTD is that it must be very easy to get your thoughts down. The idea is that if you have to think, I ve got to remember this, then you ll be stressed and worried about the things you might be forgetting. I have a Collecting page, like the Personal or Work pages, that new items appear on when I m not editing my wiki. They get there by email. MoinMoin has a nice email system. I ve set up a secret email address. Mail sent there goes directly into MoinMoin. It does some checks on it, then looks at a combination of the From and Subject lines to decide what to do with it. If I name an existing page, it will append my message the the end. If it s a new page, it ll create it. I have it set up so that it takes the subject line as a page name to create/append to under ToDos/Collecting/$subject (by putting that as the name on the To line). So, on my computers, I have a newtodo script that invokes mail(1), asks for a subject, and optionally lets me supply a body. Quick and painless. Also, I ve added the address to my mobile phone s address book. That way I don t have to carry around pen and paper. Need to get down some thought ? No problem. Hit send email, pull the last address sent to, give it a subject and maybe a body. Very slick. Wiki Software As a way of updating my posts from last year: I ve been very happy with MoinMoin overall. It has some oddities, and the biggest one that concerns me is its attachment support. It doesn t let you specify a maximum upload size, and doesn t very well let you restrict attachment work to only certain people. But the biggest problem is that it doesn t track history on attachments. If a vandal deletes the attachment on a page, it s GONE. They expect to have that fixed in 2.0, coming out in approximately November, 2010. I also looked at Ikiwiki carefully over the past few days. Several things impressed me. First, everything can be in git. This makes for a very nice offline mode, better than Moin s offline sync. The comment module is nicer than anything in Moin, and the tagging system is as well. Ikiwiki truly could make a nice blog, and Moin just couldn t. It also puts backlinks at the bottom of each page automatically, a nice feature. And it s by Joey Hess, who writes very solid software. There are also some drawbacks. Chief on that list is that ikiwiki has no built-in history of a page feature. Click History and it expects to take you to gitweb or ViewVC or some such tool. That means that reverting a page requires either git access or cut and pasting. That s fine for me, but throwing newbies to gitweb suddenly might not be the most easy. Since ikiwiki is a (very smart) wiki compiler, its permission system is a lot less powerful than Moin s, and notably can t control read access to pages at all. If you need to do that, you d have to do it at the webserver level. It does have a calendar, but not one that works like Moin s does, though I could probably write one easily enough based on what s there. A few other minor nits: the email receiving feature is not as versatile as Moin s, you can t subscribe to get email notifications on certain pages (RSS feeds only, which would have to be manually tweaked later), and you can t easily modify the links at the top of each page or create personal bookmarks. Ikiwiki looks like an excellent tool, but just not quite the right fit for my needs right at the moment. I ve also started to look at DokuWiki a bit. I was initially scared off by all the plugins I d have to use, but it does look like a nice software. I also re-visited MediaWiki, and once again concluded that it is way too complicated for its own good. There are something like a dozen calendar plugins for it, some of which even are thought to work. The one that looked like the one I d use had a 7-step (2-page) installation process that involved manually running SQL commands and cutting and pasting some obscure HTML code with macros in it. No thanks.

30 September 2009

John Goerzen: Sen. Brownback s Visit

Since I asked for suggestions on the Brownback town hall, I figured I ought to report what happened. He was scheduled to spend about 30 minutes in it; he probably spent a total of 40 or 45. He arrived a few minutes early, went around the room shaking hands and asking each person their name (including me). It was polite on the one hand, and a little odd on the other, as he was occasionally reaching over chairs to reach people far away. Anyhow, he stood up front and had a little speech to start with. He spent the first 5-10 minutes talking about local sports teams. It might have been a nice icebreaker if it was 30-60 seconds, but it just felt weird by the time it was over. Everybody knew he was here to talk about other things and had very limited time. He then launched into the economy, talking about keeping taxes down and making sure they re low in Kansas for the recovery. He touched on energy, supporting wind energy (which led to his most memorable comment of the morning: This state blows, and it s got lots of wind, too. ) He also came out in favor of coal, oil, and gas. Basically he s pro-energy. Can t go wrong with that, eh? He rode our prototype all-electric ZTR mower, and praised it quite a bit. He touched on health care briefly, saying that he favors incremental changes to bring down costs. He thought health savings accounts and high-deductible health plans (such as one option at my employer with a $2500 deductible) were good innovations. He then opened it up for questions. There was a brief awkward moment when nobody raised their hand, so I did. I started by saying where I agreed with him. I told him I agreed that health care is way too expensive, that I agree that we need liability reform, and said that we pay almost as much each month in health care as we do in all federal taxes combined. I was going to continue, but he interrupted, and said, Are you serious? Yes. 80% as much in fact. Do you have an illness in your family? My wife has asthma, a thyroid disorder, and we had a baby this year. That s it. Wow. So I continued. I told him I was concerned about the 46 million uninsured Americans, that they weren t getting the care they needed. I pointed out that many of them wouldn t be able to afford that $2500 deductible in the HDHP, let alone regular doctor visits. And I said that it is our moral and even religious duty to take care of the least of these when it comes to health care. He answered by first saying that nobody goes without care that s the law because anybody can get care in any emergency room in the country. Then he spent several minutes describing how he thinks the government can t manage anything, and thus a single-payer system wasn t right for us. After a few minutes of this, he asked if that answered my question. I said that I don t care how the 46 million people get care, whether it s a public option or subsidies for private companies, just that they get it. I also acknowledged that ERs must provide necessary stabilizing care , but pointed out that 20,000 Americans die each year due to inadequate health care and reminded him that ERs are not required to provide diagnostic or preventative services, or long-term treatment of things like cancer. For the first, but not the last, time of the day, at that, he got really quiet, sort of hung his head, and said something like, Yeah, I agree. It is hard on the least of these. I wish we could do more. And moved on to the next question. At this point, it got weird. The next question was a guy that stood up, saying how we ve got to put God back into the constitution. He said he s lived long enough to see the moral decline in this country, and it s going to hell faster than we know. We need to be God-fearing again, he said. Brownback, in reply, again did the quiet hang-his-head thing and said, I agree. Then moved on. We then had a question about state s rights, nullification , and the 10th amendment. Though it was obvious some issue was underneath it, I couldn t figure out just what it was. Brownback, predictably, thinks that the federal government has grown way too powerful. (Ironic for someone that supported the Patriot Act, but hey.) Then one other question, and as he was going to wrap it up, a yelled question from the back: What s good about Kansas? Brownback was caught a bit off-guard, but launched into talking about aerospace, and how he hopes to lure wealthy retirees from California to settle here. All in all, an interesting experience. More respectful than I feared, more scary than I imagined, and about as genuine as I expected a politician at that level gets.

Daniel Stone: fd.o affected by portland state uni power failure

As ajax quite elegantly summed up, due to a series of catastrophic power failures at PSU, where fd.o is hosted, we were down for a good chunk of yesterday. Despite the machines having redundant power supplies, being connected to separate power rails in the rack, which were hooked up to independent, UPS-backed, power supplies, we still (like a good chunk of Portland, and certainly everyone in the PSU machine room) lost our power.

As far as we can tell, when annarchy.fd.o (websites, people.fd.o, cgit, anongit, et al) came back up, power was again interrupted while the ext3 journal was being replayed. When it came up the n'th time, fsck dumped almost the entire filesystem in lost+found, then started saying increasingly unhappy things about the state of the filesystem on its second pass. In the end, we just went with mkfs, and now we have a brand new and shiny filesystem.

It's worth pointing out that even if this was another filesystem, such as /srv, which hosts all project data, we would've been fine, as they're all backed up. But, unfortunately for some, we made a decision a while ago to not back /home up, and didn't advertise that as widely as we should have. So, if you had stuff in annarchy:/home, it's now gone, and I hope you have backups.

Sorry about that. On the upside, I got to see PSU's new and really very nice machine room this morning, thanks to XDC being about 250m away from the PSU machine room, and fd.o is otherwise running fine. We've been talking this week about replacing our ageing hardware, which would also allow for more redundancy as well as better performance from those machines. But we still have no plans to back up /home, so if you put stuff there, please, please keep your own backups (or make sure the Wayback Machine knows about it).

Jon Dowland: FAC-1

This is my first attempt at homebrew. It's from a stout kit, purchased from the Elderberry Home Brew Centre, which you can find at 28 Station Road, Whitley Bay. The fellow there is very helpful and gave me lots of advice on starting out. I'd recommend anyone interested in beer or wine making in the Newcastle or North Tyneside area to make the journey. In honour of a period in modern British music I am really fond of, I've decided to call it "FAC-1". Thus, when the first stage is complete and I siphon off a sampler, I can call that "FAC-2".

Steve Kemp: I've got a sick friend. I need her help.

There was a recent post by Martin Meredith asking about dotfile management. This inspired me to put together a simple hack which allows several operations to be carried out:
dotfile-manager update [directory]
Update the contents of the named directory to the most recent version, via "hg pull" or HTTP fetch. This could be trivially updated to allow git/subversion/CVS to be used instead. (directory defaults to ~/.dotfiles/ if not specified.)
dotfile-manager link [directory]
For each file in the named directory link _foo to ~/.foo. (directory defaults to ~/.dotfiles/ if not specified.) e.g. directory/_screenrc will be linked to from ~/.screenrc. But hostnames count too! So you can create directory/_screenrc.gold and that will be the target of ~/.screenrc on the host gold.my.flat
dotfile-manager tidy
This removes any dangling ~/.* symlinks.
dotfile-manager report
Report on any file ~/.* which isn't a symlink - those files might be added in the future.
Right now that lets me update my own dotfiles via:
dotfile-manager update ~/.dotfiles
dotfile-manager update ~/.dotfiles-private
dotfile-manager link ~/.dotfiles
dotfile-manager link ~/.dotfiles-private
It could be updated a little more, but it already supports profiles - if you assume "profile" means "input directory". To be honest it probably needs to be perlified, rather than being hoky shell script. But otherwise I can see it being useful - much more so than my existing solution which is ~/.dotfiles/fixup.sh inside my dotfiles repository. ObFilm: Forever Knight

Zak B. Elep: Off We Go...

Arrived at Manila yesterday. At least in the parts we passed through (Pasay, Makati, and Malate,) everything seems back to normal, with almost no sign of the previous storm's havoc. And nary a raindrop in sight. I'm staying at my cousin Deng's place, as usual with my Manila trips. We found it in a bit of chaos, as ondoy's flooding reached here, recalling the water being almost knee-deep. The water's gone now though, but a lot of things that were not brought up to the apartment's second floor are still wet. Hopefully they will be dry before the next storm comes in... A little history: the last time I lived in Manila was in September 2006, when Orange & Bronze got me for a 3-month contract work on a project. At the time, I was fresh off my GSoC work, and I was raring to whet my skills as a software developer. That job gave me that opportunity, as well as opportunities to meet new friends, participate in a bit of commercial software development, and get used to Manila life again. Then awhile later I got into the Morph Labs bandwagon to become its first Systems Administrator, working from Daet via telecommuting. It was here that I began the long road to unrecovery^H^H^H^H^H learning the ropes of being a sysadmin, building some cool stuff in the process as well as maintaining it. Now, I'm back here in Manila, to work for Orange & Bronze again, but this time in a more permanent position, as Systems Administrator Lead. I look forward to reconnecting with my old pals as well as meet new ones, use the valuable experience I've had over the years in my new job, and get back to the long road of unrecovery^H^H^H^H^H building more cool stuff. And I start doing that tomorrow. Good times ahead.

Enrico Zini: Lessons learnt from Oracle

Lessons learnt from Oracle Lesson number 1: "how to handle temporary files".
$ rm tp*
$ proc code=cpp lines=yes dba_vm.pc 
Pro*C/C++: Release 10.2.0.1.0 - Production on Wed Sep 30 11:10:00 2009
Copyright (c) 1982, 2005, Oracle.  All rights reserved.
System default option values taken from: /usr/local/oracle/10.2.01/db_1/precomp/admin/pcscfg.cfg
$ echo $?
0
$ ls tp*
tpoHRjc8  tpqkU4Cp  tpY5Eo4G
$ 
Today we learn this: for every successful compilation of a single source file, it is a Good Thing to leave three temporary files around as a tribute to the Holy Trinity who bestowed upon us the grace of seeing our prayers fulfilled and our work rewarded. For those illiterate of us who do not want to learn from the Market Leaders, and do not want to put a rm -f tp* in their makefiles just to avoid in the future to see hours of work on tpl_support.cc disappear at the first invocation of make, here's the Yokel's Wrapper:
#!/bin/sh -ue
if [ $# -ne 2 ]
then
        echo "Usage: $0 infile outfile" >&2
        exit 1
fi
DIR= mktemp -d 
cleanup()  
        rm -rf "$DIR"
 
trap cleanup EXIT
cp "$1" "$DIR"
(
        cd $DIR
        proc code=cpp lines=yes "$1"
        mv  basename "$1" .pc .c "$2"
)
mv "$DIR/$2" .
exit 0

Romain Beauxis: About free software and yellow pages...

Even if it has happened in the past that I could support a pragmatic approach in the context of free software and media technologies, there is something that offenses me in the context of free audio players. In the world of audio streaming servers on for net radios, there are two majors competitors. In the one hand, Shoutcast, the commercial proprietary streaming server released by Nullsoft, which is also the editor of Winamp, the popular media player. On the other hand, Icecast, which started as an open source implementation of the protocol used by Shoutcast, but was rewritten in its version 2 release to support the free codecs from xiph. Now, appart from the respective quality of the two servers, the real competition takes place in another place: availability of the radios using this or that server. For that purpose, both have implemented Yellow Pages. Shoutcast's yellow pages are here, whereas Icecast/Xiph's yellow pages are there. The great thing with yellow pages is their integration into the audio player. Then, the user can browse the list of radios, and test them. It is very good both for convenience no need to download a playlist or enter manually a complicated URL and advertisement a single place to list all possible radios. The yellow pages from Shoutcast are very popular. They are integrated in Winamp of course, but also in many other popular players, in the PSP, in the iPhone, etc... The Shoutcast yellow pages are reserved for shoutcast users. Of course, the protocol has been reversed it is kind of dummy but if your radio get caught registering to Shoutcast's yellow pages while not using their proprietary software, your radio gets blacklisted and you loose a whole lot of listeners (and money too). For that reason, many net radios want to use Shoutcast as their broadcasting server. From the point of view of Nullsoft, it seems a good call. At least, they play their game and make their money out of it, so let's put this appart. But, what about the free software world ? The obvious idea would be that the free software projects should push altogether for the use of free alternatives, hence pushing for a support of Icecast's yellow pages in the free players. Of course, free software being also a matter of choice, it should be possible to also support Shoutcast's yellow pages. After all, both protocols are very simple, so it is no big deal to implement.. Well, unfortunately, that is not exactly what happens.. Guess what, Amarok and VLC only implement Shoutcast's yellow pages... Given that VLC is a massively popular audio player that spreads among a lot of plateforms, I find it quite a pity that they do not do something to encourage its users to use free alternatives to the proprietary Shoutcast yellow pages, and similarly for Amarok. The adoption of free alternatives only works with massive demand, and we should not hope that proprietary softwares help in any way, but, at least, free softwares should be a bit more aware of what they're advocating when they choose to implement only the proprietary support...

Dirk Eddelbuettel: RInside release 0.1.1, and a fresh example

Last week's 0.1.0 release of RInside, and the first to have been published on CRAN, still had some issues with builds and use on OS X. Thanks to testing and fixes by Jan de Leeuw, Jeff Horner and particularly Simon Urbanked, things are said to be better now with the new release 0.1.1 which went onto CRAN yesterday. So no new features, but fixes to the main Makefile as well as the Makefile for the examples directory, some minor fixes and editing for the examples. I also added a file THANKS to show some appreciation for the various patches and fixes I have been receiving -- they are appreciated! However, today I committed a new example to SVN archive at R-Forge. It is based on this thread on r-devel. Abhijit Bera tries to do this in C, but to me his questions provide rather clear motivation for showing how much simpler things can be via C++ and the Rcpp classes along with RInside. Using a small example, the task was to pass a weight vector to a portfolio solver from the Rmetrics package fPortfolio and to then access the computed solution. The original poster struggled with access from C to the S4 classes used by fPortfolio and could not set the weights. But when using RInside, we simply pass a C++ vector of weights down to R, solve the problem and pass a solution vector back using the handy evaluation of R expressions:
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Another simple example inspired by an r-devel mail by Abhijit Bera
//
// Copyright (C) 2009 Dirk Eddelbuettel and GPL'ed
#include "RInside.h"                    // for the embedded R via RInside
#include "Rcpp.h"                       // for the R / Cpp interface used for transfer
#include <iomanip>
int main(int argc, char *argv[])  
    try  
        RInside R(argc, argv);          // create an embedded R instance
        SEXP ans;
        std::string txt = "suppressMessages(library(fPortfolio))";
        if (R.parseEvalQ(txt))          // load library, no return value
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
        txt = "lppData <- 100 * LPP2005.RET[, 1:6]; "
	  "ewSpec <- portfolioSpec(); "
	  "nAssets <- ncol(lppData); ";
        if (R.parseEval(txt, ans))      // prepare problem
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	const double dvec[6] =   0.1, 0.1, 0.1, 0.1, 0.3, 0.3  ; // choose any weights you want
	const std::vector<double> w(dvec, &dvec[6]);
	R.assign( w, "weightsvec");	// assign STL vector to R's 'weightsvec' variable
	txt = "setWeights(ewSpec) <- weightsvec";
        if (R.parseEvalQ(txt))		// evaluate assignment
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	txt = "ewPortfolio <- feasiblePortfolio(data = lppData, spec = ewSpec, constraints = \"LongOnly\"); "
	  "print(ewPortfolio); "
	  "vec <- getCovRiskBudgets(ewPortfolio@portfolio)";
        if (R.parseEval(txt, ans))      // assign covRiskBudget weights to ans
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	RcppVector<double> V(ans);      // convert SEXP variable to an RcppMatrix
	R.parseEval("names(vec)", ans);	// assign columns names to ans
	RcppStringVector names(ans);
	for (int i=0; i<names.size(); i++)  
	  std::cout << std::setw(16) << names(i) << "\t"
		    << std::setw(11) << V(i) << "\n";
         
      catch(std::exception& ex)  
        std::cerr << "Exception caught: " << ex.what() << std::endl;
      catch(...)  
        std::cerr << "Unknown exception caught" << std::endl;
     
    exit(0);
 

Chris Lamb: GeoDjango and the UK postcode database

This post describes how to play with the UK postcode database that recently turned up on WikiLeaks using GeoDjango. You will need the following pieces of free software:
  1. PostgreSQL (MySQL does not provide sufficient geospacial features) I will be using Postgres 8.4.1; please adjust your paths for your version and/or distributor.
  2. PostGIS spacial database extensions I will be using version 1.4.0.
  3. Django I will be using version 1.1. This version includes the GeoDjango framework.
  4. GEOS geometry engine I will be using version 3.0.0 from the libgeos-c1 package in Debian unstable.
You will also need a copy of the leaked postcode data. Usage of this data is almost certainly a copyright violation.
Getting started First, create a Django project and application for this example:
$ django-admin startproject ukpostcodes
$ cd ukpostcodes
$ ./manage.py startapp postcodes
Next, edit settings.py:
  • Configure DATABASE_*. You must use the postgresql_psycopg2 PostgreSQL database driver.
  • Add django.contrib.gis & ukpostcodes.postcodes to INSTALLED_APPS.
If you have not already done so, you must install PostGIS in your database:
$ sudo -u postgres createlang plpgsql -d <databasename>
$ sudo -u postgres psql <databasename> -f /usr/share/postgresql/8.4/contrib/postgis.sql
$ sudo -u postgres psql <databasename> -f /usr/share/postgresql/8.4/contrib/spatial_ref_sys.sql
Modelling a postcode Now define a simple model to store imported postcodes:
from django.contrib.gis.db import models
class Postcode(models.Model):
    name = models.CharField(max_length=8, db_index=True)
    location = models.PointField()
    objects = models.GeoManager()
  • We import models from django.contrib.gis.db instead of django.db.
  • We override the default objects manager with GeoDjango's GeoManager() so we can perform spacial queries.
Save this model in postcodes/models.py. We can now create this model in our database using:
$ ./manage syncdb
Importing postcode data We will use a management command to import the data. First, create the necessary modules with:
$ mkdir -p postcodes/management/commands
$ touch postcodes/management/commands/__init__.py
$ touch postcodes/management/__init__.py
Next, save the following code in postcodes/management/commands/import_postcodes.py:
import sys
import csv
from django.contrib.gis.geos import Point
from django.core.management.base import BaseCommand
from postcodes.models import Postcode
class Command(BaseCommand):
    def handle(self, \*args, \*\*options):
        Postcode.objects.all().delete()
        count = 0
        for row in csv.reader(sys.stdin):
            try:
                name = row[0].upper().strip().replace('  ', ' ')
                location = Point(map(float, row[13:15]))
            except ValueError:
                print "I: skipping %r" % row
                continue
            Postcode.objects.create(name=name, location=location)
            count += 1
            if count % 10000 == 0:
                print "Imported %d" % count
Finally, we can import the data using:
$ bunzip2 /path/to/uk-post-codes-2009.bz2   ./manage.py import_postcodes
This takes approximately one hour on my computer. If you plan to import the data multiple times, consult the Django documentation for faster methods of importing initial data.
Quering the postcode database We can now query our postcode database. First, let's find the postcode for Buckingham Palace:
>>> from postcodes.models import Postcode
>>> palace = Postcode.objects.get(name='SW1A 1AA')
>>> palace
<Postcode: SW1A 1AA>
Next, let's find all postcodes within 1 mile. For this, we will need to construct a tuple representing the area we want to search in.
>>> from django.contrib.gis.measure import Distance
>>> area = (palace.location, Distance(mi=1))
>>> Postcode.objects.filter(location__distance_lte=area)
[<Postcode: SE1 7BF>, <Postcode: SE1 7SG>, <Postcode: SE1 7JA>, <Postcode: SE1 7PB>,
 <Postcode: SE1 7PD>, <Postcode: SW1A 1DW>, <Postcode: SW1A 1EA>, <Postcode: SW1A 1EE>,
 <Postcode: SW1A 1EF>, <Postcode: SW1A 1EG>, '...(remaining elements truncated)...']
>>> Postcode.objects.filter(location__distance_lte=area)
7024
To find "neighbour" postcodes, we can pass a Point object to the .distance(..) method on our queryset this adds a field we can then sort on:
>>> Postcode.objects.distance(palace.location).order_by('distance')[:4]
[<Postcode: SW1A 1AA>, <Postcode: SW1E 6JP>, <Postcode: SW1E 6LA>, <Postcode: SW1E 6WG>]
Putting it all together Let's pretend we are making an app to find pizza from an arbitrary user-submitted postcode. (We will assume we have parsed the user's postcode and have found its Postcode object, but we will continue to use Buckingham Palace here.) First, define the following model in postcodes/models.py and re-run ./manage.py syncdb:
class PizzaJoint(models.Model):
    name = models.CharField(max_length=200)
    postcode = models.ForeignKey(Postcode)
    objects = models.GeoManager()
    def __unicode__(self):
         return self.name
Now, add some local pizza places:
>>> def add_pizza_joint(name, loc):
...     code = Postcode.objects.get(name=loc)
...     PizzaJoint.objects.create(name=name, postcode=code)
...
>>> add_pizza_joint('Pizza Hut', 'SW1E 6SQ')
>>> add_pizza_joint('Pizza Express', 'SW1E 5NA')
>>> add_pizza_joint('Pizza On The Park', 'SW1X 7LY')
If we were not using GeoDjango, we would have to be content with simply matching the postcode:
>>> PizzaJoint.objects.filter(postcode=palace)
[]
However, we can now find the cloest pizza places:
>>> PizzaJoint.objects.distance(palace.location,
            field_name='postcode__location').order_by('distance')
[<PizzaJoint: Pizza Hut>, <PizzaJoint: Pizza Express>, <PizzaJoint: Pizza On The Park>]
The .distance(..) method also sets a distance attribute on all the objects returned; this makes it easy to show the actual distance to the user:
>>> for x in PizzaJoint.objects.distance(palace.location,
                     field_name='postcode__location').order_by('distance'):
...    print "%s (%d metres away)" % (x, x.distance.m)
...
Pizza Hut (499 metres away)
Pizza Express (509 metres away)
Pizza On The Park (902 metres away)
For more details, see the GeoDjango documentation.

Next.

Previous.