Search Results: "nate"

25 August 2010

Kumar Appaiah: Browsing with Elinks and using hooks

Even in the modern times of AJAX, Web 2+ and what not, Elinks is one of my favourite browsers, at least for browsing some sites. The reasons are simple: it cuts the unnecessary distractions, and allows you to digest text based information rather quickly. In addition, since it doesn't display any images or flashy elements like Flash, pages load lightning fast even on poor connections, making it an excellent option.Unfortunately, some sites don't really work very well with Elinks, and for those, I do one of two things. For the ones which are really complex, such as secure login sites with too much AJAX, I just resort to one of the "standard" graphical browsers. For some others, I try to work around the problem using Elinks' rich scripting interface available in many programming languages including Perl and Ruby. I chose Lua, since it seemed simple enough to learn. I wanted to share with you some of the simple workarounds I implemented for some sites I visit often.The first quirk is with the Debian BTS, whose CSS isn't properly interpreted by Elinks (see 593804 and 593840 for details). The short story: the BTS renders like this:So, I grabbed the CSS, and made some small changes (described in the above mentioned bug reports), and got a new CSS file. I put it in /home/kumar/Kumar/bugs.css, and added the following to my ~/.elinks/hooks.lua:
-- Use an alternate CSS for the Debian BTS to work around some quirks
function debian_bts_format_html_hook (html)
   return string.gsub(html, "/css/bugs.css", 'file:///home/kumar/Kumar/bugs.css')
end
function pre_format_html_hook (url, html)
   if sstrfind (url, "bugs.debian.org") then
      return blonnet_format_html_hook(html)
   end
   return nil
end
and with it, the BTS renders like this:Much more readable!The second simple example, for which I won't include images, is the Mobile Twitter interface. The mobile Twitter interface is very convenient and functional in Elinks; however, by default, the text font seems to be a light grey, which is not discernible in the bright background. A default user.css file would be a possible solution, but I have not decided whether to go for it just yet. So, for the Twitter site, I add a function like this:
-- Make the status messages black on Mobile Twitter
function mobile_twitter_format_hook(html)
   return string.gsub(html, ".status.- ", '.status  word-wrap: break-word; color : #000; ')
end
And alter the actual hook function to:
function pre_format_html_hook (url, html)
   if sstrfind (url, "bugs.debian.org") then
      return debian_bts_format_html_hook(html)
   elseif sstrfind (url, "mobile.twitter.com") then
      return mobile_twitter_format_hook(html)
   end
   return nil
end
Finally, a more complicated example. I like reading the Indian business newspaper the Hindu Business Line. However, if you try browsing to the site with Elinks, you'll notice that, barring the top hyperlinks in each sectcion, the links on the left are not "linked" at all.In addition, there are some other irritants which are easy to remove, such as the navigation bar on the top, and the text colour being too light for my taste etc. So, I write the following Lua blurb. Why I write this is left for the interested readers to figure out, though it shouldn't be surprising if you open the page. The workaround for the missing links is interesting, though; it is because the website authors have put a hyperlink (<a>) tag before a list item (<li>) tag, and Elinks seems to not like that too much. So, I just swap the two around, and Elinks is happy.
-- Reformat the Business Line website
function blonnet_format_html_hook (html)
   html = string.gsub(html, "<body ", '<body text="#000000"') -- I like the text in black on the white background
   html = string.gsub(html, "<[Ff][Oo][Nn][Tt] class=leftnavi color=brown>.-</[Ff][Oo][Nn][Tt]>", '') -- An ugly hack to remove the navigation bar
   html = string.gsub(html, "<TD align=left vAlign=top width=\"500\"><IMG.-</FONT><BR>", '', 1) -- Remove description
   html = string.gsub(html, "<A class=groupnavi href=\"http://epaper.thehindubusinessline.com/\">.-</FONT><BR>", '', 1) -- Remove other group links
   html = string.gsub(html, "(<[Aa] class=navi.-)<li>(.-</[Aa]>)", "<li>%1%2</li>") -- Fix the erroneous section sidebar
   return html
end
function pre_format_html_hook (url, html)
   if sstrfind (url, "blonnet.com") or sstrfind(url, "thehindubusinessline.com") then
      return blonnet_format_html_hook(html)
   elseif sstrfind (url, "bugs.debian.org") then
      return debian_bts_format_html_hook(html)
   elseif sstrfind (url, "mobile.twitter.com") then
      return mobile_twitter_format_hook(html)
   end
   return nil
end
And here's the old and new rendering:

Christian Perrier: [life nolife] Debconf 10 was...

...awesome. OK, I'm writing this while I'm still in USA, but there are so many things to say about these weeks that I can't write them in only one blog post. And, still, this one will be quite long as it will talk about hacking, running and sightseeing...:) Let's start about hacking: after all, this is the first reason for being there in US, isn't it? I cam to DebConf with a very long TODO list and, for the first time in seven DebConfs, I'm pretty happy with what I achieved from it: As one can see, a lot of planned work happened while I still could maintain the usual flow of recurrent work with localization (Smith reviews, l10n NMUs). Some asked me why I didn't propose l10n sessions this year. Indeed, I wasn't feeling I could sustain animating them and I had no clear idea about which topic I could bring to be discussed. Last year, these sessions slightly killed my free time and I wanted to keep some this year for "impromptu" things. I didn't attend many talks, sorry for the speakers. The most I attended were during Debian Day, which I found highlyinteresting and motivating, just like Eben Moglen's talk. Marga's talk was also one I wanted to attend, though I regreted that things went mostly out of control during the talk (too many comments from the audience to allow Marga pushing her important points). As usual, I invested a big part of my time in "social" activities, the most proeminent being of course the Cheese and Wine party, which turned ut to be a great success. The help of my son Jean-Baptiste and the tremendous support of Michelle Lynn Hall helped a lot, though I still regret that we screwed about accessibility. I also ran a lot..:-)..that may be counted as social activities as I organized several group runs. The one I'm proud of has been participating to a local race, namely the Van Cortland Track Club Summer Series of cross-country running, in Bronx. We went there with no less than 10 DebConf participants and 1 kilt (hey, Luca!). All of us completed the race (that had 170 runners for 5 kilometers) and No l K the even finished 17th scratch and 2nd in his age/gender category. Besides that, we had a great run/sightseeing to Georges Washington Bridge (that links New Jersey and Harlem and offers an unusual view of Manhattan "from behind"). All this with a 17km run. We also ran several times in Central Park, and No l and me happened to go to Coney Island for the Day Trip by doing half of the trip by running (all around Manhattan and over the Broolyn Bridge), for about 20km. Then we "showered" in the Atlantic Ocean....:). At the end of DebConf, I think that I had my record broken with 112km run in 10 days and only one day *without* running. What about sightseeing? Well, this blog post is too long and we reach the end of Interstate-90, close to Albany, so that will be for an upcoming blog post. Aug 25th update: back home, so now I can publish this blog post...

24 August 2010

Richard Darst: The DebConf selection process

Selecting a DebConf location hard, and can certainly get intense at times. This is an area where the DebConf team and the Debian project can work to improve our processes. There is a wiki page to collect our thoughts in preparation for a discussion later, as planned during the DebConf&Debian BOF.
First off, there is a time at DebConfN-2 for interested localteams to describe their rough plans. These are videoed, and you can view the events online if you would like: for dc11 at dc9, for dc12 at dc10 .There has been a nominal deadline of December 31, 20XX-2 for submitting a bid (not finalizing), but this has been bent in the past.While teams are preparing their bids, they should look at the location checklist. This summarize what past organizers know to look for in a location. There are questions sorted by category and priority.To organize their information so that people can understand, teams prepare bid documents on the wiki (DC10, DC11, here are the winning bid documents: DC10-New York City, DC11-Banja Luka). These start off by just answering all of the questions in the location checklist, but then get a lot of work to make the pages clear and complete. Anyone can browse the pages and ask questions of the teams anytime.There are several mini-meetings where teams describe their progress, and organizers and other interested people can ask questions. In these meetings, the channel is not moderated. Questions tend to be about the location checklist criteria related things.In February/March, there is a big selection meeting. This is the only DebConf meeting where the channel is moderated and only voiced people can speak. The voiced people historically are two moderators who keep things moving, two other past DebConf organizers to ask questions and relay questions from the audience discussion channel. (These facilitators have been self-selected in the past, usually in the minutes leading up to the meeting.) Then, there are two people per location whom are voiced. There is a audience channel for everyone else to discuss, and the moderators/helpers will look there for questions and relay them to the main channel. There have been unvoiced Q&A periods and periods which are designated for relaying questions from the discussion channel.These meetings are logged (dc10 selection, dc11 selection)Teams are rated based on the priority list. There has been a simple point list the last two years, -1 to +1 for each point on that list. For each item, there are debates on how many points each location should get. Points are added up, and a result appears. In the past, this has been a simple arithmetic sum, so all items are weighted the same (despite nominally having different weights). In the case of a tie or close call, there is more discussion. For DC10, there was a tie, and NYC won after opening the meeting to the audience and getting their opinions. I didn't pay enough attention to the DC11 selection process to be able to say how that close call was resolved.While this isn't really part of the selection process, local teams can give presentations at DebConfN-1 about their location and encourage attendance: dc10 at dc9, dc11 at dc10
So, there it is. Overall, there is a lot of openness in the process, but the exact rating schemes in the decision meeting could use discussion. The criteria are established, but can be refined each year based on what we learn.If you would like to comment on how to improve this in the future, please use the selection process wiki page (first person create it). We can follow up with a list discussion after our ideas have been fleshed out.Tags: debconf

22 August 2010

Gregor Herrmann: RC bugs 2010/27 - 2010/33

long time no RC bug posting. in july I was at the Munich BSP, & during DebCamp/DebConf I had the excuse of being one of the judges of the RCBC (which was the only reason for the freeze, I'm sure :)).

to keep up with the tradition of posting about my other RC bug activities, here comes a short list. as you can see most of the work was done in or around the pkg-perl group; luckily I often only have to upload packages prepared by ansgar, carnil, & others. I hope we as a project can keep up the momentum to fix the remaining RC bugs, & see a release in a not so far future.

Christine Spang: DebConf 10 postmortem and SD talk followup

DebConf launched with a bang the day I arrived by bike I was up until 3am meeting and greeting in the basement lounge of the Carmen Columbia dormitory, where I was staying. No idea how I managed to be so awake for that. The rest of the week alternated between hacking like crazy on code for my talk and spending a lot of time socializing with Debian folks new and old. For the day trip to Coney Island, I joined the dkg-led bike expedition which ended up running to nearly 30 miles, which was a bit more than expected. The fact that this was all in actual dense city really drove home the scale difference between Boston and New York (I'd never been to NYC before this). We took several breaks to lounge around and eat and drink, so it took quite a long time even given the distance. I hadn't planned on seeing the baseball game that was a part of the trip, but I ended up going anyway and it turns out that a bunch of geeks at a minor league game is actually quite a lot of fun! I hope someone else will put some pictures from the bike ride and game online soon, since I didn't really take any myself. This was the first DebConf where I gave a talk, which resulted in me skipping almost all of the other talks, because my talk was on the last day and I reaaally wasn't ready at the start of the conference due to the rest of life being pretty crazy this summer. I missed some things I would have liked to see because of this, but ultimately I think it was worth it. The good news is: it went well! I was nervous until I actually started talking (never given a talk at a conference before), and then it was fine. If you missed it, the talk video is on the web in low and high quality; slides are here. The audience was great there were excellent questions and people were excited and interested in the project. I couldn't have asked for a better reception. After the talk finished I spent some time aisle-chatting with some folks, and totally failed to recognize Joey despite having met him before, because he'd shaved off his hair. DebConf was, like usual, both inspiring and exhausting. I haven't managed to follow up on much that happened during the conference yet. I definitely plan to do so, though, now that real life is calming down again. I'd hate to waste the post-conference buzz about SD. My todo list includes:

Russell Coker: Telling People How to Vote

Yesterday I handed out how to vote (HTV) cards for the Australian Greens. The experience was very different to the one I had when I handed out cards for the Greens in the Victorian state election in 2006 [1]. The Labor party (ALP) hadn t spread any gross lies about the Greens and there were no representatives from the insane parties (Family First and Citizens Electoral Council/Commission (CEC)). So we didn t have any arguments among the people handing out the HTV cards. The atmosphere among the volunteers that were present was a good match for some ideals of a sporting contest. Everyone wanted their own team to win but acted in a sporting manner. When no voters were around we had some friendly conversations. One thing that was interesting to note was the significant number of families where the parents in their 40s deliberately snubbed me while their children in the 18-22 age range took the Greens cards. It seemed that for families with adult children there were two likely voting patterns, one was children voting Green and parents not liking it, and the other was when the entire family voted informal (when someone refuses all offers of HTV cards it s a safe bet that they will cast an informal vote). In Australia submitting a vote card is mandatory but making it legible and formal is optional. The last report I heard suggested that about 5% of the total votes were informal. This seems to be strong evidence showing that civics lessons are needed in high school. Also there were a disturbing number of people who stated that they didn t know which party to vote for when they were collecting the HTV cards. A HTV card has one or two sentences about the party and there are almost no requirements for truth in such statements. Anyone who votes according to such brief summaries of the parties is quiet unlikely to end up casting a vote that gives the result that they desire. The result of the election is a significant swing to the Greens, more senators and the first Green MP! For the lower house it seems that Labor will have great difficulty in forming government even when in a coalition with the Greens and some independents. It seems unlikely that the Liberal party could ever make a deal with the Greens, the Liberal position on almost every significant issue contradicts that of the Green policy, but there is a chance of a Liberal coalition with independent MPs. In any case it seems that the Greens will have the balance of power in the senate so the excesses of the Howard government can t be repeated. If you like the nail-biting drama of watching several columns of figures slowly changing over the course of several days then you would love watching the analysis of this election! Whatever coalition government is created is not likely to be stable and we can probably expect another election in a year or two.

19 August 2010

Steve Kemp: You are in a maze of twisty little passages, all alike

Debian package building scripts consist of several inter-related tools, which are documented well with manpages, but not in the code itself. Should you have some free time I urge you to take a peak at the source code of dpkg-buildpackage, debsign, and debuild. They work. They work well, but they're not shining examples of clear code are they? Given that they work, and that they are complex I'm not sure I'm interested in improving things, but I do have a personal desire to run something like: 
debuild -sa --output=/tmp/results/
What would that do? Instead of placing the resulting package in the parent directory, which is what we have by default, it would place them in the named directory. However looking over the code I see that there are too many places where "../$file" is hardwired. e.g. Take a look at dpkg-buildpackage and you see: 
my $chg = "../$pva.changes";
..
open OUT, '>', $chg or syserr(_g('write changes file'));
If you change that to the simple code below it works - but suddenly debsign fails to find the file: 
my $dir = $ENV 'OUTPUT'    "..";
my $chg = $dir . "/" . $pva . ".changes";
Ugh. I guess I can come up with some hack to index files in the parent, run a build process, and move the only new files into place. But that's sleazy at best, and still doesn't solve the problem that we have tools which we rely upon which are .. unfriendly to changes.
I thought I could handle this via: 
debuild \
 --post-dpkg-buildpackage-hook="move-changes-contents /tmp/output/ ../%p_%v_ dpkg-architecture -qDEB_HOST_ARCH .changes" \
 --no-tgz-check -sa -us -uc
(Here "move-changes-contents" would read the given .changes file and move both it and the contents listed in it to the specified directory.) Unfortunately %v corresponds to the package version - which isn't the same as the version in the .changes filename. (i.e. epochs are stripped out.)
ObQuote: I am the servant of the power behind the Nothing - The Neverending Story.

18 August 2010

Gerfried Fuchs: Jerboa

A while ago I was with my SO at a concert of a band I might cover at a later point. This time I want to cover the support act of that concert: Jerboa. It was a bit weird to see a single person on stage turning some knobs, but he quickly caught both our attention and heart. And he was a charm to talk with afterwards when we bought his CD. He's definitely worth your attention, and if you are able to see him live take the chance for it. Actually I would have loved to include What if in the songs but unfortunately that one wasn't available so I had to look for a replacement for it. Enjoy!

/music permanent link Comments: 2 Flattr this

17 August 2010

Russ Allbery: Policy and time management

After reading Do It Tomorrow, I was inspired to try some of the techniques that Mark Forster proposes. I commented in the review about the closed list system, which I'm still mostly using with some success. But I also started playing with the initiative slot. In his system, the initiative slot is first thing in the morning each (eligible) morning, which is used for a special project that you want to make forward progress on. There are several criteria for choosing a task for the initiative slot. One of the strongest is dealing with a backlog. Since I'd been feeling guilty about the backlog of Debian Policy work that had built up, that seemed like a good choice. I picked up Debian Policy backlog clearing as my initiative starting at the beginning of June after the AFS and Kerberos Best Practices Workshop, with a goal of getting the open bug count below 100. This has been very instructive, both from a time management perspective and from a Debian Policy perspective. Here are a few things that I learned on the Policy side: Things I learned on the time management side: I stopped using the initiative slot while travelling and haven't gotten back into the hang of regular time management since I've gotten back. I haven't yet decided whether to leave Policy work in the initiative slot for right now but change the goal, or whether to swap it out for something else. I'd like to keep momentum, but I'd also like to start using that slot for other things. Still pondering.

15 August 2010

Tim Retout: SmoothWall Express on Debian

SmoothWall Express is a GNU/Linux distribution geared towards firewalling, with an installer, a web interface, and some common software like squid that can be useful when running a small business router. It is theoretically the basis for the corporate products of SmoothWall Ltd., who happen to employ me; but all opinions here are my own, and I'm not speaking for them. Unfortunately, the SmoothWall Express kernel is somewhat "stable", which leads to problems installing the distro on modern hardware. There is a new version of Express in the works, but I'm afraid SmoothWall Ltd. currently has a bit of a "code dump" mentality with respect to delivering updates to their community, because they don't recall seeing any significant contributions from outsiders. At DebConf I created proof-of-concept Debian packages of two components of Express 3.0: the swe3 web interface, and the smoothd daemon which executes privileged commands. Currently these can show a basic web interface; some of the less complicated bits will even run, and I can shut down my laptop using the "shutdown" button via smoothd. (Note that I still need to add boring stuff like debian/copyright files, but I plan to release these as soon as I can.) In the near future, hopefully I can implement some of the more important features (like, er, firewalling), and add some other components like the traffic shaping and IM filtering daemon. I'm working towards a demonstration Debian Pure Blend that can show off some of the advantages of working with a third-party distribution as a base. If anyone would like to help me... send me patches. :) I expect I'll be blogging my progress occasionally.

12 August 2010

Kartik Mistry: Report from MiniDebConf India 2010

* So, here is late report! Sometime back Praveen started discussion at DebianIndia mailing list about having MiniDebConf kind of conference here in India. MiniDebConfs are generally focused on development rather than at introduction to Debian . But, since this was our first attempt, out focus was more on to introduce Debian to student community and after writing this after 5 days I realize that we ve done good job! Many thanks to Joerg Jaspert for setting up Mini website: http://in2010.mini.debconf.org/ Praveen got bunch of sponsors, many thanks to CDAC and other sponsors for making event happen with testy food and tea that kept me awake. Now what really happened at Pune? My flight was nice. I went to guest house of COEP (the host college, 2nd old engineering college in India) and took some rest and called Praveen. He was already at computer lab and setting up Virtualbox and other things. We had two places hall and lab. Unfortunately, both were at different buildings and subway between them :) I discussed with couple of students and we went to dinner at late night. Pro. Abhijit who was kindly give permission to use lab and college hall was another source of energy. He is handling many FOSS events in college and probably only faculty who encourage students to use and learn FOSS. The Day 1 started with long lines of registration process. As we had around 200 registration, turn out of students were 100% on first day. Hall was almost full. Dr. Shashikumar from CDAC gave nice lecture about contribution which was well received. Praveen and Onkar then took the stage and gave nice introduction about Debian. I met Onkar first time. We had really nice discussion ranging from becoming Ubuntu MOTU to his Powerbook s adapter. After that Vikram took lively session on i18n and l10n in Debian and I took boring session on Contributing to Debian . Yet another boring session from me was Developer Toolbox . Packaging workshop by me at hall made many faces to yawn heavily while Onkar and Praveen handled it nicely at Lab. Main reason for yawning was Wifi and my Macbook s capacity to deal with ndiswrapper and crappy broadcom s drivers. (Note: I should get nice machine.) I met Sham first time. I met Kushal once again and felt energetic again! Day 2 As we expected number was very low compared to Day 1. So, we closed hall and moved entire stuffs to Lab only. Vikram took again nice session on Debian Edu . It was very interactive and I should learn something from his act of presentation. Another nice thing from Day 2 was discussion with BOSS Linux team. Hold on. Lets keep some old woes aside. They are basically doing what their bosses are telling. Developers were clueless about Sending patches to upstream. Their bugzilla had Zorro bug reported. I made promise to follow-up on this and set-up patches from boss at somewhere depending upon their response. That mail is my draft let me finish it by weekend. Pavi and others took video from session. It will be nice if someone will post it somewhere :) We then continued on discussing bugs and packaging area. It was amazing to see Praveen become DM during conference and also to see: upstream, maintainer and sponsor in one room. I really had nice food at dinners. Mango Mastani was nice. Beer was nice. How can I forget discussion with Ramki, Ninad, Pro.Abhijit, Praveen, Pavi and others after dinner? So lets work for MiniDebConf 2011 now!

Olivier Berger: My report after the BOF on distributed bugtrackers at DebConf

Here are my notes about the very insteresting BOF on distributed bugtrackers held at DebConf This is just some elements of report (some taken from the gobby edits, and written down by me after watching the video (unfortunately missing 2 mins of sound in the beginning of the video stream/recording)). This is not a full report of all the discussions, as some discussed elements weren t of interest IMHO. You ll find here many ideas of mine, links and pointers (resulting of our work in the context of the Helios project on similar topics), so this is in no way a report of what was said exactly during the BOF. Aspects of interest: Note that these topics may in principle be discussed on the Distributed bug trackers mailing list: http://kitenet.net/cgi-bin/mailman/listinfo/dist-bugs Tools / implementation ideas : Storage of repositories / databases of remote bugs : About such storage, and the RDF store option, here s a prototype of ours that exposes UDD s bug facts through a SPARQL interface, allowing to publish bug facts as RDF (setup in the frame of the Helios project). It includes both Debian and Ubuntu bugs, and allows interlinking them in a standard way (with Semantically described links in RDF). See http://testforge.int-evry.fr/d2r-server/resource/spackage/evolution for an example. The problem is not how to distribute (distributed database systems exist), but the semantics of bug properties, and the different implementations of interfaces of different tools : which standard to implement, or how many converters are necessary if no standard ? We believe that navigation/referencing/tracking of external bugs would be much easier if bugs were LinkedData citizens, i.e. they would be exposed with (standard) formats like Semantic Web ontologies served with Semantic Web standards (like RDF). The worst case is currently to have to perform web scraping on bugtracker interfaces. How to avoiding scraping ? A solution could be to publish RDFa / microformats using common standard semantics, directly from the bugtracker pages at bugs URLs (see my wishlist for debbugs on that subject : 590931: Would be great to integrate RDFa metadata into debbugs pages). Now some pointers to historical details on the same subject and Debian (old posts by Lars Wirzenius) : Overall, the discussion was quite interesting, but didn t reach any sensitive progress regarding the state of the art. Much needs to be worked on, mainly on standardizing APIs, semantic descriptions of bugs, and implementation in tools like SD for instance. Stay tuned for more progress.

Gerfried Fuchs: Why new Design isn't deployed yet

This blog post is actually a response to a talk that was given at the debconf last week. It was marga's talk about Making Debian Rule, again. Now that the beta versions of the talks are available for download I'm able to proper quote what I want to respond to: Is there a reason why we haven't yet updated the website?, perhaps someone can answer. I feel the need to respond to this as I am the main person driving that effort. Unfortunately the question was asked in a talk with no-one involved in that work around instead of addressing those people directly, and the question wasn't brought to my attention before the talk so that I could have provided an answer to offer in the presentation already. Also, as disclaimer, this is my personal story for it and doesn't need to get shared by the other people involved. I hope it can be seen as an answer anyway. As a little of background, Kalle's proposal went a long path before even I found out about it. I was simply in awe in several ways about it because it wasn't just a great mockup of the page (which it actually is, IMHO) but was accompanied by thoughts about not only the main site but also about several sub sites. Also, it was accompanied by patches, so it was seemingly ready to get deployed right ahead. I started working on it, got accused of being discouraging while doing that, got told that it would require a proper vote for a decision and can't happen just as, but those are just the Debian way of communication. I nevertheless did set up several test sites for some over the time to play with it. Some did work out better (like the git one, though this one came last and Kalle improved the CSS over time a fair bit), some weren't ready yet (like wiki, missing e.g. the coloring for the version diff, or the packages one where the separation between the different sections isn't that visible yet). Time passed, other things demanded their attention too. Like the thing where our system administrators did send me a request along about a new www-master server that needs to get set up from scratch. Given that the documentation about the required packages for the website was lacking a fair bit (to say the least) it did require a lot of attention, especially when doing things from scratch the wish to document the dependencies properly is just natural. There are still some issues with that as can e.g. be seen on this page in the empty table at the bottom. The same issue also appears on my testsite for the new design. The last part though is currently the biggest blocking issue for both efforts. There is no way to move forward with either without having that addressed. Simon Paillard did a great job on helping the move along so far, keeping the thread about the server move requirements that I started on the debian-www mailinglist updated with information about what's still pending. Unfortunately Kalle, Simon and me are also facing some private time constraints (amongst other duties that require attention) which weren't helping to move forward at a bigger pace. Unfortunately not very many people actually has shown interest in helping out neither, after all it's easier to go and ask questions about why it's not done yet to the completely wrong audience. My plan is to switch many sites at the same time instead of one after the other simply for the effect but also for less distraction for our users, including less people repeatedly pestering about when this or that site gets done, too. At least we need a final discussion on what should go into the header links and what should be at the bottom before a switch can be made anyway. And for some there is still the opinion that such a switch is nothing that we are rightfully allowed to decide without a vote, so that's also part of the reason to not just deploy it. I hope this answers the question to some extend and like said, if only I would had been addressed about this before the talk was given an answer to it would had been ready for presentation right ahead instead of having a pretty terse statement getting relayed through IRC (thanks, Yoe!) because I was fortunate enough to be around and watching the talk at that time. Feel free to discuss it further or offer your help on debian-www, especially if you are familiar with working on CSS files.

/debian permanent link Comments: 3 Flattr this

David Moreno: DebConf 10

I was in New York for three full weeks, before, during and after DebConf 10 took place. Unfortunately, I wasn't able to attend talks or go hang out with folks as much as I would've wanted. Most of the time I was busy at the office with a new product launch we have been preparing for the last few months. However, I was able to come and say hi to a few people, attend the C&W party and talk with friends and ran into. I was, however, following all conversations on IRC so I was pretty informed of all of what was going on. Some random thoughts: DebConf 10 rocked and I appreciate everybody's participation in it. See you in Banja Luka, next year!

10 August 2010

Tim Retout: Sunny Southampton

On my last night in New York, I didn't sleep much. At 6am, I said farewell to Central Park by running round the reservoir, which I hadn't yet done. There was a very nice red sunrise to be seen from the west side. Unfortunately I didn't sleep much on the flight home either. The British accents sounded quite unusual when we landed in Heathrow, and it was quite confusing not being able to find a Starbucks. Once I was back home, I crashed, and woke up at 10pm. I spent last night clearing the pkg-perl review queue - gregoa is taking a short break after DebConf. Then I went running at sunrise again. This is quite a different experience to Central Park - first, you have to run 2.5km just to get to Southampton Common, and secondly it is raining quite heavily. I dug out some winter gear that had turned out to be completely inappropriate for New York.

Gunnar Wolf: Back home, back from DebConf And, hopefully, cleaner than ever!

So, DebConf time is over once again. The two weeks worth of fifty weeks waiting are left behind once again, and it's back to get back to normal. DebConf was great Yes, it always is, and that's what we are all saying, but hey - Seriously! Being in the same building than 300 crazed developers is always fun, and it's always better than last year's fun. A good highlight this year is that, given the number of Free Software and Free Culture groups that exist in USA's north-eastern coast, we had the opportunity to join a large crowd which has never been part of DebConf. Also, I must agree that the USA bid for DebConf was aiming to attract as many Debian people (developers, maintainers, or just happy users) which had not yet been to a DebConf before as possible. And it was a great success! I finally met several people I have long read in the mailing lists, in blogs or in IRC. A much higher proportion than usual, I'd venture to say. Another interesting phenomenon /methinks is that this year's DebCamp started much more staffed than usual: I arrived on the first day, Sunday 25, and there were ~40 people there already; I don't have the actual numbers, but we quickly grew, and the number started to stabilize past mid-week, only to (sharply) rise in the weekend, in time for DebianDay and DebConf start. Great time! But, they say, nobody can go to the USA without buying some sweet toys, right? Well, being the proud owner of six very hairy cats, I have thought into entering the looming and weaving industry... But cat hair, while abundant, I have heard is untreadable... Maybe due to the indisciplined, natural and independent personality of the cats (catonality should I say?)... So I had two choices: Clean up my home quite often, or live in a literally hairy mess. Enter choice #3: The Roomba! I had been waiting to buy this thing for several years, as they refuse to send to Mexico or charge Mexican cards. So, I walked across Manhattan and got my very own robot cleaner! For my further surprise, although I have not yet tried it (I don't even have a suitable cable yet), I found this: Yay, the Roomba is actually hackable (via a 7 pin miniDIN serial port)! Wikipedia says that:
Roomba comes with a Mini-DIN TTL serial interface, which is incompatible with standard PC/Mac serial ports and cables, both electrically and physically. However, third-party adapters are available to access the Roomba's computer via Bluetooth, USB, or RS-232 (PC/Mac serial). New, 500-series, and 410/420 series Roombas upgraded with the OSMO hacker device allow the user to monitor Roomba's many sensors and modify its behavior. The Roomba Open Interface (formerly "Roomba Serial Command Interface") API allows programmers and roboticists to create their own enhancements to Roomba. ( )
My first impressions? Well, the Roomba lazily charged its battery throughout the day today, and was hungry and ready when I arrived home. It is a but louder than what I expected, and of course my cats were not thrilled by the presence of a eighth animated and apparently sentient being at home. Their initial reaction was of course to be verrry alert of the thing. Twelve eyes were constantly pointing at the Roomba while mine alternated between them. As they measured the thing's speed and (I guess) inferred its movement patterns, they started escaping upstairs A flat, round thing with no legs to be seen will not likely be able to climb the stairs. And they were completely right. At first, only Chupchic remained downstairs. After a bit, I went up to show them the Roomba didn't jump on us to eat our brains, and after a bit, Santa and Macusa joined. The Roomba roombed for maybe 90 minutes (this space is large, and decided it was enough... And slowly, the rest of them started coming down. I would not say Roomba's cleaning is perfect, of course. Its room discovery algorithm is funny, and it even seems it's based on the mere chance of covering most (never all) of the space it has to clean. I had, of course, not fully studied it (after a single run, how could I?). It does make a honestly good attempt at cleaning under coaches, chairs and tables. It collected a fair amount of dust (on a house that seemed quite clean to me, I cannot imagine what would happen on a messy one). I have not yet played with the virtual walls (infrared transmitters which limit rooms as if a door was closed), but given the size of this house (and that I don't want it to clean around the cats' designated bathroom area), I guess I will end up using them regularly. During DebConf, I heard one bad (stupid useless noisy thing) and two very good (it has radically changed my life) comments on the Roomba. I hope to shift the balance towards 3/4 and not towards 2/2! Anyways... Thanks to each and every one of you. DebConf is great. Always great. Always a success. I cannot even thank specific teams. Debian Rules, and DebConf Rocks!

9 August 2010

Steve Kemp: OepnBSD rocks. Until it doesn't.

Recently I've been jumping upon the LDAP bandwagon, with one of my aims to consolidate a lot of different login systems. Configuring Linux, Apache, OpenVPN and similar things to authenticate against an LDAP server was almost painless. Unfortunately OpenBSD is being a bit more painful, primarily because it doesn't use PAM. Instead you have two choices: Neither solution is particularly pleasant, but the former is marginally less effort. The downside? I still have to run "adduser" to add the user to the system - which makes me think "why did I bother in the first place?" Otherwise I spent the tail end of last week in York, taking pictures of ducks, geese, the city walls and similar things of fun. ObQuote: "Well, well I see we have visitors... " - Hot Fuzz

8 August 2010

Michael Banck: 8 Aug 2010

So, DebConf is over and it was a blast. I wanted to blog about my talks for a couple of days, but the conference was so great that I did not get around to it until now. The unique thing about this year's conference were the outstanding contributions by non-Debian FLOSS people from the east coast. I am really glad the organizers decided to reach out to the communi ty and take this opportunity when a lot of great minds were just a couple of hours away. Also, discussing and hanging out with the local team people was so much fun and interesting that it was wor th the visit alone. The venue was just perfect, the dorms were on campus, the cafeteria had an all-you-can-eat buffet, everything was in short walking distance and the Columbia campus is beautiful. I would have liked to go to a couple more places in the evenings, but hanging out in the Carman basement lounge with awesome people was just as good. A big thanks to Richard Darst, Biella, Micah and the rest of the crew. The Debian GNU/Hurd talk went quite well, I was pleasantly surprised so many people made it to the Davis auditorium. I wanted to do the presentation on Debian GNU/Hurd (and I had it working before the talk), but as my notebook has a different resolution than the projector, I decided to play it safe and just show a d-i run in qemu. Nevertheless, Jeremie's wo rk on debian-installer is impressive, I got it installed on my ThinkPad without a problem (using qemu) and it automatically installed and setup grub2. Unfortunately, grub2 seems to be having issues when booting my notebook natively, but I got it to work with grub-legacy, including X and evince. There were quite a few comments and I had interesting conversations afterwards with a couple of people. It is a shame Emilio Pozuelo Monfort (pochu) could not make it to DebConf to give the talk himself, he did lots of great work on porting packages and fixing the Hurd and glibc for various testsuites over the last couple of months. My other talk about GOsa and FAI was a bit rougher, I scrambled to get FAI integration in GOsa to work based on Mark Pavlichuk'sinstallation scripts which I fixed up over the last couple of weeks to the point where one can install a client using the FAI simple demo classes (which I ported to GOsa's FAI LDAP). There were some problems with the demonstration during the talk and I guess it was a tough audience for a web-based admin tool but I hopefully got my point across that we should salvage this work done for the city of Munich. Indeed, I had great discussions with Andreas Mundt from debian-edu afterwards who posted a summary and call for discussion to the debian-edu mailing list.

5 August 2010

Matt Zimmerman: DebConf 10: Day 3

How We Can Be the Silver Lining of the Cloud (Eben Moglen) Eben s talk was on the same topic as his Internet Society talk in February, which I had downloaded and watched some time ago. He challenges the free software community to develop the software to power the freedom box , a small, efficient and inexpensive personal server. Such a system would put users more in control of their online lives, give them better protection for it under the law, and provide a platform for many new federated services. It sounds like a very interesting project, which I d like to write more about. Statistical Machine Learning Analysis of Debian Mailing Lists (Hanna Wallach) Hanna is bringing together her interests in machine learning and free software by using machine learning techniques to analyze of publicly available data from free software communities. In doing so, she hopes to develop tools for studying the patterns of collaboration, innovation and other behavior in these communities. Her methodology uses statistical topic models, which infer the topic of a document based on the occurrence of topical words, to group Debian mailing list posts by topic. Her example analyzed posts from the debian-project and debian-women mailing lists, inferring a set of topics and categorizing all of the posts according to which topic(s) were represented in them. Using this data, she could plot over time the frequency of discussion of each topic, which revealed interesting patterns. The audience quickly zoned in on practical applications for things like flamewar and troll detection. Debian Derivatives BoF (Matt Zimmerman) I organized this discussion session to share perspectives on Debian derivatives, in particular how we can improve cooperation between derivatives and Debian itself. The room was a bit hard to find, so attendance was relatively small, but this turned out to be a plus. With a smaller group, we were able to get acquainted with each other, and everyone participated. Unsurprisingly, there were many more representatives from Ubuntu than other derivatives, and I was concerned that Ubuntu would dominate the discussion. It did, but I tried to draw out perspectives from other derivatives where possible. On the whole, the tone was positive and constructive. This may be due in part to people self-selecting for the BoF, but I think there is a lot of genuine goodwill between Debian and Ubuntu. Stefano Zacchiroli took notes in Gobby during the session, which I expect he will post somewhere public when he has a chance.

4 August 2010

Russell Coker: Why Clusters Usually Don t Work

It s widely regarded that to solve reliability problems you can just install a cluster. It s quite obvious that if instead of having one system of a particular type you have multiple systems of that type and a cluster configured such that broken systems aren t used then reliability will increase. Also in the case of routine maintenance a cluster configuration can allow one system to be maintained in a serious way (EG being rebooted for a kernel or BIOS upgrade) without interrupting service (apart from a very brief interruption that may be needed for resource failover). But there are some significant obstacles in the path of getting a good cluster going. Buying Suitable Hardware If you only have a single server that is doing something important and you have some budget for doing things properly then you really must do everything possible to keep it going. You need RAID storage with hot-swap disks, hot-swap redundant PSUs, and redundant ethernet cables bonded together. But if you have redundant servers then the requirement for making one server reliable is slightly reduced. Hardware is getting cheaper all the time, a Dell R300 1RU server configured with redundant hot-plug PSUs, two 250G hot-plug SATA disks in a RAID-1 array, 2G of RAM, and a dual-core Xeon Pro E3113 3.0GHz CPU apparently costs just under $2,800AU (when using Google Chrome I couldn t add some necessary jumper cables to the list so I couldn t determine the exact price). So a cluster of two of them would cost about $5,600 just for the servers. But a Dell R200 1RU server with no redundant PSUs, a single 250G SATA disk, 2G of RAM, and a Core 2 Duo E7400 2.8GHz CPU costs only $1,048.99AU. So if a low end server is required then you could buy two R200 servers that have no redundancy built in which cost less than a single server that has hardware RAID and redundant PSUs. Those two servers have different sets of CPU options and probably other differences in the technical specs, but for many applications they will probably both provide more than adequate performance. Using a server that doesn t even have RAID is a bad idea, a minimal RAID configuration is a software RAID-1 array which only requires an extra disk per server. That takes the price of a Dell R200 to $1,203. So it seems that two low-end 1RU servers from Dell that have minimal redundancy features will be cheaper than a single 1RU server that has the full set of features. If you want to serve static content then that s all you need, and a cluster can save you money on hardware! Of course we can debate whether any cluster node should be missing redundant hot-plug PSUs and disks. But that s not an issue I want to address in this post. Also serving static content is the simplest form of cluster, if you have a cluster for running a database server then you will need a dual-attached RAID array which will make things start to get expensive (or software for replicating the data over the network which is difficult to configure and may be expensive), so while a trivial cluster may not cost any extra money a real-world cluster deployment is likely to add significant expense. My observation is that most people who implement clusters tend to have problems getting budget for decent hardware. When you have redundancy via the cluster you can tolerate slightly less expected uptime from the individual servers. While we can debate about whether a cluster member should have redundant PSUs and other expensive features it does seem that using a cheap desktop system as a cluster node is a bad idea. Unfortunately some managers think that a cluster solves the reliability problem and therefore you can just use recycled desktop systems as cluster nodes, this doesn t give a good result. Even if it is agreed that server class hardware is used for all servers so features such as ECC RAM are used you will still have problems if someone decides to use different hardware specs for each of the cluster nodes. Testing a Cluster Testing a non-clustered server or some servers that use a load-balancing device at the front-end isn t that difficult in concept. Sure you have lots of use cases and exception conditions to test, but they are all mostly straight-through tests. With a cluster you need to test node failover at unexpected times. When a node is regarded as having an inconsistent state (which can mean that one service it runs could not be cleanly shutdown when it was due to be migrated) it will need to be rebooted which is sometimes known as a STONITH. A STONITH event usually involves something like IPMI to cut the power or a command such as reboot -nf , this loses cached data and can cause serious problems for any application which doesn t call fsync() as often as it should. It seems likely that the vast majority of sysadmins run programs which don t call fsync() often enough, but the probability of losing data is low and the probability of losing data in a way that you will notice (IE it doesn t get automatically regenerated) is even lower. The low probability of data loss due to race conditions combined with the fact that a server with a UPS and redundant PSUs doesn t unexpectedly halt that often means that problems don t get found easily. But when clusters have problems and start calling STONITH the probability starts increasing. Getting cluster software to work in a correct manner isn t easy. I filed Debian bug #430958 about dpkg (the Debian package manager) not calling fsync() and thus having the potential to leave systems in an inconsistent or unusable state if a STONITH happened at the wrong time. I was inspired to find this problem after finding the same problem with RPM on a SUSE system. The result of applying a patch to call fsync() on every file was bug report #578635 about the performance of doing so, the eventual solution was to call sync() after each package is installed. Next time I do any cluster work on Debian I will have to test whether the sync() code seems to work as desired. Getting software to work in a cluster requires that not only bugs in system software such as dpkg be fixed, but also bugs in 3rd party applications and in-house code. Please someone write a comment claiming that their favorite OS has no such bugs and the commercial and in-house software they use is also bug-free I could do with a cheap laugh. For the most expensive cluster I have ever installed (worth about 4,000,000 UK pounds back when the pound was worth something) I was not allowed to power-cycle the servers. Apparently the servers were too valuable to be rebooted in that way, so if they did happen to have any defective hardware or buggy software that would do something undesirable after a power problem it would become apparent in production rather than being a basic warranty or patching issue before the system went live. I have heard many people argue that if you install a reasonably common OS on a server from a reputable company and run reasonably common server software then the combination would have been tested before and therefore almost no testing is required. I think that some testing is always required (and I always seem to find some bugs when I do such tests), but I seem to be in a minority on this issue as less testing saves money unless of course something breaks. It seems that the need for testing systems before going live is much greater for clusters, but most managers don t allocate budget and other resources for this. Finally there is the issue of testing issues related to custom code and the user experience. What is the correct thing to do with an interactive application when one of the cluster nodes goes down and how would you implement it at the back-end? Running a Cluster Systems don t just sit there without changing, you have new versions of the OS and applications and requirements for configuration changes. This means that the people who run the cluster will ideally have some specialised cluster skills. If you hire sysadmins without regard to cluster skills then you will probably end up not hiring anyone who has any prior experience with the cluster configuration that you use. Learning to run a cluster is not like learning to run yet another typical Unix daemon, it requires some differences in the way things are done. All changes have to be strictly made to all nodes in the cluster, having a cluster fail-over to a node that wasn t upgraded and can t understand the new data is not fun at all! My observation is that the typical experience of having a team of sysadmins who have no prior cluster experience being hired to run a cluster usually involves learning experiences for everyone. It s probably best to assume that every member of the team will break the cluster and cause down-time on at least one occasion! This can be alleviated by only having one or two people ever work on the cluster and having everyone else delegate cluster work to them. Of course if something goes wrong when the cluster experts aren t available then the result is even more downtime than might otherwise be expected. Hiring sysadmins who have prior experience running a cluster with the software that you use is going to be very difficult. It seems that any organisation that is planning a cluster deployment should plan a training program for sysadmins. Have a set of test machines suitable for running a cluster and have every new hire install the cluster software and get it all working correctly. It s expensive to buy extra systems for such testing, but it s much more expensive to have people who lack necessary skills try and run your most important servers! The trend in recent years has been towards sysadmins not being system programmers. This may be a good thing in other areas but it seems that in the case of clustering it is very useful to have a degree of low level knowledge of the system that you can only gain by having some experience doing system coding in C. It s also a good idea to have a test network which has machines in an almost identical configuration to the production servers. Being able to deploy patches to test machines before applying them in production is a really good thing. Conclusion Running a cluster is something that you should either do properly or not at all. If you do it badly then the result can easily be less uptime than a single well-run system. I am not suggesting that people avoid running clusters. You can take this post as a list of suggestions for what to avoid doing if you want a successful cluster deployment.

Next.

Previous.