Search Results: "ag"

14 February 2009

Wouter Verhelst: Debian @ FOSDEM 2009: Postmortem

Well, no, neither FOSDEM nor Debian are dead. But FOSDEM '09 is gone, over, and dealt with. It was a breeze, for the most part. I'm still not very happy with the booth; that needs to be done better next year. Help and suggestions in that area are more than welcome. One thing that more than deserves a follow-up is Lucas' post about FOSDEM:
On a more sad note, the worst talk of the week-end was without any possible doubt Frans Pop's release team bashing. Nobody is claiming that the release management of the lenny release cycle was perfect: there's always room for improvement. But given the context and the constraints, I think that they did a very good job. Frans' talk boiled down to: "The release team doesn't know what they are doing, I would have done much better because I'm so qualified."
As the person responsible for allowing Frans on the schedule in the first place, while being fully aware of the relationship between Frans and the release team (not at first, but certainly in time to kick him off if I'd have wanted to), I'd like to comment on that. First of all, I do not agree that the last statement in the above is true. Frans showed a few stories of things that went on in the release team, and gave his opinion on what he believed went wrong. He also explicitly said, at the beginning of the talk, that none of the talk was meant personally; that he wanted to offer some constructive criticism instead. I believe he did not fail in doing that, but of course YMMV. Secondly, and more importantly, I do not believe it is healthy for Debian (or any project, for that matter) to reject criticism. Indeed, nobody is claiming perfection. I do not believe any venue where talks can be had should be a good news show. People in a position of power not just the release team, but also, say, the DPL, ftp-masters, buildd maintainers and whatnot have received our trust to do what they need to do to the best of their abilities. If someone in the project would believe that I can significantly improve my work as buildd maintainer, it is not just their right, but indeed their duty to inform me of that fact. This is exactly what Frans was attempting to do, and there is nothing wrong with that. It is also not as if he's not tried reasoning with the release team first. He has made suggestions, which have gotten ignored. I feel that talking to more people about what he feels is right, to see whether they agree with him, is the next logical step to take, and that this it is exactly what Frans was attempting to do at FOSDEM. Having said that, I agree that finishing the talk late was a very bad thing. If anything, a talk on a subject like this should have more, not less, time for discussion. So while I do not agree that his intentions were wrong, I do agree that the execution could have been better. There, that was criticism too. Now, what's next? Oh yes, suggestions. If people have suggestions on how to improve the Debian presence at FOSDEM next year (especially, as above, ideas for creative use of the booth would be welcome), then please send me an email, making sure the subject contains the word 'FOSDEM', so my mailfilters know what to do with it. You'll preferably do so now, while FOSDEM is still somewhat fresh in your memory; I'll take notes and use those for doing better next year. Thanks,

Brice Goglin: Debian/X.org notes - X behavior changes in experimental

Apart from interesting features such as DRI2, KMS or input-hotplug, there are some minor changes in X in experimental that actually appear to disturb many users. The first one is that Ctrl-Alt-Backspace does not kill X anymore. There is no easy consensus here, but many people were annoyed of killing X by mistake, so it's disabled by default now. To reenable it, add to the ServerFlags section of your xorg.conf:
        Option "DontZap" "off"
Another one is the background during X startup. Say goodbye to the old well-known grey background. Now you get a black background by default. To revert to the old behavior, pass -retro on the server command line (for instance in /etc/X11/xinit/xserverrc). Note that this option also reenables Ctrl-Alt-Backspace killing the server. Finally, you might also see glxgears reporting very low frame rates (60) on some hardware. Well, please remember that it is not a benchmark, the output basically means nothing. This is why some distros even removed the fps output by default. The thing is that recent DRM stacks will just synchronize frame rendering on vertical blanks (can anybody here see 1000fps with human eye?). So if you have a 60Hz refresh rate, glxgears will report 60fps, that's it. But it has nothing to do with DRI or 3D being slow. Please try some relevant 3D programs or benchmarks before complaining :)
(Permanent link)

Russell Coker: Do Spammers target Secondary MX Servers

Rumour has it that some types of spammer target the secondary MX servers. The concept is that some people have less control over the secondary MX server and less ability to implement anti-spam measures. Therefore if they accept all mail from the secoondary then a spammer will have more success if they attack the secondary server. True secondary servers are becoming increasingly uncommon, the lower priority servers listed in MX records tend to have the same configuration as the primary, so the benefit for the spammer in attacking the secondary server is probably minimal. But it would be good to know whether they do this. I decided to analyse the logs from a mail server that I run to see if I can find evidence of this. I chose a server that I run for a lient which has thousands of accounts and tens of thousands of messages delivered per day, my own server doesn t get enugh traffic to give good results. I analysed the logs for a week for the primary and secondary MX servers to see if the ratio of spam to ham differed. Now this does have some inherent inaccuracy, some spam will slip past the filters and occasionally a legitimate email will be rejected. But I believe that the accuracy required in a spam filter to avoid making the users scream is vastly greater than that which is required to give a noteworthy result. I produced totals of the number of messages delivered, the number rejected by SpamAssassin (which has a number of proprietary additions), the number of message delivery attempts that were prevented due to rate limiting (most of which will be due to spammers), and the number of attempts to deliver to unknown accounts (some of which will be due to spammers having bad addresses in their lists). For each of these rejection criteria I produced a ratio of the number of rejections to the number of delivered messages for each of the servers. The rate limit count didn t seem useful. While the primary server had a ratio of 0.75 messages rejected due to rate limiting to every message accepted the secondary had a ratio of 0.08. It seems that the secondary just didn t get enough traffic to trigger the limits very often. This is an indication that the more aggressive bots might not be targetting the secondary. The ratio of messages rejected by SpamAssassin to legitimate mail was 0.76:1 on the primary server and on the secondary server it was 1.24:1. The ratio of messages addressed to unknown users to successful deliveries was 3.05:1 on the primary and 7.00:1 on the secondary! This seems like strong evidence to show that some spammers are deliberately targetting the secondary server. In this case both the primary and secondary servers are in server rooms hosted by the same major ISP in the same region. The traceroute between the two mail servers is only 7 hops, and there is only one hop between the two server rooms. So it seems unlikely that there would be some connectivity issue that prevents spammers from connecting to the primary. One other factor that may be relevant is that the secondary server has been in service for some years while the primary is only a few months old. Spammers who store the server IP address with the email address (which happens - change the DNS records to send your mail to a different server and you will see some spam go to the old server) will be sending mail to what is now the secondary server. The difference in the rejected mail volume on the secondary server and the amount that would be rejected if it had the same ratio as the primary amounts to 7% of all mail rejected by SpamAssassin and 14% of all mail addressed to unknown users. I think it s unlikely that any significant fraction of that would be due to spammers caching the server IP address for months after the DNS records were changed. So therefore it seems most likely that something between 7% and 14% of spam is specifically targetted at the secondary server. While the ratio of spam to ham seems significantly worse on the secondary it is still a relatively small portion of the overall spam sent to the service. I had been considering setting up secondary mail servers with extra-strict anti-spam measures but the small portion of the overall spam that is targetted in such a way indicates to me that it is not going to be worth the effort. Another thing that has occurred to me (which I have not yet had time to investigate) is the possibility that some spammers will send the same messages to all MX servers. If that happens then the ratio of spam to ham would increase every time the number of MX servers is increased. In that case it would make sense to minimise the number of MX servers to reduce the amount of CPU power devoted to runing SpamAssassin. Note that I have intentionally not given any numbers for the amount of mail received by the service as it is a commercial secret. Update: One thing I realised after publishing this post is that the secondary MX server is also the main server for mail sent between local users. While the number of users who send mail to other users on the service is probably a small portion of the overall traffic (it s not a really big ISP) it will make a difference to the ratios. Therefore the ratio of spam to ham would be even worse on the secondary MX (assuming for the sake of discussion that local users aren t spamming each other).

Jaldhar Vyas: Digits

1234567890

Christian Perrier: [life] PG in America Latina

37 years since I first heard Peter Gabriel's voice (Genesis' Nursey Crime, one of the first LP I ever bought) and about 34 since I first saw him on stage in what turned out to be Genesis' last show in France (St-Etienne, my home city, may 1975). Since then, I tried to keep up with PG's appearances here and there and he has always been my favourite artist. So, if any of this blog's readers happen to be in Latin America and enjoy PG's music, just try to go there:

Christian Perrier: [life] Sam made it

No, Debian folks, not our former DPL...:-) My, by far, favourite skipper in the greatest race around the world (alone on a 60-feet boat without assistance), the Vend e Globe, arrived last night. I mean Samantha Davies, 34 y.o. female british skipper, who finished 3rd (or 4th, depending on Marc Guillemot's arrival time), who illuminated these weeks with her videos and messages always full of positive attitude and happiness....which is certainly harder to do when surfing in the southern latitudes over 20 knots and thousands of miles away from any land. I recommend those of you who never went on the VG web site to just navigate around the videos and spot those sent by Sam, to understand. She made it with a 9-year old boat (that completes its third VG), keeping up with performance while preserving the boat (probably the one in the best condition after arrival). and all Vend e Globe fans will always remember her constant smile. All this explains why I have even more admiration for Sam's performance than I have for Michel Desjoyaux (the winner) or Armel Le Cl ac'h (2nd), both among the world's top skippers (along with Lo ck Peyron, imho). The race is not over, far from this. Eight skippers still have to arrive. Three of them (Marc Guillemot, Brian Thompson and Dee Caffari) will arrive on Monday or so, while Arnaud Boissi res and Steve White wtill need one or two weeks and Rich Wilson, Rapha l Dinelli and Norbert Sedlacek still are in the southern Atlantic ocean. And kudos also have to go to the 21 skippers who couldn't complete the race, certainly the most difficult Vend e Globe ever. I wish I would live close to Sables d'Olonne to have a chance to see a VG arrival some day. Great race, great adventure. Sad that it happes only every 4 years..:-)

Christian Perrier: The Lenny day?

Suspense..... Will we release Lenny today? :-) Whatever happens, I like the way these last two weeks were handled. The clear communication from the release team made it clear to everybody what the deadlines were. That helped a lot in cooking up good Release Notes and the great Release Annoucencement, as well as their translations. Of course, one can regret that all had to happen in the very last weeks (the RN could have been prepared earlier but nobody was in that mood except certainly Martin Borgert who is trying to shake up the tree for months...). Again, great work, everybody and I wish the best to the folks who will burn their week-end up to polish the last bits of that release.

13 February 2009

Kees Cook: World Of Goo compiled on Ubuntu

I first played World of Goo on the Wii. I loved it. Great stuff, reminded me a little of Lemmings, but way way better. Today I found out it s also available for Linux, and I immediately downloaded it. I was pleased to see a .deb available and thought I d dig into the binary a little to see how it was put together. I m hugely excited to see that it looks like the game was built on Ubuntu Intrepid for both the .deb and .tar.gz packages:
$ strings -a WorldOfGoo.bin grep -i gcc
...
GCC: (Ubuntu 4.3.2-1ubuntu12) 4.3.2
The bundled libraries appear to come from Gentoo:
$ strings -a libs/libogg.so.0 grep -i gcc
...
GCC: (GNU) 4.2.3 (Gentoo 4.2.3 p1.0)
OpenSSL (and libpng?) from Gentoo seem to have been statically linked into the binary:
$ strings -a WorldOfGoo
...
GCC: (GNU) 4.2.3 (Gentoo 4.2.3 p1.0)
_x86_AES_encrypt
...
What I m so very excited about is that third party software producers are using Ubuntu for their releases. This means they get all the security and code-quality improvements that are built into the Ubuntu compiler for free:
$ objdump -CR WorldOfGoo.bin grep _chk
...
084043f0 R_386_JUMP_SLOT __memcpy_chk
Many thanks to Maks Verver for doing the porting. I m really enjoying the demo. Also, I can see your devel path! :)
$ strings -a WorldOfGoo.bin grep openssl
OPENSSLDIR: "/home/maks/Projects/wog/libs/openssl-0.9.8j-linux-i386/ssl"
...
All this said, it may be that they re actually only compiling pieces (like the Open Dynamics Engine, which seems to contain the bulk of the *_chk calls) on Ubuntu (or using .a files from a distro-compiled library that enables hardening only in builds RedHat, SuSE), but doing the .deb and .tar.gz project linking on Ubuntu (which is why the compiler name shows up in the resulting binary). The RPM shows the *_chk functions too actually the disassembly is identical between deb and RPM, which would imply that only final linking was done on the native distros but compiled elsewhere. I d love to hear more details if Maks stumbles on this blog. :)

Rob Bradford: GNOME web services integration: Introducing librest

When OpenedHand was first acquired by Intel I was asked to look into ways of making it easier to access web services from within GLIBish (i.e. Clutter & GTK) applications. We figured that as web services more important to the users of Moblin, and thus by extension, GNOME we needed some way to help developers access them. So, I did some research, read up on RESTful. And thought yay, lots of services are claiming to be RESTful, let s see if we can come up with some kind of service description language that we can autogenerate code from (ala, dbus-glib). Turns out that very few services actually adhere to this RESTful definition and my attempts to produce some kind of description language resulted in more lines of XML than code, hmmm. Anyway, I persevered and looked at building an API that I would fine easy to drive RESTful web services with. After a few iterations I decided upon designing something with a somewhat Cairo-esque API.



Marco thinking RESTful thoughts.
The API is comprised of two parts, the first half is for making the requests (and is based on libsoup.) Here is an example:
  proxy = rest_proxy_new ("http://www.flickr.com/services/rest/", FALSE);
  call = rest_proxy_new_call (proxy);
  rest_proxy_call_add_params (call,
                              "method", "flickr.test.echo",
                              "api_key", "",
                              "format", "xml",
                              NULL);
  rest_proxy_call_run (call, NULL, NULL); /* obviously can do async instead */
  payload = rest_proxy_call_get_payload (call);
  len = rest_proxy_call_get_payload_length (call);
Easy, huh? Anyway, enough rambling for today s blog post, next week i ll talk about the really interesting stuff we re (Ross and I) are working on to make parsing XML fun (yes, really, honestly, fun.) Oh and and the source code can be found in git and it s licensed under LGPLv2.1, etc etc. (Yes, we have too many libraries, when mature perhaps we can move this into libsoup.)

Neil Williams: What's this em1 version thing?

From tomorrow, if you start seeing bugs or getting email about your packages where the version string ends in em[0-9], or where dependencies mentioned by reportbug include such a version suffix, the user is running one of the two Emdebian distributions to be released alongside lenny (and based on lenny packages). You might also see Emdebian in the output of 'apt-cache policy' if your bug reports request or supply such data.

If there is any doubt that the bug is present in Debian, please immediately re-assign the bug to the buildd.emdebian.org pseudo-package in the BTS.

I'll be posting to d-d-a with more information on the releases and the em1 version suffix - just as soon as Lenny itself is released.

A patch for reportbug is being tested and I'll file the bug probably tomorrow. A large proportion of the development work to actually get Emdebian to the point of two releases only happened after the lenny freeze began. (Not the recent deep freeze, the initial toolchain freeze before DebConf8). (reportbug itself is not currently part of Emdebian for a variety of reasons.)

Emdebian only uses a small subset of Debian packages, so this principally involves the core packages in Debian, GTK+2.0 and dependencies, XFCE and GPE as well as a small number of applications like iceweasel, sylpheed, xchat-gnome and grisbi.

The two Emdebian distributions will be:

Emdebian Grip 1.0 (lenny)
A small Debian-compatible Emdebian installation
Full release title:
Emdebian GNU/Linux Grip 1.0 (based on Debian GNU/Linux 5.0 "lenny").
Short release name:
Emdebian Grip 1.0 (lenny)
Common release names:
Emdebian Grip or Grip (lenny)
Architectures: i386, amd64, powerpc, arm, armel, mips and mipsel


Emdebian Grip explicitly supports mixing Debian and Emdebian and that is how a Debian package could end up depending on a package from Emdebian Grip - this is not expected to be a problem because Grip packages are binary-compatible with the Debian version, just with some files removed from the package. (i.e. Grip is post-processed, not recompiled.) Grip typically comes in at ~600Mb for a full XFCE desktop and uses the Lenny debian-installer images. (Grip base system ~250Mb.) I'm currently running Emdebian Grip on my Acer Aspire1.

More information: http://www.emdebian.org/grip/

Emdebian Crush 1.0 (lenny)
The smallest Emdebian installation - cross-built from Debian sources.
Full release title:
Emdebian GNU/Linux Crush 1.0 (based on Debian GNU/Linux 5.0 "lenny").
Short release name:
Emdebian Crush 1.0 (lenny)
Common release names:
Emdebian Crush or Crush (lenny)
Architectures: arm only


More information: http://www.emdebian.org/crush/

Crush is the cross-built distribution from Emdebian - significant functional changes exist compared to the Debian package, libraries have been recompiled with options turned off and functionality removed etc. and Crush has dropped support for python or perl, and uses a modified, more recent version of busybox instead of coreutils. Crush is the hardest distribution to install, maintain and update and requires extensive customisation for particular machines with the benefit that it is an ultimately flexible Debian where the base system (equivalent to a debootstrap in Debian) can fit into 24Mb installed and a GPE GUI in 75Mb. uClibc support is expected in Crush 2.0 (squeeze), allowing sizes to shrink by another 10Mb or so. I'm currently running Emdebian Crush on my balloon3.

Dirk Eddelbuettel: New project: RInside

A few days ago, I started a new project called RInside by uploading a few files to a new SVN repo at R-Forge. RInside makes it easy to embed R into your own C++ application by hiding the nitty gritty of initializing an R interpreter behind a simple abstraction. More information is at a (currently pretty simple) RInside page, and you may want to look at the related Rcpp and possibly littler projects. The former is helpful for data exchange, and the latter provided my first real use of R embedding which in some ways also lead to RInside.

12 February 2009

John Goerzen: Video Hosting Sites Review

Last July, I wrote about video uploading sites. Now that I m starting to get ready to post video online, some public but a lot of it just for friends or family, I ve taken another look. And I m disappointed in what I see. Youtube has made the biggest improvements since then. Now, they can handle high-definition video, an intermediate HQ encoding, and the standard low-bandwidth encoding. Back then, there was no HD support, and I don t think any HQ support either. There are two annoying things about Youtube. One is the 10 minute limit per video file, though that can be worked around. The other is the really quite terrible options for sharing non-public videos. In essence, the only way to do this is to, on each video, manually select which people you want to be able to see it. If suddenly a new person gets a Youtube account, you can t just give them access to the entire back library of videos. What I want it to tell Youtube that all people in a certain GROUP should have access, and then I can add people to the group as needed. That s a really quite terrible oversight. Vimeo, on the other hand, has actually gotten worse. Back a year ago, they were an early adopter on the HD bandwagon. Now that they ve introduced their pay accounts, the free accounts have gotten worse than before. With a free Vimeo account, you can only upload 1 HD video a week. You also get dumped in the 4-hour encoding line, and get the low-quality encoding. Yes, it s noticeable, and much worse than Youtube HQ, let alone Youtube HD. You have no time limit, but a 500MB upload limit per week. The sharing options with Vimeo are about what I d want. blip.tv seems about the same, and I m still avoiding them because you have to pay $100/yr to be able to keep videos non-public. Then there s viddler. I am not quite sure what to make of them. They seem to be, on the one hand, Linux fans with a clue. On the other hand, their site seems to be chock full of get-rich-quick and real estate scheme videos, despite a ToS that prohibits them. They allow you to upload HD videos but not view them. They have a limit of 500MB per video file, but no limits on how many files you can upload or the length of each one, and the sharing options seem good. So I m torn. On the one hand, it would be easy to say, I ll just dump everything to viddler. On the other hand, are they going to do what Vimeo did, or worse, start overlaying ads on all my videos? Any suggestions?

David Welton: Javascript Charts

I've been doing some work on LangPop.com, and one of the things I'd like to do is update the chart software. What I have now, Plotr seems to work ok, but being the tinkering type, I want to fix things, even if they're not broke. Truth be told, my worry is that Plotr isn't maintained any more, so it would be a good idea to find something that's receiving a bit of attention. The candidates: These seem to be fairly up to date in the sense that someone worked on them recently. There are some older ones like PlotKit, that do not appear to be maintained any more. It is entirely possible that I missed a good one. Another possibility would be to use Google's chart API, but I'd rather be a little bit more in control of things than to farm that out, and I also am planning on doing some interactive features in the near(ish) future. So let's have a look: Flot Based on JQuery, this one looks fairly complete, and has a lot of different, nice looking charts. Since I don't really care what library it's based on JQuery seems as good as any other, being quite popular these days. Installation is pretty simple, and, having good defaults, it's easy to get nice looking data up on the screen in short order. Flotr Modeled after Flot, Flotr uses Prototype (the Rails default) instead of JQuery, and is the follow up to Plotr. I'm not quite sure what the motivation was behind the name/project change, but this seems to be where the guy is spending his time. Since he did a good job with Plotr, this one seemed worth a look too. ProtoChart This one is based on Prototype too. It immediately annoyed me by uncompressing from the .zip file in the current directory, scattering files around. I've always found this behavior a bit antisocial. This code claims to be "motivated by Flot, Flotr and PlotKit libraries", which indicate that it's fairly recent. However, my feeling is that the problem is not necessarily how old code is in these projects, but how quickly they spring up, bloom, and then stagnate. I'd like to use something that's got some staying power... But anyway, ProtoChart looks like pretty good code, even if the distribution is a bit minimal, and doesn't come with examples. If you look at LangPop.com, you'll notice something that's very important for my choice: I need a barchart that has horizontal bars, rather than the more traditional vertical bars. This is because it would be quite difficult to squeeze so many languages across the screen horizontally. This ended up being the deciding factor: like Plotr, Flotr supports horizontally oriented bar charts, making it the obvious choice. The other two libraries looked pretty good too - Flot, in particular, looks like quite solid code. So, there you have it - very brief reviews of an incomplete selection of libraries! In my defense, the goal is to attempt to dominate my maximizing nature, pick one, and get on with doing some cool stuff with LangPop. I do, however, welcome comments and suggestions.

Julian Andres Klode: debimg core: creating a repository


As I wrote Monday in debimg core example , I have an extended version which creates a repository. The problem on Monday was that the code was not far enough to be published. Not much has changed yet, but I considered to create a temporary branch temp/repository to be able to show you the example, and to give you an impression of what the final API will look like (it s not complete yet, trust me). Now I will present you the example. It is not much longer than the pool example I gave on monday, but it does much more - it creates a repository (and it has some commandline options). The concept of the Repository is very simple, too. We have two classes, Repository and Distribution, and a Repository object contains multiple Distribution objects, which are responsible for packages and related stuff.
#!/usr/bin/python
# Copyright (C) 2009 Julian Andres Klode.
#
# Released under the terms of the GNU General Publice License, version 3
# or (at your option) any later version.
#
"""Example to demonstrate the power of debimg's repository management.
This example creates a directory example.debimg, with two subdirectories:
    - pool: This holds debimg's file pool (see debimg.core.files.Pool)
    - repo: This holds the created repository (debimg.core.repository)
It creates a repository of all required packages, using your local apt sources.
"""
from __future__ import with_statement
import operator
import optparse
import sys
from debimg.core.resolver import Resolver
from debimg.core.files import Pool
from debimg.core.repository import Distribution, Repository
def main():
    """Called when the script is executed."""
    parser = optparse.OptionParser()
    parser.add_option('-u', '--user', help='Same as in GPG')
    parser.add_option('-c', '--contents', action="store_true",
                     help='Create Contents-*.gz files')
    opts, args = parser.parse_args()
    # Create a pool, which manages the access to the files.
    pool = Pool('example.debimg/pool')
    # Create a resolver using your local apt configuration.
    pkgs = Resolver()
    # Create a new repository
    repo = Repository(pool, 'example.debimg/repo')
    # Create a new distribution and store it in dist.
    dist = repo.add_distribution('lenny')
    # Add all packages with priority required to the resolver
    pkgs.add_priority('required')
    # Add the packages from the resolver to the distribution
    for group in pkgs.groups():
        for package in group:
            dist.add_package(package)
    pool.compact()   # Optional step
    pool.fetch()     # Fetch all the packages from the mirror
    dist.finalize_files() # Link the files into the repository
    dist.finalize_packages() # Create Packages files (uncompressed and gzip)
    ##
    # If we want to create Contents-*.gz files, we can do this as well:
    # but we don't want to
    if opts.contents:
        dist.finalize_contents()
    # Now create our Release files, and sign them using the key provided on
    # the commandline. If no key has been provided, do not sign it.
    dist.finalize_release(Suite='testing', key=opts.user)
    for file in sorted(pool._files, key=operator.attrgetter('uri')):
        print file   # Print information about every file, sorted by URI.
if __name__ == '__main__':
    main()
To try the code, checkout the temp/repository branch at git://git.debian.org/users/jak/debimg.git and execute the script named test-repository, which is the same as printed above. If you have questions, please ask. Also read the docstring of debimg.core.repository to see which changes are not yet done (eg. common base class for Distribution and Repository, and of course adding files manually). Posted in Debian, debimg

Martin-Éric Racine: Here's hoping for the best...

Life sometimes has extreme ways of waking people up and making them realize what matters for them. I'm going through such a phase right now in my romantic life. I won't discuss what happened here, but suffice it to say that I haven't felt so depressed in a long, long, long, long time. The good news is that life has proven to me once again how having good friends who truly care for me makes a whole world of difference. The big slap in the face that life gave me over these last few days was a harsh lesson, but I think that it was worth it. Of course, it helps that people like Gilles, Jaana, Henni, Pia, Linda, Arno and Sailesh beleived in me enough to support me through this rough spot in my life. Although their advices were vastly different, they all pointed in a very positive direction and gave me the strenght to carry on. I also must thank my beloved Anastassija; what she did was cruel beyond beleif and yet, if her few SMS from the last 24 hours are any indication, there's still hope after the storm. I'll be seeing her tonight again, after a few days of separation and I prepared her a truly romantic evening. It ain't much, but I hope that it truly succeeds in bringing the point accross that Love is stronger than everything and that even the unforgivable can be forgiven, over time, if everyone involved has enough faith in each other's future as a couple. Tonight, I truly could use the prayers of a million voices to support me and Anastassija in this difficult time. Life slapped me hard and I got the message... I think... and whatever happens after tonight, I need to know that I gave it my best shot and that, whatever her and I decide, Life can only take a turn for the better from now on.

Julien Danjou: sysrqd 10

I've just released sysrqd 10. I've rewritten big chunks of code, mostly because my C skills have quite improved between now and 4 years ago, and because I've blindly merged contributors patches which were crap.

Norbert Tretkowski: MySQL Workbench for Debian

Based on the official Ubuntu 8.04 package from Sun (thanks to Alfredo Kojima) I prepared a package of MySQL Workbench 5.1.7 for Debian. I am going to upload it to experimental soon, in the meanwhile it is available (amd64 only) from here:

http://people.debian.org/~nobse/mysql-workbench/

Feedback appreciated!

Ana Beatriz Guerrero Lopez: FOSDEM 2009

Some stuff I want to reminder for next year s FOSDEM:

Ingo Juergensmann: Automatically restore files from lost+found

I already wrote two days ago about it, but the problem then was to find a way of unique identifiers for files. Using md5sum and openssl sha1 seemed to deliver multiple files for each hash, which confused me.
A closer look later revealed that there were in fact multiple times the same file, so the same hash is of course correct.

Anywa, I'm now using the following two scripts to create an index file about the content of my filesystems and to be able to restore files from lost+found.

CODE:
#!/bin/bash
#
# Usage: ./make-lsLR.sh
#
# Purpose:
# to make a file that is parseable for recovering
# a filled /lost+found directory by parsing
# filesize, md5sum, permisions and path+filename
#
# Author: Ingo Juergensman - http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.
#
# first: get all directories
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type d -print > /root/ls-md5sum-dirs.txt


# next: get all relevant information
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type f -printf "%s %U:%G %#m " \
-exec nice -15 md5sum \; tr -s " " > /root/ls-md5sum-files.txt


CODE:
#!/bin/bash
#
# usage: check_lostfound.sh <pathtolost+found> [make_it_happen]
#
# Purpose: to find files in lost+found and trying to restore
# original files by comparing ls-md5sum-files.txt (generated by
# make-lsLR.sh
# Option make_it_happen cause the data actually being written/moved
# whereas the script runs in dry mode per default.
#
# Author: Ingo Juergensman - http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.
#
PFAD=$1
dry=$2

# generating md5sum/information of lost+found
echo "Examing $ PFAD /lost+found..."
find $PFAD/lost+found -type f -printf "%s %U:%G %#m " -exec nice -15 md5sum \; > /root/lostfound-files.txt


# creating missing directories if necessary by parsing ls-md5sum-dirs.txt
echo "Creating missing directories..."
for dir in cat /root/ls-md5sum-dirs.txt sed -e 's/\ /,/g' ; do
dir= echo $ dir sed -e 's/,/\ /g'
if [ ! -d "/space/check_lostfound/$ dir " ] ; then
echo " Missing dir "$dir" - creating..."
if [ "$dry" = "make_it_happen" ]; then
mkdir -p "/space/check_lostfound/$ dir "
fi
fi
done


# next, get the md5sum of files in lost+found and compare it
# against stored md5sum in ls-md5sum-files.txt
echo "Restoring/moving files..."
for z in cat /root/lostfound-files.txt tr -s " " sed -e 's/\ /,/g' ; do
z= echo $z sed -e 's/,/\ /g'

size1= echo $z awk ' print $1 '
ugid1= echo $z awk ' print $2 '
perm1= echo $z awk ' print $3 '
md5s1= echo $z awk ' print $4 '
path1= echo $z awk ' print $5 '

file= grep -m 1 $md5s1 /root/ls-md5sum-files.txt
if [ ! -z "$ file " ] ; then
file= echo $file sed -e 's/,/\ /g'
size2= echo $file awk ' print $1 '
ugid2= echo $z awk ' print $2 '
perm2= echo $file awk ' print $3 '
#md5s2= echo $file awk ' print $4 '
path2= echo $file awk ' print $5 '

if [ ! -e /space/check_lostfound/$ path2 ]; then
echo "$path1 -=> $path2"
if [ "$dry" = "make_it_happen" ]; then
cp "$ path1 " "/space/check_lostfound/$ path2 "
fi
fi
fi
done


It's actually working for me, but use them at your own risk - as always! ;)

Anyway, there are some drawbacks with these versions:

  1. it is dog slow. Grepping through 75M of ls-md5sum-files.txt for each file found in lost+found is CPU-intensive and slow.

  2. files with spaces in their names are not handled well. In fact they are not handled properly at all. You'll get "file not found" errors for each filename with spaces.

  3. currently grep -m 1 stops grepping when the first match was found.



If anyone has an idea how to speed up the script or improve it otherwise, please let me know! I already though by myself to remove found files from ls-md5sum-files.txt to reduce the workload on grepping every time all of the 75M and to work around the last point from above. But rewriting the file everytime a fiel was found and moved will put extra workload on disk I/O, which - of course - will get better over time because the file gets smaller. Another idea was to store the file within an array in memory, but then I would prefer to use a Python script for this purpose.

So, when you have some ideas, tips or improvemts, please comment!

MJ Ray: The South West RDA s Business Stakeholder Conference: Shooting the Economic Rapids

I was at this event back in November and they ve just emailed me this week to point out the speeches and some recordings are now available online. The appointed Regional Development Agency is getting more powerful as our indirectly-elected South West Regional Assembly is phased out, so if you re based in SW England, you may find their current views affect a lot of your daily life over the next few years. After the meeting, I passed on a few comments about how Business Link is failing to support cooperatives and I was asked for numbers, but I don t think I have them yet. I m still following this up and will post about it again later.

Next.

Previous.