Search Results: "tar"

14 July 2008

Steve Kemp: It's no use pretending it hasn't happened cause it has

Yesterday I was forced to test my backup system in anger, on a large scale, for the first time in months. A broken package upgrade meant that my anti-spam system lost the contents of all its MySQL databases. That was a little traumatic, to say the least. But happily I have a good scheme of backups in place, and only a single MX machine was affected. So, whilst there was approximately an hour of downtime on the primary MX the service as a whole continued to run, and the secondary (+ trial tertiary) MX machines managed to handle the load between them. I'm almost pleased I had to suffer this downtime, because it did convince me that my split-architecture is stable - and that the loss of the primary MX machine isn't a catastrophic failure. The main reason for panicing was that I was late for a night in the pub. Thankfully the people I were due to meet believe in flexible approaches to start times - something I personally don't really believe in. Anyway the mail service is running well, and I've setup "instant activation now", combined with a full month of free service which is helping attract more users. Apart from that I've continued my plan of migrating away from Xen, and toward KVM. That is going well. I've got a few guests up and running, and I'm impressed at how stable, fast, and simple the whole process is. :) ObQuote: Brief Encounter (That is a great film; and a true classic. Recommended.)

Neil Williams: State of play in Emdebian ARM

Autobuilder Summary:
138 packages built successfully. 19 packages failed to cross build. 2 packages are tagged for a manual upload. (Dependency issues - rather than break the archive, these packages will not be uploaded until relevant dependencies have been fixed and uploaded.)
Stats
87% built OK
12% failed
Notes:
  1. All these packages built previously and fail at the latest versions in Debian Sid. Many could be upstream errors. All exist in Emdebian at
    previous versions (only 1 failure is known to contain bugs in the current versions in Emdebian and this only affects GUI installations. Two other GUI packages contains bugs but build OK.).
  2. Some of the packages in the repository (including some of the failures) are not used in *any* of the default emsandbox configurations. Two need to be removed from the repository completely.
  3. Some fixes are waiting for updates in Debian, either new uploads migrating to mirrors or pending fixes awaiting upload by me. (Probably tomorrow.)
  4. The history of each package is now available via the Emdebian Autobuilder Report. The package name links to the history of previous build attempts for this package. The version string links to the repository data for the current version in Emdebian. The result of the build links to the build log.
  5. A comparison with Debian unstable is constantly updated.
  6. 254 source packages exist in Emdebian but only 137 are currently built by the autobuilder. The rest will be built when new versions are uploaded to Debian (in some cases, that won't be until after Lenny). If I get time, I'll add build logs for these packages.
  7. The repository contains a fair number of packages that we don't currently use. This is a reflection of the problems of determining the "path" through the dependencies when starting at the bottom (glibc uClibc etc.). Bear this in mind when considering new architectures and new package sets. The process involves a lot of trial and error, particularly as cross-building potentially alters the dependency path at each iteration.

Problems
  1. libselinux and libsepol have just been uploaded with a bug fix that is needed for e2fsprogs and file cross-builds, amongst others. Waiting
    for the ARM mirror to receive the new builds, hopefully tomorrow.
  2. gtk+2.0 fails to cross-build because the patches now try to build the udeb which comes up against a bug in dpkg-cross. I'll be uploading the new version (including a couple of other bug fixes) tonight. (Final testing currently in progress.). The Gtk package also needs some work to run /usr/lib/libgtk2.0-0/update-gdkpixbuf-loaders so that the icons can be read in the GUI.
  3. pango1.0 also needs a fix to update the pango modules. This is usually done at build time but it means running a cross-built binary. It is a minor task and not CPU intensive so I plan to do this in postinst. Without this fix, no text is rendered in the GUI, only empty glyphs.
  4. xfonts-base is way too large still. More work is needed here to optimise just what is meant by the 'fixed' font for X, precisely which font files can be removed, which might be needed outside C or English locales etc.
  5. Other packages not currently cross-building include: (NOTE: Some of these build logs are VERY large.)
    • adduser - needs to be removed from the repository, it is perl and we implement it via busybox.
    • cups cupsys - /usr/include/pthread.h:653: warning: '__regparm__' attribute directive ignored.
    • gcc-4.2 - needs to be removed from the repository, it no longer builds libgcc1 which was the main reason for cross-building it.
    • glib2.0 - more generated content: /bin/sh: line 2: /usr/bin/glib-genmarshal: No such file or directory
    • gnomevfs - hasn't built for ages and isn't usable anyway due to dependency issues. Currently preventing the use of gpe-filemanager. Due to be replaced by GIO in Debian.
    • libusb - build failure, so far resistant to usual fixes.
    • ncurses - build failure - uses gcc in wide char support build.
    • ntp - build failure - assembly error. authkeys.c:445: error: invalid 'asm': invalid operand for code 'w'
    • pcre3 - build failure - apparently simply unresolved symbols resistant to usual fixes.

    • slang2 - possibly needs a patch update, tries to build -pic archives and then fails.
    • xorg-server - build failure: make[4]: *** No rule to make target -L/usr/arm-linux-gnu/lib', needed by Xprt'. Stop.


Of the problems above, only problems 2, 3 and 4 (gtk, pango and xfonts-base) are "critical" as these are the only ones where the current packages are sufficiently broken that the new version is essential before Emdebian can make a "release" of a series of three root filesystems for ARM. Sizes are still larger than desired - a fix for that requires changes in glibc which I hope to investigate during DebConf.

Any help on the above issues is appreciated.

Problem-solving:
$ emsource -c $package
$ cd /path/to/$package
$ emdebuild

If you have SVN access, use 'emdebuild --svn-only' to commit your changes (even if your changes don't solve all the problems).

Without SVN access, post the result of 'svn diff ../emdebian*' to this list (or 'svn diff ../debian*' if you have added a patch for the upstream code to debian/patches/).

Platforms:
All the above is based solely on Emdebian ARM for balloon3:
Linux balloon 2.6.25.2-pxa270 #1 Sun May 18 22:38:11 BST 2008 armv5tel unknown
balloon3 in CUED case
http://balloonboard.org/gallery/300/balloon3-0v1-fpga.jpg

AFAICT no other ARM platforms have been tested. :-(

I have a few fixes for the balloon3-config package that also needs an update in Emdebian - I've had some feedback from the touchscreen driver
upstream that we might be trying to use the wrong device (/dev/input/mice is the wrong target for the symlink, we can set whatever device we want in /etc/X11/xorg.conf and the driver will use that in preference to /dev/event0). As this is device-specific, /etc/X11/xorg.conf will be put into balloon3-config which is available via Emdebian SVN.

Evan Prodromou: 26 Messidor CCXVI

Since the launch of Identi.ca a few weeks ago, I've had a very busy time. Not much sleep, but lots of fulfilling and exciting work. It's invigorating to work on something that is popular and that you believe in. And I'm glad that the Franklin Street Statement so succinctly encapsulates those beliefs. As some people may have read in my previous blog post about the motivations behind creating Identi.ca (see Journal/14 Messidor CCXVI), I was part of a group convened at the FSF to discuss the impact of the growth of software as a service on user autonomy. It was a very loose organization of hackers, activists, and scholars who come from different backgrounds but all share an interest in user rights online. As computing moves into "the cloud" (see cloud computing), what power does the user retain to control their own computing experience? As much of our social lives -- romance, family, work, friends -- becomes Web-enhanced, what can we do to assert our right to manage that data and its use? How can software developers and service providers gauge their own proper ethical behaviour, and how can users of services judge what is and is not acceptable to use? We didn't come up with any easy answers, but we've summarized our thinking in a new document: the Franklin Street Statement on Freedom and Network Services. In essence, we've tried to point a direction towards what software developers, service providers, and software users should think hard about when thinking about network services. [image] Our group will continue to explore these issues on our new group blog, http://autonomo.us/ . We're going to concentrate on the effects of software services on user autonomy -- people's ability to make their own informed choices about their data and creative works and the software that processes them. It is a realm that as a society -- a cluster of societies -- we're only beginning to understand, and I think that there is still a lot of exploration to do. Autonomo.us includes a wiki where we'll be exploring some of these ideas, and the blog will feature guest submissions about the subject. We're a loose and unofficial group with some smiling benevolence but no sanction from the FSF or any related organization, so we're really going to dig into some of the gray areas of this issue without worrying about making official statements for any one organization. I'm looking forward to the coming months and I hope this issue captures the imagination of the Web's Open Source and Open Content communities. P.S. You can see the FSF press release about the statement and the launch of autonomo.us. tags:

Open Service Definition On a related note, one of the first organizations to approach the issue of Open Services has been the Open Knowledge Foundation. Today they've launched the 1.0 version of the Open Software Service Definition (OSSD). I think the OSSD 1.0 is a great step. It's a bar against which we can start measuring Open Software Services. For example, I think that Identi.ca meets the requirements of the OSSD. Other sites, like Wikipedia, are also clearly compliant. Some services that I really like, such as OpenStreetMap and geonames.org, seem to be compliant. But are they? It's good to start this investigation. I think other kinds of services are on their way there. The announcement by Reddit that their code will now be Open Source is a great step for user autonomy in the social news arena. Now, Reddit needs to consider what an Open Content/Open Data policy would mean for their service... or see others implement it on other sites. I look forward to a rich ecology of open software services growing, now that we have a name for it and a clear community of people interested in the topic. tags:

Cyril Brulebois: Package updates

This (extended) weekend features: pkg-phototools collab-maint More to come

13 July 2008

Bastian Venthur: Porting rng to QT4

Today I was busy porting reportbug-ng to QT4. One really cool thing I noticed is the new QWebView widget, which is the main widget of the QWebKit module. It works surprisingly easy compared to the good old QTextBrowser of QT3 — now you can just throw an URL at it and it will render the page nicely complete with CSS, images, etc. Here’s the obligatory screenshot: reportbug-ng qt4 as you can see, the bugreport looks much better than in the old version of rng. WebKit does a really good job rendering the HTML, I think the fonts look even smoother than under firefox. To ease the transition to from QT3 to QT4 I’ve extracted all the Debian BTS specific stuff to python-debianbts and uploaded it to unstable. Rng uses it to query our Bug Tracking System via it’s SOAP interface and get the data in Python’s native data types. There is still the problem with rng (QT3) crashing when showing bugreports. I’ve investigated the problem a bit and found out that it is not rng’s fault but probably a problem with python-qt3 or one of it’s underlying libs. QTextBrowser causes segfaults in nearly 50% of the cases where you try to render html from a different thread: here’s an easy way to reproduce:
# crash.py
# Usage: python crash.py "some text"
#        python crash.py "some evil html”
from qt import *
import sys
import threading
class Form(QMainWindow):
    def __init__(self,parent = None,name = None,fl = 0):
        QMainWindow.__init__(self,parent,name,fl)
        self.browser = QTextBrowser(self, “browser”)
        self.setCentralWidget(self.browser)
if __name__ == ‘__main__’:
    if len(sys.argv)  2:
        print 'Usage: %s "some text"' % sys.argv[0]
        sys.exit()
    s = str(sys.argv[1])
    # setup qt stuff with a browser
    app = QApplication(sys.argv)
    app.connect(app, SIGNAL("lastWindowClosed()"), app, SLOT("quit()"))
    form = Form()
    app.setMainWidget(form)
    form.show()
    # start a different thread and try to set the text in the browser
    # we wait 1.0 second -- just to make sure the change happens after the
    # start of app.exec_loop() below
    t = threading.Timer(1.0, form.browser.setText, (s, ))
    t.start()
    app.exec_loop()
QWebView doesn’t seem to have this problem, the QT4 version of rng didn’t crash a single time. I’m not quite ready with the porting to QT4, and since I’ll be quite busy during the next weeks I suppose it will take a few weeks until the new version will be updated to unstable.

Russell Coker: New SE Linux Policy for Lenny

I have just uploaded new SE Linux policy packages for Debian/Unstable which will go into Lenny (provided that the FTP masters approve the new packages in time). The big change is that there are no longer separate packages for strict and targeted policies. There is now a package named selinux-policy-default which has the features of both strict and targeted. When you install it you get the features of targeted. If you want the strict features then you need to run the following commands as root: semanage login -m -s user_u __default__
semanage login -m -s root root Then you can logout and login and you get the main benefit of the strict policy (users being constrained). IE you can convert from targeted to strict without a reboot! The above only changes the access for user login sessions (and cron jobs). To fully convert to the strict policy you need to remove the unconfined module with the command “semodule -r unconfined“, currently that results in a system that doesn’t boot - I’m working on this and will have it fixed before Lenny. Also it’s possible to have some users unconfined and some restricted in the way that strict policy always did. When running in the full strict configuration you need to run the command “newrole -r sysadm_r” immediately after logging in as root. When you login you default to staff_r which doesn’t give you the access needed to perform routine sys-admin tasks. Due to the change in the function of the policy packages (in terms of not having a strict package) it made sense to revise the naming (Fedora 9 has a package named selinux-policy-targeted which also provides the strict configuration - I don’t want to do that and don’t have as much legacy as Fedora). This is why I decided to not have package names that include the word “policy” twice. Of course all policy packages get new names, but the ones that matter needed new names anyway. Another new feature is the package selinux-policy-mls, as the name suggests this implements Multi Level Security [1]. I don’t expect that the MLS policy will boot in enforcing mode in a regular configuration at this time (you could probably hack it to boot in permissive mode and switch to enforcing mode just before it starts networking). I uploaded it in this state so that people can start testing it (there is a lot of testing that you can do in permissive mode) and so that it can get added to the package list in time for Lenny. I expect that I’ll have it booting shortly (it should not be much more difficult than getting the strict configuration booting). In terms of the use of MLS, I don’t expect that anyone will want to pay the money needed for LSPP [2] certification. NB The wikipedia page about LSPP really needs some work. I believe that the main benefit for having MLS in Debian is for the use of students. I periodically get requests from students for advice on how to get a job related to military computer security. Probably the best advice I can offer is to visit the career section of an agency from your government that works on computer security issues, for US readers the NSA careers page is here [3]. The second best advice I can offer is to work on MLS support in your favourite free OS. Not only will you learn about technology that is used in military systems but you will also learn a lot about how your OS works as MLS breaks things. ;) Finally I’d like to thank Manoj for all his work. For a while I didn’t have time to do much work on SE Linux and he did a lot of good work. Recently he seems to have been busy on other things and I’ve had a little more time so I’m taking over some of it.

Obey Arthur Liu: Back from Zurich

I finally got back from Switzerland. Google invited all the Summer of Code students to visit their offices. I chose to go to the Zurich one. It wasn’t the closest (London was) but it was the largest and it had more engineers than MBAs. img_0164medium It has been very interesting meeting other Summer of Code participants from all Europe and a lot of other people of the free software community. There were about 40 people overall for the meeting at Google, along with free beer, cake and nice food. img_0161medium I gave a talk about Aptitude, how it could be useful and how I worked so far. It was nice to see that nearly a third of the audience used Debian or a derivative. By the way, the proprietary Nvidia X driver seriously messed up with the projector. It just wouldn’t detect the external display at more than 640×480, which Impress didn’t like at all. I didn’t feel like hacking around my xorg.conf while on stage so I went without the slides (well, you could see about a half of each slide actually). I heard that it was because of my old GPU (GeForce 7400) that I had poor support and the impossibility of, for example, doing RandR correctly. I had to restart X and stuff… The talk went smoothly otherwise and Google staff graced some of us with a guided visit of the 6 stories building, complete with slides, cable-cars and 18th century-themed rooms. I had to sign a NDA over whatever secrets I may see and surender my DSLR, but you can find pretty pictures in this blog post from Valleywag.
025_f0
The Google office really lived up to the hype. I wondered: do people actually work in there ? I guess they must do, and very well.

Sune Vuorela: kde / java / packaging

The following is not a joke: more precise, soprano, used by nepomuk (semantic desktop thingie), has a sesame backend (java) and a redland backend (c++). According to upstream and others, the sesame backend is way faster than the redland backend. In the soprano svn dir (and in the pristine tarball), there are a few jars put in precompiled: This isn’t completely beautiful, but as they all are bsd licensed, it is at least legal.
But it isn’t good enough for debian, as we ship sources and build them. (I think I have heard at least fedora mumble something similar)
The two last jar files, the slf4j ones, are available packaged in debian in the libslf4j-java package. but openrdf-sesame is not available. If some persons out there has java knowledge, debian packaging knowledge, preferably knowledge of debian java policy and some spare time, packaging openrdf-sesame would be much apprecaited. And after that, getting the soprano sesame backend built is next goal. Thanks in advance. applications welcome.

12 July 2008

Petr Rockai: fast forward

Lucy left for England yesterday (OK, now it’s two days ago — last Friday). That means that I have unusual amount of free time at my hands, and yet even more things that could be done with it. But I suppose it’s time to update this little blog. It’s probably pointless to enumerate what happened, or what changed. Everything is fine and smooth over here — the wrinkles get worked out over time (with Lucy, without Lucy). Hobbies still take time and I still haven’t given up. My latest determination is to get myself a bassoon for next Christmas. (Now, that will be a year since we got Lucy a Marigaux 901, used, in great condition for a great price, lo and behold, on Austrian ebay. World is so weird sometimes…) And in the land of [LVM], I have finally started the process to merge code to improve LVM’s behaviour in presence of failed storage hardware (physical volumes gone missing). In the land of Adept, well… first things first. FOSSCamp I have visited Prague for the latest Ubuntu- (well, Canonical-) organised event, the FOSSCamp. I have met Johnatan (KUbuntu), Seli (KWin), Lidya (Amarok), Robert (Konsole), Jos (Strigi) and Inge (KOffice). See also Johnatan’s Blog (including a real blurry picture). So back to Adept — I had some hacking time over there in Prague, and I have almost brought Installer and Updater back to life for Adept 3. I unfortunately didn’t have as much time for it since then — but my current free time situation does open up some possibilities. First and foremost, I should really make a text interface to the underlying libraries for myself, maybe with fancy colourful UI, hopefully one that is comfortably close enough to apt-get and still offers advantages. Hmmmm… Intermezzo 1 Now zoom out and zoom in somewhere else, enough of coding matters. Our research group at the University had a (tool) paper accepted for ATVA 2008, meaning that I am not unlikely to visit Seoul, South Korea in October. Another piece of distant world to visit. FOSDEM Now, that reminds me… Since the last time, I have also been in Belgium — to visit FOSDEM — meeting Alasdair (of LVM) and Bart and Pino of Krita and Okular, respectively… although unfortunately, I didn’t spend nearly enough time with Pino… At least we have spend a day walking around Ghent with Bart (and Lucy, who visited Belgium with me, also having friends of her own there). Moreover, I have spent a lot of mostly productive time with Alasdair, discussing LVM2. And Belgium is nice and pretty, although I didn’t really get around to taste any beer. Maybe next time. And Antwerps were nice, too. Photos? Someday. LinuxTag And after that, I have visited Berlin again, for LinuxTag as usual — plus the accompanying LVM discussions, with Milan, Kabi and Mikul (Blek) of the Czech part of the team, and Heinz, John and of course Alasdair for the rest of the world… Also as usual, we walked around, had dinners, discussed non-work stuff, etc… a good event all in all. At LinuxTag, I have briefly seen Lidya again, as well as Ossi (whom I nearly didn’t recognise…) and Aaron who (for a change) didn’t recognise me (but to be fair, I didn’t quite stop by to chat and he’s been busy…) and Sebas, cordial as ever (and always a pleasure to meet)… reminds me of Paris two years ago, too. Intermezzo 2 Less than a month ago, I have finished my first semester of master’s study — two or three more to go (I have completed bachelor’s the semester before). Yes, I am a bachelor of computer science, or something like that, anyway, now. Or so I hope. I did not attend the whatever ceremony and I don’t really have the diploma (or maybe I do, but gods know where it is…). A short note on Debian with best intentions I am now sponsoring Trent Buck’s ?darcs packages, contributing a little on the go (making me wish that alpha buildd would make a little more progress…). With Enrico, we have uploaded new versions of wibble and libept, both of which I think make both of us fairly happy. (Although we again managed to hit a way strange compiler issue (only manifested on arm… what have I done that the gods punish me so?), as documented in Debian bug 487406…) I have packaged dzen2 and taken over haskell-mode (I am losing track of my own packages again — I really need to set up reliable watch files so I don’t miss out too many releases… apparently neither has new upstream versions, so I can sleep peacefully for a little longer). Intermezzo 3 I am not going to DebConf nor Akademy this year. That makes me a little sad, but it’s all my own fault (and laziness). Next time folks, next time. I haven’t seen K vin in aeons and he probably removed me from his memory by now… No matter, I’ll try a little harder next year, promise (oh, how many have I made to date?). Finale Nothing grandiose, just best wishes to everyone, I have to land in bed now it seems, as I am ever so sleepy. (Just So Stories, anyone?) There I go.

David Pashley: Rebuilding a RAID array

I recently had a failed drive in my RAID1 array. I've just installed the replacement drive and thought I'd share the method. Let's look at the current situation:
root@ace:~# cat /proc/mdstat 
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sda3[1]
      483403776 blocks [2/1] [_U]
      
md0 : active raid1 sda1[1]
      96256 blocks [2/1] [_U]
      
unused devices: <none>
So we can see we have two mirrored arrays with one drive missing in both. Let's see that we've recognised the second drive:
root@ace:~# dmesg   grep sd
[   21.465395] Driver 'sd' needs updating - please use bus_type methods
[   21.465486] sd 2:0:0:0: [sda] 976773168 512-byte hardware sectors (500108 MB)
[   21.465496] sd 2:0:0:0: [sda] Write Protect is off
[   21.465498] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   21.465512] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   21.465562] sd 2:0:0:0: [sda] 976773168 512-byte hardware sectors (500108 MB)
[   21.465571] sd 2:0:0:0: [sda] Write Protect is off
[   21.465573] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   21.465587] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   21.465590]  sda: sda1 sda2 sda3
[   21.487248] sd 2:0:0:0: [sda] Attached SCSI disk
[   21.487303] sd 2:0:1:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[   21.487314] sd 2:0:1:0: [sdb] Write Protect is off
[   21.487317] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[   21.487331] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   21.487371] sd 2:0:1:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[   21.487381] sd 2:0:1:0: [sdb] Write Protect is off
[   21.487382] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[   21.487403] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   21.487407]  sdb: unknown partition table
[   21.502763] sd 2:0:1:0: [sdb] Attached SCSI disk
[   21.506690] sd 2:0:0:0: Attached scsi generic sg0 type 0
[   21.506711] sd 2:0:1:0: Attached scsi generic sg1 type 0
[   21.793835] md: bind<sda1>
[   21.858027] md: bind<sda3>
So, sda has three partitions, sda1, sda2 and sda3, and sdb has no partition table. Let's give it one the same as sda. The easiest way to do this is using sfdisk:
root@ace:~# sfdisk -d /dev/sda   sfdisk /dev/sdb
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdb: 60801 cylinders, 255 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an MSDOS signature
 /dev/sdb: unrecognised partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0
   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *        63    192779     192717  fd  Linux RAID autodetect
/dev/sdb2        192780   9960299    9767520  82  Linux swap / Solaris
/dev/sdb3       9960300 976768064  966807765  fd  Linux RAID autodetect
/dev/sdb4             0         -          0   0  Empty
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
If we check dmesg now to check it's worked, we'll see:
root@ace:~# dmesg   grep sd
...
[  224.246102] sd 2:0:1:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[  224.246322] sd 2:0:1:0: [sdb] Write Protect is off
[  224.246325] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[  224.246547] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  224.246686]  sdb: unknown partition table
[  227.326278] sd 2:0:1:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[  227.326504] sd 2:0:1:0: [sdb] Write Protect is off
[  227.326507] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[  227.326703] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  227.326708]  sdb: sdb1 sdb2 sdb3
So, now we have identical partition tables. The next thing to do is to add the new partitions to the array:
root@ace:~# mdadm /dev/md0 --add /dev/sdb1
mdadm: added /dev/sdb1
root@ace:~# mdadm /dev/md1 --add /dev/sdb3
mdadm: added /dev/sdb3
Everything looks good. Let's check dmesg:
[  323.941542] md: bind<sdb1>
[  324.038183] RAID1 conf printout:
[  324.038189]  --- wd:1 rd:2
[  324.038192]  disk 0, wo:1, o:1, dev:sdb1
[  324.038195]  disk 1, wo:0, o:1, dev:sda1
[  324.038300] md: recovery of RAID array md0
[  324.038303] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
[  324.038305] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
[  324.038310] md: using 128k window, over a total of 96256 blocks.
[  325.417219] md: md0: recovery done.
[  325.453629] RAID1 conf printout:
[  325.453632]  --- wd:2 rd:2
[  325.453634]  disk 0, wo:0, o:1, dev:sdb1
[  325.453636]  disk 1, wo:0, o:1, dev:sda1
[  347.970105] md: bind<sdb3>
[  348.004566] RAID1 conf printout:
[  348.004571]  --- wd:1 rd:2
[  348.004573]  disk 0, wo:1, o:1, dev:sdb3
[  348.004574]  disk 1, wo:0, o:1, dev:sda3
[  348.004657] md: recovery of RAID array md1
[  348.004659] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
[  348.004660] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
[  348.004664] md: using 128k window, over a total of 483403776 blocks.
Everything still looks good. Let's sit back and watch it rebuild using the wonderfully useful watch command:
root@ace:~# watch -n 1 cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb3[2] sda3[1]
      483403776 blocks [2/1] [_U]
      [=====>...............]  recovery = 26.0% (126080960/483403776) finish=96.2min speed=61846K/sec
      
md0 : active raid1 sdb1[0] sda1[1]
      96256 blocks [2/2] [UU]
      
unused devices: <none>
The Ubuntu and Debian installers will allow you create RAID1 arrays with less drives than you actually have, so you can use this technique if you plan to add an additional drive after you've installed the system. Just tell it the eventual number of drives, but only select the available partitions during RAID setup. I used this method when a new machine recent didn't have enough SATA power cables and had to wait for an adaptor to be delivered. (Why did no one tell me about watch until recently. I wonder how many more incredibly useful programs I've not discovered even after 10 years of using Linux)
Read Comments (3)

John Goerzen: Kernel interrupt weirdness?

I've had a problem with recent kernels. (I think it's the kernel that's doing this.) When my workstation is doing heavy I/O, it repeats keystrokes. For instance, while I was typing this paragraph, audacity was writing audio to disk, and I got this word:

heavvvvvvvvvvvy

It seems as if it thinks I haven't let up on the keys.

I've seen this on two different machines and it seems to have started with 2.6.24 or 2.6.25.

Has anyone else seen this? Any ideas where I'd go to fix it? Incidentally, I'm in X when this happens. I don't use the console much when there would be a chance for it to happen.

This is such a weird problem I've struck out googling, and I'm not even sure which mailing list to take it to.

Rob Taylor: Thoughts from GUADEC

Frederico’s talk had some bits that reminded me of a “Topaz” desktop design that came up on desktop-devel-list back in ‘06. It wasn’t given much time of day back then though. Check it out. The concept of the computer as Personal Assistant appeared in a few forms this year. Concepts like it helping you organize your information and time and managing your attention. Its obvious there’s been a few threads of thought on this bubbling away for a while now and this year you could see ideas starting to crystallize. Very cool! Its obvious we’re going to need some good query capabilities over large RDF trees. To my simple mind this seems to basically involv unification over tuples, so prolog keeps popping into my head. Has anyone checked out SeRQL?

11 July 2008

Kartik Mistry: Managing Ideas


* What do you do for managing ideas.. - Any software? - Sticky notes? - Diary? I always gets good (sometime even creative too) ideas during bath, loo or breakfast. And, from sometime, I started to maintain ‘Crazy Morning Ideas’ diary with me. And these Google’ internal tools are also interesting..

Gerfried Fuchs: Plug And Pray

Sometimes one stumbles upon interesting pictures. It happens too regularly in recent times and I don't think that it will end too soon. This new section of my blog will contain such photos from time to time.
I'll start off with a quite interesting poster we stumbled upon in my hometown. If you don't understand German, it's about church service targetted at youths—and everyone knows that young people always mix their language with english vocabulary all the time to sound hip, don't we... Don't forget to also read the small text at the bottom, or you don't know where the "Chill Out - Meet & Greet" is going to happen! Plug & Pray

Adam Jensen: 11 Jul 2008

Yesterday, 10 July, was the due date for the second milestone of my work on DebGraph. I am happy to report that we are roughly two weeks ahead of schedule, so meeting this milestone was not a cause for worry. We now have support for the following graph operators: The next milestone includes the development of a high-level language (or integration with an existing extension language) that streamlines the construction of complex queries using the operators listed above. We can build arbitrarily complex queries using the C++ operators, but dealing with the static typing and compiler toolchain can be very clunky. As such, I have spent the past week working on Lua bindings for DebGraph, which will enable us to query the graph of Debian packages from a clean, dynamically typed language. Lua has a powerful C API that exposes the Lua stack and lets us move DebGraph information to and from the Lua interpreter. I'm writing documentation that outlines how to use DebGraph from both C++ and Lua in order to make this work accessible to more folks. Sneak peak
As an example, we can utilize the FindCycles operator in Lua as follows:
libdg = package.loadlib("./libdebgraph.so",
"luaopen_libdebgraph")
libdg()
LoadPackages('cache')
-- 'g' is the main graph of unstable binary-arm packages
cycles = FindCycles(g)
print("Found " .. #fc .. " cycles")
for comp_key,comp in pairs(cycles) do
    comp_nodes = ""
    for node_key,node in pairs(comp) do
        prop = GetProperty(node, "Package")
        comp_nodes = comp_nodes .. prop .. " "
    end
    print("* " .. comp_nodes)
end
The above Lua script produces the following result:
reading cache/unstable/main/binary-arm/Packages
Found 61 cycles
* debconf-english debconf-i18n debconf
* perl-modules perl 
* cdebconf fontconfig-config fontconfig gsfonts-x11
libcairo2 libfontconfig1 libgtk2.0-0 libpango1.0-0
libpango1.0-common libxft2 ucf x11-utils xutils
[...]
These are the package names in the strongly connected components, shown in alphabetical order. There is still a lot of work to do before the Lua binding will achieve feature parity with the core library, but we have now laid the foundation (and a brick or two).

Robert Collins: 11 Jul 2008

What the git vs bzr discussion is about IMO is usability. The following blog post about DTrace on linux talks about the same issue, and I'd like to use Bryan's words: "Over and over again, we made architectural and technical design decisions that would yield an instrumentation framework that would be not just safe, powerful and flexible, but also usable. The subtle bit here is that many of those decisions were not at the surface of the system (where the discussion on the Linux list seems to be currently mired), but in its guts." -> "Over and over again, we made architectural and technical design decisions that would yield a Distributed VCS be not just safe, powerful and flexible, but also usable. The subtle bit here is that many of those decisions were not at the surface of the system (where the discussions going on at the moment seem to be currently mired), but in its guts."

Lucas Nussbaum: 3G Internet access using mobile phone + laptop in France

In France, we only have 3 mobile network operators: Orange, SFR, and Bouygues Telecom. They usually discuss their rates together to make sure that one of them doesn’t break the market by creating too interesting rates (that’s why it would be so cool if Free.fr could get a 3G license, and why the governement can tell them “do what we want, or we will give Free a 3G license” - source). I’ve long been interested in using my mobile phone with my laptop to access the Internet. With the 3 operators, you can get a contract for a 3G USB key, but that’s not really interesting: it’s expensive, and there’s a limit on volume of data, so you have to monitor your bandwidth. Also, I only need to use my mobile phone as modem a few times a month. So I’m not really interested in paying an expensive monthly fee for that. What do french MNOs offer? Most of them have rather interesting offers when you want to get unlimited internet access from your mobile, but that doesn’t cover the case of using your phone as a modem, to access the internet on another device (e.g laptop). I’m not sure how they can see the difference, but apparently they do. (It seems that there are hacks making use of those unlimited rates, using your phone as a router between the Wifi network (but your phone has to support Wifi) and the 3G network, using apps like WMWiFiRouter. Another hack is to use the same HTTP proxy on the laptop than on the phone, which seems to give unlimited HTTP access.) But offers for using your phone as a modem are clearly less interesting (probably because they want to push their USB 3G cards offers): Bouygues:
Apparently, you are billed only based on the volume (see page 46 of this document), with a progressive rate. 5 MB/month costs 9 EUR, 100 MB costs 20 EUR, and over 100 MB, you pay 1 EUR/MB (!). SFR:
See page 40-41 of this document. You pay 0.50 EUR for a 30 minutes session, with 2 MB of data included. After those 2 MB of data, you pay 1 EUR/MB. Those rates are really crazy. Using a mobile phone on an HSDPA network (called “3G+” in France), you can easily reach 1 Mbps. (and I did see this rate while fetching files using rsync, so I was not particularly aiming at performance!). 1 Mbps translates to 7.5 EUR/min using the 1 EUR/MB rate. Or 12.5 cts/sec!!! Surprisingly, Orange (ex-France Telecom), which is usually not the most innovative MNO and ISP in France, saves us. With Orange, you pay 0.5 EUR for a 20 mins session, with unlimited volume (sessions called “session multim dia” by Orange). I was so surprised that I called the assistance to check. And after trying it yesterday, I was a bit anxious when checking my online bill this morning. But it works! Important note before you try (from Orange website):
La session multim dia est une tarification valable pour les clients forfaits mobile Orange (hors forfaits Orange pour iPhone, Mobicarte et cartes pr pay es). Pour les clients Classique, Intense, Pro, Click le forfait et Initial dont la souscription ou le r engagment est ant rieur au 14 juin 2007, pour les clients forfaits bloqu s dont la souscription ou le r engagement est ant rieur au 16 ao t 2007 et pour les clients b n ficiant d une offre blackberry, l accord expr s de l abonn b n ficier de la session multim dia est n cessaire. Les autres clients b n ficient de la tarification au volume.

Rob Bradford: GUADEC Flickr group

Despite a couple of false starts with the Flickr group (thanks Claudio for nudging me), the group should now be ready to accept your photos!

Andrew Pollock: [life] She's unstoppable

Sarah just got her University results for semester 1. A distinction and a credit. That was starting the semester recovering from heart surgery. I'm very proud of her. She's doing three subjects next semester, and then she's finished first year. Pretty impressive progress really. I know I wouldn't stand a snowflake's chance in hell of doing well studying externally.

10 July 2008

Steve Kemp: He'll save every one of us!

So recently there have been a few posts by people discussing the idea of distributed bug reporting systems. This is topical, becuse I've been working on something vaguely related - a support system. Why is a support system, or ticketing system, related to a bug tracker? To answer that you must first clarify what you mean by a support system (or a bug tracker for that matter). There are two distinct types of support systems:
"Full"
This is a system which does "everything". Each submitted support, or bug, request has an associated status, severity, priority and work log. It might be assigned to multiple people through its lifetime, and it might be moved around various internal queues. Example: Request Tracker.
"Minimal"
A reported ticket is an email submission. A "ticket" from start to finish is nothing more than a collection of mails upon a particular topic - there is no assigned "owner", no "priority", no object-specific attributes at all.
I've neither the time, patience, or desire to create any system like the first one. But I've become increasingly dissatisfied with my current support system, roundup, for various reasons. So I need something new. Previously I ruled out several support systems and most of my objections still stand, although I admit I've not really looked again. I will certainly do so shortly. So what have I done? Well I figure I care very little about queues, priorities, reports, and all that stuff. (On the basis that priority is mostly in the ticket itself, and that I'm the sole recipient of the submissions. Previously I had a partner to share them, but these days just me.) I want something that is basically e-mail centric. That reduces the functionality of my support system to two distinct parts:
Ticket Submission
An email to sumbit@example.com should result in three actions:
  • A message being sent back to the submitter. "Hey got your ticket. We'll fix it. Love you long time.". (Set the reply-to address appropriately!)
  • A message to the site admin(s) saying "Hey, you've got a new ticket submission. Deal."
  • A new ticket being created/stored somewhere.
(There are additional nice things to have - such as SMS alerts outside 10:00-17:00, etc, but those are trivial bolt-ons.)
Ticket Updates
When a mail comes in to "ticket-XXXX@example.org" - which was setup in the submission process, or when a mail arrives with subject "[issueXXXX]" we need to append the correspondance to the existing ticket. If that ticket doesn't exist we report an error. If the ticket is closed we re-open it.
So, what do we have? A script that is conceptually simple, and can be invoked via a pipe transport of Exim4. A script that just reads the submitted email from STDIN and only has to decide a couple of things based pretty much on the contents of the "To:" and "Subject:" headers. Simple, no? OK it was actually amazingly simple once broken down. The amazing part is that it all works. I figured that I'd manage new tickets by writing them to mbox files - simple to do. Simple to understand. So the process goes: There are a couple of helper scripts: But basically one script and mutt does it all. So far it's been tested and it is rockin'! (Look at me, hangin' with the cool kids!) Obviously it is pretty Steve-specific, and there is only a toy CGI process for viewing history online, but I'm actually feeling pretty good about it. Once I've closed all my existing tickets I will probably migrate over to it. At least this handles (read: ignores) MIME - which makes it a clear winner of roundup, which just eats, bounces, or corrupts submitted multipart messages. ObQuote: Flash Gordon

Next.

Previous.