Search Results: "cas"

17 February 2008

Neil Williams: Monitoring long running scripts via ssh

I'm trying to work out how to monitor a remote script - in this case a mass build script. The script itself is trivial - a simple for loop in perl. It calls a script from emdebian-tools that calls a variety of other scripts and programs.
What's the best way for me to monitor that script from another machine on the network?
Yes, 'ps wax' is usable but not particularly friendly - the script has to work its way through a list of packages that need building. I want to know which package is being built at the time and the 'ps' output does show this information but only if you recognise parts of the build process for particular packages.
I thought about getting the script to output the current package name to some file in /var/ but that seems a bit of a hack (it's a userspace command and should not need to use sudo or require particular permission setups). Sending email is overkill. A file in /tmp/ is possible, true, but that means adapting the script itself. I'd like a solution that could work with (almost) any package, including binaries, without altering the source code. I've also thought of running the script within screen but then when the machine is in the middle of a compilation, it isn't easy to work out which package is actually being built just from the gcc output, plus there is the continual problem that screen doesn't support scrollback.
I have a feeling this is a potential reinventing the wheel problem so I'm looking for ideas.

Martin F. Krafft: The Penny & Martin adventure: phase 2

Following the first phase, the second and final phase of the Penny and Martin adventure ended last Friday at Wellington airport. I am now 11 500 metres high on my way from Bangkok to Zurich, looking back at seven days on the New Zealand South Island, another in Wellington, and a night in Melbourne. Here s the summary of the 2200km we travelled, with selected photos inline. Penny has more pictures in her Flickr album. My New Zealand beer top-ten, in reverse order: Monteith s Original, Monteith s Black, Mac s Great White, Monteith s Golden, Monteith s Celtic, Monteith s Pilsener, Emerson s Pilsener, Mac s Hop Rocker, Emerson s 1812 India Pale Ale, and (drum-roll): Mac s Sassy Red. Yum! The Emerson beers get no links because the brewery website uses Flash. Thanks, Penny, for a fabulous two weeks! NP: The Flower Kings: Back in the World of Adventures

16 February 2008

Russell Coker: Hot Plug and How to Defeat It

Finally I found the URL of a device I’ve been hearing rumours about. The HotPlug is a device to allow you to move a computer without turning it off [1]. It is described as being created for “Government/Forensic customers” but is also being advertised for moving servers without powering them down. The primary way that it works is by slightly unplugging the power plug and connecting wires to the active and neutral terminals, then when mains power is no longer connected it supplies power from a UPS. When mains power is re-connected the UPS is cut off. Modern electrical safety standards in most countries require that exposed pins of a power plug (other than the earth) be shielded to prevent metal objects or the fingers of young children from touching live conductors. The image above shows a recent Australian power plug which has the active and neutral pins protected with plastic such that if the plug is slightly removed there will be no access to live conductors. I have photographed it resting on a keyboard so that people who aren’t familiar with Australian plugs can see the approximate scale. I’m not sure exactly when the new safer plugs were introduced, a mobile phone I bought just over three years ago has the old-style plug (no shielding) while most things that I bought since then have it. In any case I expect that a good number of PCs being used by Australian companies have the old style as I expect that some machines with the older plugs haven’t reached their three year tax write-down period. For a device which has a plug with such shielding they sell kits for disassembling the power lead or taking the power point from the wall. I spoke to an an electrician who assured me that he could with a 100% success rate attach to wires within a power cord without any special tools (saving $149 of equipment that the HotPlug people offer). Any of these things will need to be implemented by a qualified electrician to be legal, and any electrician who has been doing the job for a while probably has a lot of experience working before the recent safety concerns about “working live“. The part of the web site which concerns moving servers seems a little weak. It seems to be based on the idea that someone might have servers which don’t have redundant PSUs (IE really cheap machines - maybe re-purposed desktop machines) which have to be moved without any down-time and for which spending $500US on a device to cut the power (plus extra money to pay an electrician to use it) is considered a good investment. The only customers I can imagine for such a device are criminals and cops. I also wonder whether you could get the same result with a simple switch that cuts from one power source to another. I find that it’s not uncommon for brief power fluctuations to cause the lights to flicker but for most desktop machines to not reboot. So obviously the capacitors in the PSU and on the motherboard can keep things running for a small amount of time without mains power. That should be enough for the power to be switched across to another source. It probably wouldn’t be as reliable but a “non-government” organisation which desires the use of such devices probably doesn’t want any evidence that they ever purchased one… Now given that such devices are out there, the question is how to work around them. One thing that they advertise is “mouse jigglers” to prevent screen-lock programs from activating. So an obvious first step is to not allow jiggling to prevent the screen-saver. Forcing people to re-authenticate periodically during their work is not going to impact productivity much (of course the down-side is that it offers more opportunities for shoulder-surfing authentication methods). Once a machine is taken the next step is to delay or prevent an attacker from reading the data. If an attacker has the resources of a major government behind them then they could read the bus of the machine to extract data and maybe isolate the CPU and send memory read commands to system memory to extract all data (including the keys for decrypting the hard drive). The only possible defence against that would be to have multiple machines exchanging encrypted heart-beat packets and configured to immediately shut themselves down if all other machines stop sending packets to them. But if defending against an attacker with more modest resources the shutdown period could be a lot longer (maybe a week without a successful login). Obviously an attacker who gets physical ownership of a running machine will try and crack it. This is where all the OS security features we know can be used to delay them long enough to allow an automated shut-down that will remove the encryption keys from memory.

Share This

15 February 2008

Joerg Jaspert: PostgreSQL Continuous Archiving and Point-In-Time Recovery (PITR)

A nice feature from PostgreSQL are “write ahead log” (WAL) files. Quoting from the PostgreSQL website they describe every change made to the database’s data files. […] if the system crashes, the database can be restored to consistency by “replaying” the log entries made since the last checkpoint. This basically enables online backup of your database cluster. And as you end up having the complete log of all actions within your database(s) you are also able to go back in time to any point you like. I won’t write a complete howto about it, as the PostgreSQL docs are pretty good, just some notes how I did my setup, as a kind of reminder for me when I have to do it again.
There must be a place to store WAL files which ends up not being on the same machine, otherwise it doesn’t make much sense. For me that means scp to a different host, but an NFS mount or some storage array or drbd device might also be an option. Much of the following is for the scp method, but easily adaptable to other storage methods. I need a target user on the host I copy files to, and as I did the setup for the pentabarf database running on host skinner, I helpfully named it pentabarf-backup. The copy process can’t be interactive so the postgres user on the database host needs to have a passwordless SSH key to login to the pentabarf-backup user on the backup host. To limit my bad feeling about “passwordless key to login” I restrict the key by adding 
from="skinner.debconf.org,72.32.250.224,72.32.79.241",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
in front of it while copying it to the pentabarf-backup users .ssh/authorized_keys file. I need a few directories, namely pentabarf-WAL-skinner, pentabarf-tarballs and pg_xlog-skinner for the stuff to backup, created in the home of the pentabarf-backup user. Next step is to tell PostgreSQL our wish to have online backups. Add the following statements to your postgresql.conf, adapting the values to your needs: 
archive_command = 'ssh backup test ! -f pentabarf-WAL-skinner/%f && rsync -az %p backup:pentabarf-WAL-skinner/%f'
checkpoint_timeout = 1h
archive_timeout = 12h
The above tells PostGreSQL that i want it to archive the files using ssh to my backup host and transfer files with rsync (via ssh), but only if they do not exist yet. It also tells it to archive the files at least once every 12 hours. The effect is that WAL files get copied over when they are “full” or after 12 hours passed, whatever happens first. Don’t set it too low, as the file will always have the full, fixed size and you waste space transferring mostly-empty files. In case you are concerned with the possibly slow speed of a network link, the PostgreSQL site has the following to say about this: The speed of the archiving command is not important, so long as it can keep up with the average rate at which your server generates WAL data. Normal operation continues even if the archiving process falls a little behind. I haven’t had a problem with the speed yet. A 16MB file every (at least) 12 hours isn’t that bad. Now that we have done all the setup and are able to transfer WAL files, we need a base backup to start with. Issue a SELECT pg_start_backup(‘BASE_BACKUP’); as a database superuser and then backup the database cluster by backing up the whole clusters data/ directory (and every other location if you use tablespaces). When you are done issue SELECT pg_stop_backup(); and you are done with your base backup. You might want to exclude pg_xlog/ to save space, you don’t need it.
As I’m lazy I use a script (written by Peter ‘weasel’ Palfrader), you can get it here, but whatever you take is fine. If you use the script just make sure to adapt the variables in the script to values that fit your system. From now on you should do such base backups in regular intervals, I run them once a week, as recovery does need the last base backup plus all WAL files from then on. Doing regular base backups limits the amount of WAL files you need to keep around / you need to replay.
Reading the above you might find that there is a window of up to 12 hours from which I do not have backups. True, and to make this window way smaller, I am using another script (also written by weasel), get it here, together with an init script. This synchronizes the pg_xlog/ directory to my backup host every 5 minutes, so in case of a server crash I can replay everything up to the last 5 minutes. Recovery using WAL files This is pretty simple, and also good documented at the PostgreSQL site again. For my scenario it basically boils down to
  1. Stop postgres
  2. Move away the cluster. 
    mv /var/lib/postgresql/8.2 /var/lib/postgresql/8.2_old
  3. Restore the backup dump, be careful to have the file rights correct. For the above described setup that means copy and untar the latest tarballs created weekly.
  4. Recreate pg_xlog: 
      mkdir /var/lib/postgresql/8.2/main/pg_xlog
  chown postgres:postgres /var/lib/postgresql/8.2/main/pg_xlog
  chmod 40700 /var/lib/postgresql/8.2/main/pg_xlog
  mkdir /var/lib/postgresql/8.2/main/pg_xlog/archive_status
  chown postgres:postgres /var/lib/postgresql/8.2/main/pg_xlog/archive_status
  chmod 40700 /var/lib/postgresql/8.2/main/pg_xlog/archive_status
  5. Copy possibly unarchived WAL files from /var/lib/postgresql/8.2_old/main/pg_xlog into the newly created pg_xlog.
  6. Copy all the WAL segment files that are needed over from the backup host. That should be all the files on the backup host, directory pentabarf-WAL-skinner/. You should be able to limit it by taking all those files >= the latest *.backup file.
  7. Create a recovery command file recovery.conf in the cluster data directory. You may also want to temporarily modify pg_hba.conf to prevent ordinary users from connecting until you are sure the recovery has worked. The content of recovery.conf is 
          restore_command = 'cp /tmp/pg-WAL/%f "%p"'
    Make sure that the file location you specify is the location you put the WAL segment files into in the prior step.
  8. Start the server, it will go into recover mode and proceed to read through all the archived WAL files it needs.
  9. Look at the database and decide if you like what you see.
Of course the procedure can be slightly adapted, so you end up with a database state from 2 weeks ago. Or 3 days. Or whatever you may need (and have WAL files for). There are many possibilities, and there is a lot of text on the PostgreSQL page that I don’t want to duplicate here, so please read it. Also, I’m not using this as the sole and only way to backup my database(s), they are all still dumped into SQL files every day. This is just additional.

Stefano Zacchiroli: XQuery and XSLT 2.0 with Debian

XQuery 1.0 and XSLT 2.0: news and how to use them in Debian (long post about the second generation of the XSL family and how to use the related languages in Debian)
The 2nd generation of the XSL family At the beginning of 2007, W3C released a family of interrelated specification: XPath 2.0, XSLT 2.0, XQuery (1.0). The 3 specifications are based on the very same underlying data model which (finally!) supports typed values and exploits them to some extent, for example permitting some form of static type checking in XPath and XQuery. Types usually come from XML Schema, with both built-in and user-defined types supported. A brief overview of the 3 specifications and what they change in the state of the art follows. XPath 2.0 XPath 2.0 basically consists in XPath 1.0 + 3 new macro features. The first one is a revamp of those language features which are dealing with types, in order to better integrate them with the new typed data model. XPath now has operators to check whether a value belongs to a given type (so yes, type information are kept at run time), and to cast them from one type to another. The basic new type constructor is now the sequence, which replaces the old node-set solving many annoying issues, such as the impossibility of having a node ordering other than document order. The second feature is the improvement of standard library of functions which was ridiculously small in XPath 1.0; it is much better now. Additionally, 2nd generation languages (XSLT 2.0 and XQuery) now support the ability to define functions which will then be visible to inner (XPath) expressions, pushing yet forward the possibilities of plain old XPath. Functions can now also declare types in their signatures (for their arguments and return value) and untyped arguments will be automatically casted to them upon invocation. Hence, even if you are not using an implementation which assigns types to your XML trees, once you "enter" the typed world calling a typed function (almost all standard library functions are decently typed) you will be able to stay there avoiding annoying casts everywhere. Finally XPath 2.0 has turned into a powerful purely functional language and is now powered by constructs like conditionals, for-each loops, existential/universal quantifiers, and existentially-quantified comparison operators for sequences. Here is a complex expression to hwet your appetite (or scare you away ...), comments come as (: smiley faces :) 
for $book in /bookshelf//books
return
  if ((every $author in $book/authors/author
       satisfies $author/nativeLang eq "it_IT")
      and $book/lang eq "it_IT")
  then $book
  else ()
(: think about the trouble of writing this in XSLT/XPath 1.0 ... :)
XSLT 2.0 XSLT 2.0 is what I would call a "bug fix release" of XSLT 1.0 + the routinary reworking of the language to deal with typed values, which is not sensibly different than what has been done for XPath 2.0. The fixed "bugs" are several, starting from the annoying issue of result tree fragments. They are basically tree snippets that in XSLT 1.0 you were able to generate for future use. Unfortunately they were not thaaat reusable, given that you were not even able to navigate them with XPath operators! Now the specification is much more clear and distinguishes final result trees from ordinary variables, which can now contain sequences of (navigable) tree nodes. Another important "bug" fixed is the new ability to output multiple documents with a single XSLT stylesheet: it marks the end of stupid extra post processing to be added in pipeline to a XSLT processor. Other minor "bugs" fixed are a limited amount of backtracking capabilities among imported templates, regular expression support directly in the language, and powerful grouping constructs on the lines of SQL's GROUP BY (but much more powerful). Here is a template snippet exploiting the latter feature: 
<xsl:for-each-group select="*" group-starting-with="h1">
  <div>
    <xsl:apply-templates select="current-group()" />
  </div>
<xsl:for-each-group>
XQuery XQuery is the end of the chains imposed by XML-based syntaxes. Why the heck one has to use an XML syntax (as in the above snippet) only because she is manipulating XML tress is one of the mysteries of XML technologies which have always been floating around in my head. XQuery is the (supposedly) SQL equivalent for databases of XML documents, but is actually much more than that. I depict it in my head as the XML manipulation language with a syntax I can finally stand. Technically it is XPath 2.0 (say 80% of the whole language) + some extra ingredients (say 20%); so remember that every XPath 2.0 expression is also a XQuery expression. The main extra ingredient is the so called FLWOR expression (to be read: "flower expression", which in addition to the "smiley faces" used for comments gives a "back to 1968"-flavour to the language .... erm FLWOR/flower/flavour, no pun intended). A FLWOR expression is very similar to SQL's SELECT-FROM-WHERE: it lets you generate a tuple stream by iterating on sequences (F: for clauses), binding expression values to names (L: let clauses), filter out tuples which do not satisfy a required condition (W: where clause), order the survived tuples (O: order by clause), and finally return a sequence built using the residual tuple stream (R: return clause). The other interesting extra ingredient is the ability to build the XML snippets you want to manipulate. Within XQuery you do that using plain XML syntax (the only place where a sane-minded programmer actually wants to see it!) which also supports a classical interpolation mechanism to embed expressions which will be evaluated inside XML snippets, and also the other way around. A canonical XQuery example is: 
for $t in doc("books.xml")//title,
    $e in doc("reviews.xml")//entry
where $t = $e/title
return <review>  $t, $e/remarks  </review>
(: braces denote the escaping context where XQuery expressions will
   be evaluated inside snippets; plain XML syntax is used for the
   other way around :)
But remember: XQuery for XML is much more than SQL for RDBMS, thanks to the implicit templating mechanism implemented by interpolation, and thanks to several language features fostering modularity (user-defined functions, library modules, XPath 2.0 standard library, ...) you can basically do with it any kind of XML manipulation you can imagine. I don't think I will ever write myself any other single line of XML output in DOM or XSLT ...
Cool, how can I use it in Debian? ... if only this stuff were decently supported in the open source world. Last time I checked, the author of most parts of the GNOME toolchain for dealing with XML (libxml2, libxslt, ...) was not intentioned to implement XSLT 2.0, not even mentioning XQuery. This comes as no surprise, the whole GNOME XML toolkit is written in C, and XSLT 2.0 / XQuery have reached a level of complexity and formal specification which usually entails a higher level approach. So on the GNOME side we are stuck. The other open source implementations of XSLT 2.0 / XQuery I'm aware of are Saxon and Galax. Saxon is an XSLT 2.0 and XQuery implementation written in Java, which is unfortunate per se. Additionally, it is also unfortunate that it is only partially open source. Indeed, Saxon is split into SaxonB (for "basic") which is open source under the Mozilla Public License and SaxonSA which is commercial. While SaxonSA is a fully conformant, XML Schema-aware processor, with support for static typing, SaxonB is a basic-conformant processor with no type-aware features and actually much less optimized than SaxonSA. This is annoying. (Much more annoying is the fact that SaxonSA's author is the only editor of the XSLT 2.0 specification and that instead of his mail address the specification includes an URL pointing to the website selling SaxonSA ...) Galax is an open source (IBM CPL / Lucent license) OCaml implementation of XQuery which is not fully conformant to the specification (though it gets quite close) which is type-aware and implements static typing. The only XQuery / XSLT 2.0 implementation available in Debian at the time of writing is SaxonB. The binary package is libsaxonb-java, kudos to Michael Koch and the Debian Java Maintainers for having packaged it (and to have stood some annoying pings of mine :-) bug #408842). To execute XQuery code you just have to aptitude install libsaxonb-java, prepare a query.xq file containing your query, and then execute something like: 
CLASSPATH=/usr/share/java/saxonb.jar \
java net.sf.saxon.Query query.xq
note that the information in README.Debian are still referring to old Saxon versions, see bug #465894 which proposes a more up to date README.Debian. Similarly, to perform a XSLT 2.0 transformation you have to do something like: 
CLASSPATH=/usr/share/java/saxonb.jar \
java net.sf.saxon.Transform -ext:off -s:input.xml -xsl:style.xsl -o:output.xml
Do not remove the -ext:off flag when processing untrusted stylesheets!, see bug #465885 for the reason. I've written some handier (1-liner) shell script helpers which remove the need of invoking java manually. They are attached to bug #465894 and I've proposed their addition to the saxonb package. Using them the above invocations become: 
saxonb-xquery query.xq
saxonb-xslt -ext:off -s:input.xml -xsl:style.xsl -o:output.xml
Galax in Debian ... well, since long time I've been planning to package it (the ITP has been filed some months ago: bug #447984) and the authors sent me a newer version than what is available online to gather feedback before the long overdue final release. Unfortunately I've been lagging behind in finishing the packaging (which is tricky due to the need of binding libraries to several different languages: OCaml is native, but there is also Java for example). Hopefully this post will give me some renewed motivation for finishing the work ...
References
Acknowledgements Thanks to godog for his helpful comments.
Update: the helpers I've proposed have been accepted into the official package, I've also made available their manpages. Also the pending changes to README.Debian has been accepted. Kudos again to Michael Koch for his quick feedback on my patches!

MJ Ray: ISPs should not be forced to disconnect filesharers

Requiring Internet Service Providers to bar users for illegal downloading is stupid because:- 1. ISPs can't tell what we're sending over the network and aren't in a position to verify whether we have permission to send it. However, cutting off small businesses and home offices could kill some businesses. I own a share of my ISP (one of 6000 or so equal shares) and would vote against opening it up to that liability. I'm not surprised that ISPA has already said ISPs want to pass any liability back to the accusers. 2. government sometimes doesn't understand what can be downloaded legally. There's a famous case where a Trading Standards Officer complained to a Mozilla Foundation worker about their free Firefox browser because
"it makes it virtually impossible for us, from a practical point of view, to enforce UK anti-piracy legislation"
Source: Times Online 3. there is permission for "fair dealing" in copyrighted material for various purposes, including current affairs reporting and education, unless someone changed the law since I last looked! However, some organisations, including the BBC's 100-million-pound iPlayer, try to deny viewers those permissions, so it is easier to start from illegal copies than it is to deal with iPlayer (only works on some computers, installs bandwidth-using code into the system startup and so on). Finally, where did the 6 million illegal downloaders figure come from, ultimately? The film and music dominant producers? The Recording Industry Association of America has already spoken out against legally-required filtering, in http://www.news.com/8301-10784_3-9861460-7.html?part=rss&subj=news&tag=2547-1_3-0-5 Our software is available for free download and free sharing. The music and movie industries need to adapt, not be propped up by making ISPs pay through this sort of bad law. This isn't so much ISP disconnect as BPI logical disconnect!

14 February 2008

Julian Andres Klode: Ubuntu: aufs with casper 1.118 / ndisgtk 0.8.1 / ndisgtk in main?


Colin Watson today uploaded casper 1.118, now supporting aufs. To use aufs in future Ubuntu disks built with casper 1.118 or newer, use union=aufs. Please test it. Another upload today was ndisgtk 0.8.1-1ubuntu1, bringing Ubuntu up-to-date with Debian and closing 3 bugs. BTW, I requested to include ndisgtk in main, see https://wiki.ubuntu.com/MainInclusionReportNdisgtk. Having ndisgtk on the Ubuntu disks would be very useful for users without linux network drivers available and without enough experiences to use ndiswrapper from the commandline.

Adeodato Sim : Hating the Spanish voting system

I’m obviously not discovering the Americas here, given the sheer amount of research, theory and whatnot one can find for this topic, but I need to vent. Also, please excuse my terminology and my bias. In Spain we have two main political parties, the one that advertises itself as the rightish-center option, but is sliiiightly more to the right than that (PP), and the one that wears socialism in its name, but is sliiiightly less to the left than that (PSOE). Then there are some minority parties, like the nationalists, and United Left (IU). So, ideally, my ballot -la-Debian would always be: 
  [1] IU
  [2] PSOE
  [4] PP
  [3] None of the above
But sadly you can’t vote like that in the real world (?), and can only pick one. Without a fscking second round. So every election I’m left wandering in the cold land of tactical voting. One last interesting bit: when talking about my wishes for a two round system with a good friend, who also happens to be a PP fanboy, he said: “But with that system PP would never govern!” And I think there’s some truth to that.

12 February 2008

Erich Schubert: New flash, usability++

With the latest flash for Linux update, two of the biggest annoyances have disappeared. Yay for usability!It still sucks, and I'm only using it in rare cases by selectively loading Flash. I love FlashBlock, which replaces any flash crap with a 'Play' button. Now if we could only do the same for some of the Web 2.0/DHTML annoyances...[Note: it's not a particularly new version. r115 according to about:plugins. I was told it's like 3 months old. I just noticed the difference, maybe cause the Debian flash-installer package was just updated to this release, or because I use Flash so little. It was so unreliable for like ever that I wouldn't even start using any site that uses Flash for anything serious. Granted: I do use youtube, because there are some great Swing dancing clips there. But that only uses a tiny fragment of Flash but still has been killing my browser every now and then.][Update: I was misled by a clever Javascript that actually disabled the flash or shrank it or so when the menu popped up. E.g. the Toyota site http://www.toyota.com/corolla/ is still unusable unless you're using some flash blocker or do not have flash installed (apart from the flash assuming it were transparent, but it's rendered on all white). The menu still pops up under the flash, making it impossible to access many pages.]

11 February 2008

Biella Coleman: From coast to coast

One of my favorite radio stations, KEXP, based in Seattle is coming to NYC by parenting with a local station. Not much will change except that we will have the incredible John in the Morning for some of his time and he will, according to their website, scout out local talent:
Does this mean that John in the Morning is leaving Seattle?
No. Starting in June 2008, John will split time between Seattle and New York, broadcasting his show live from both locations throughout the year. While he s in New York, he ll be searching for new bands and discovering new music that The Morning Show listeners will be the first to hear.

Anand Kumria: No so grubby.

Wanting to help along GRUB2 adoption as per Jordi's suggestion. I tried out apt-get install grub-pc. I'm not so sure that a bugreport entitled Installing grub-pc achieves nothing would be so useful. Here is what seemed to happen: 
eve:[~]% diff -u /boot/grub/menu.lst_backup_by_grub2_postinst /boot/grub/menu.lst
--- /boot/grub/menu.lst_backup_by_grub2_postinst        2008-02-09 18:59:46.000000000 +0000
+++ /boot/grub/menu.lst 2008-02-09 18:59:47.000000000 +0000
@@ -9,7 +9,7 @@
 #
 # You can specify 'saved' instead of a number. In this case, the default entry
 # is the entry saved with the command 'savedefault'.
-default                saved
+default         0saved
 ## timeout sec
 # Set a timeout, in SEC seconds, before automatically booting the default entry
@@ -113,6 +113,11 @@
 ## ## End Default Options ##
+title		Chainload into GRUB 2
+root		(hd0,0)
+kernel		/boot/grub/core.img
+savedefault
+
 title		Debian GNU/Linux, kernel 2.6.24-1-686
 root		(hd0,0)
 kernel		/boot/vmlinuz-2.6.24-1-686 quiet root=/dev/hda1 ro resume=/dev/hda5 usbcore.autosuspend=1
Booting into the chain loaded GRUB2 and I get a nice prompt with no idea what to do next.

Steve Kemp: A good cockerel always points north

I spent a while yesterday thinking over the software projects that I'm currently interested in. It is a reasonably short list. At the time I just looked over the packages that I've got installed and the number of bugs. I'm a little disappointed to see that the bugfixes that I applied to GNU screen have been mostly ignored. Still I have the day off work on Thursday and Friday this week and would probbly spend it releasing the pending advisories I've got in my queue, and then fixing N bugs in a single package. The alternative is to build a quick GPG-based mailing list manager. I'd like a simple system which allowed users to subscribe, and only accepted GPG-signed mails. The subscriber could choose to receive their messages either signed (as-is) by the submitter or encrypted to them. So to join you'd do something like this: 
subscribe foo@example.org [encrypted]
--BEGIN PUBLIC KEY --
...
--ND PUBLIC KEY--
There is the risk, with a large enough number of users, that a list could DOS the host if it had to encrypt each message to each subscribers. But if the submissions were validated as being signed by a user with a known key it should be minimal, unless there is a lot of traffic. The cases are simple: There are some random hacks out there for this, including a mailman patch (did I mention how much I detest mailman yet today?) but nothing recent.

9 February 2008

Russell Coker: The Bill - Computers for Police

I was watching the British police drama show The Bill [1] and I was impressed by their use of computers. They were analysing the evidence of a homicide and one of the tasks was to assemble a time-line of the related events. They had a projector connected to a computer which displayed the data and used what was apparently an infra-red pen (presumably similar in technology to Rusty’s infra-red pen for Wii-Mote Pong [2]) to write on the wall and the computer then performed OCR to convert it to printed text. When new evidence was discovered about the time of events they used drag and drop to move the events around to the correct time slots. I’ve seen many horribly faked renditions of computer stuff on TV, but this seemed quite realistic. It was all technically possible, in many cases what was displayed would be quite inconvenient to fake if you didn’t have software to do it, and little touches like closing the session when the case was resolved are things that you wouldn’t bother with if faking it. The down-side to my analysis is that I had a couple of glasses of red wine so I might be more gullible than usual. ;) Some years ago I swore off watching The Bill after a horribly stupid episode about computer crime (teenage haxor sells copies of cracked software to computer store owner for resale to the public, then computer store owner murders him for disrespect and price-gouging). I refrained from watching for almost a decade after that. But it seems that they have redeemed themself to some degree. I wonder whether the software they were showing is really used by law enforcement, or whether it’s designed for corporations to use in tele-conferences etc. In the same episode they showed the recordings of two security cameras on projectors side by side for comparison, they could fast-forward them independently but they couldn’t zoom in or do other silly and impossible things.

Share This

Daniel Stone: ionish

Dear lazyweb,
I want to try out a window manager like Ion, but I don't want to actually use Ion due to the fact the author's an irritating nutcase who I'd rather not encourage any further, and its utterly daft license. What's a man to do?

Stefano Zacchiroli: installing debian on zonbu mini desktop

installing Debian on Zonbu mini desktop ... ... it's trivial! Me and godog have bought a Zonbu Mini Desktop to replace our previous noisy caseful server. The supposed duties of such a server are just running some P2P clients and a slimserver for a Squeezebox (so yes, the Zonbu will also need to proxy a very big USB disk). I've discovered the Zonbu mini thanks to a blog post by caesar, and I'm quite happy about the hardware. It's absolutely noiseless, small, eye catching. Though I don't need particular performances, the only aspect that worried my was the I/O on the CF card, which apparently is a non-concern. The only missing bit in the hardware is a video output (only VGA is provided), a must if you want to use the Zonbu as a multimedia station. But this kind of usage wasn't on my plans, so ... FWIW, before buying the Zonbu I've also tried to evaluate the cost of building something similar by myself, but the only cost of the mother board almost dominates the final price of the Zonbu. So apparently it's a good device also price-wise. I haven't even tried the on-board Gentoo-derived installation. Well, in fact I had, but considering that the only monitor I've at home is a B/W 15 years old CRT, my chances to see something vanished at the precise moment the frame buffer started, of course they did not get any better when X started. The minute after that, I've fired up D-I.
Debian Installer vs Zonbu mini Installing Debian on the Zonbu mini is actually trivial: /me happy about the new geeky gadget
Sad note: according to godog's tests, the provided wifi (which is optional to buy) works either with ndiswrapper or with a binary kernel module built against Fedora kernel, that sucks.

8 February 2008

Lucas Nussbaum: dd + /dev/random fun

From Adrien Lebre, who seems to be having fun working on kDFS: 
$ dd if=/dev/random of=file bs=1024 count=2
0+2 records in
0+2 records out
256 bytes (256 B) copied, 0.000705717 s, 363 kB/s
$ ls -l file
-rw-r--r-- 1 lucas lucas 256 2008-02-08 17:53 file
Can be easily explained using strace: 
open("/dev/random", O_RDONLY O_LARGEFILE) = 0
[..]
open("file", O_WRONLY O_CREAT O_TRUNC O_LARGEFILE, 0666) = 1
[..]
read(0, "p(361241,360-330~M245y10=274U201c207v336a273"..., 1024) = 128
write(1, "p(361241,360-330~M245y10=274U201c207v336a273"..., 128) = 128
read(0, "362,LW5l.?2422225222320637510326335316374372"..., 1024) = 128
write(1, "362,LW5l.?2422225222320637510326335316374372"..., 128) = 128
close(0)                                = 0
close(1)                                = 0

Gunnar Wolf: cat STDERR rot13

Cannot help but laugh and share.
I've been triaging and trying to reproduce some oldish bugs on pkg-perl's packages. Some bugs are no longer there, some have to be forwarded upstream, and so on. Usual tasks, yes.
Until I stumbled with #406227. I just have to laugh and share! Hope nobody feels ashamed - The bug is the result of different people coding maybe under pressure and with quite different mindsets :)
For some reason I fail to understand, the submitter's test case (rot13 implemented over a HTTP proxy) is invoked in the report as ./rot13 2>/dev/null. Of course, when trying to debug a bug report, the first thing to do is not to ignore STDERR. So, off goes the 2>/dev/null. What happens next? 0 gwolf@mosca[2]/tmp$ perl ./rot13 &
[1] 4394
0 gwolf@mosca[3]/tmp$ GET -p http://localhost:8080/ http://www.debian.org/
Can't locate object method "filter" via package "UGGC::Cebkl::ObqlSvygre::fvzcyr=UNFU(0k604160)" (perhaps you forgot to load "UGGC::Cebkl::ObqlSvygre::fvzcyr=UNFU(0k604160)"?) at /usr/share/perl5/HTTP/Proxy/FilterStack.pm line 126.
500 EOF when chunk header expected WTF... Well, at least the program name gives me a clue... Lets try to "decrypt" the error message... gwolf@mosca[4]/tmp$ echo 'UGGC::Cebkl::ObqlSvygre::fvzcyr=UNFU(0k604160)' rot13
HTTP::Proxy::BodyFilter::simple=HASH(0x604160) hrm... How comes the filter is filtering its own code and only then refusing to find itself!? Ok, time to open up the manpage - Remember, I'm only group-maintaining this pacakge, I am not yet at all familiar with it! Ok, so the core of the filter is when the submitter states: my $proxy = new HTTP::Proxy();
$proxy->push_filter(response => new HTTP::Proxy::BodyFilter::simple(sub tr/a-zA-z/n-za-mN-ZA-M/; )); While the manpage states it should be invoked as: my $filter = HTTP::Proxy::BodyFilter::simple->new( sub $ $_[1] =~ s/foo/bar/g; );
$proxy->push_filter( response => $filter ); Of course, once looking at it, the answer is simple: The submitter left out which element to act on in the anonymous function body - The $ $_[1] =~ part. Adding it makes gur svygre jbex nf rkcrpgrq... Err, sorry - makes the filter work as expected. Now, bonus points: For the non-Perlers out ther: How come we get the namespace translated as well? Oh, that's very simple: In Perl, as in Python (and concievably other languages I'm unaware of), the object is passed to any of its methods as the first argument. Functions in Perl receive their arguments via @_ (read: the default array). And, of course, the tr (regex-based transliteration) takes by default the first thing it sees - the object itself. And what happens when you apply a (string-oriented) regex to an object? Of course, it gets stringified - which, by default, in Perl means converting it to the closest possible description: "a hash reference blessed as an object of the class such-and-such at this memory location". That string gets worked on, and we get UGGC::Cebkl::ObqlSvygre::fvzcyr=UNFU(0k604160). This proxy does not die on very-very-short web pages, where the whole content fits on one iteration of the code (although it does not work correctly - the text remains unaltered, of course, as it was not worked on), but if the request spans several chunks, the second time the filter is called, it will be... just gibberish. Oh, and what about the extra $ (...) around $_[1]? Oh, simple: The string is passed as a scalar reference, so it can be modified in place. Yes, it's the Perl way of pass-by-reference instead of pass-by-value (the default behaviour): Of course the parameter is only passed as a value. Only that the value is incidentally a reference - but who cares? ;-) Anyway... Many oddities. I would implement the module in a completely different way, and it looks quite backwardish in my book. But then again, TIMTOWTDI.

Mohammed Sameer: Automatically insert a statement in each function in C++

And I thought I'll never do such things again ?
I wanted to insert a debugging statement at the beginning of each function.

for i in *.cc *.c; do sed -i -e 's/\([a-zA-Z0-9 _*:~]*([^).]*) * \)/\1 \nprintf("%s\\n", __PRETTY_FUNCTION__); /g' $i; done

It can probably be used to insert anything too.

It's dirty but it worked for me. Just keep a backup or make sure the latest code is in any RCS just in case.

It only failed with a function that was commented using C++ style comments "//".
I didn't try it with class methods defined inside header files read more

7 February 2008

Russell Coker: Political Blog Posts

Currently in the US the main political parties are deciding who will contest the next presidential election. Naturally this gets some commentary from all sides. Planet Debian has syndicated two blog posts commenting on these issues, it’s interesting to compare them: First John Goerzen writes a post about an issue he (and almost everyone in the Debian community) considers important - copyright law [1]. He quotes the Randall Munroe who is the author of www.xkcd.com (described by the author as “A webcomic of romance, sarcasm, math, and language“) which is wildly popular in the free software community (I believe that there is a fan-club which has meetings). Randall’s commentary is quite interesting and I recommend reading it [2]. The most noteworthy fact is that Barak Obama has sought advice from Lawrence Lessig - who has done more for the free software community than any other lawyer. John’s post doesn’t contain much original content, but citing a lengthy post from a highly regarded source and quoting a particularly relevant paragraph make it very noteworthy. Next Jaldhar Vyas writes about “the war against Islamic terrorism” [3] and makes the claim “we are finally gaining the upper hand in Iraq“. To consider that claim it’s best to read some expert commentary. William S. Lind [4] is a widely respected conservative who is an expert on military strategy, his comments about stabilising Iraq are always interesting [5]. It’s sufficient to note that William has been predicting failure for the US occupation of Iraq right from the start and that events over the last five years have proven him correct. William S. Lind’s thoughts on Democracy are interesting too [8], I don’t agree with him in this regard (I generally disagree with him on most things other than military matters) but it’s an interesting thought. Jaldhar also claims that “Democrats on the other hand have a closer association with the media who are behind many of the sillier IP laws“, this seems to be a reference the the “liberal media” conspiracy theories. There seems no clear party association with bad IP legislation (both the Democrats and Republicans do silly things in this regard), and debunking the liberal media claims can be done simply by watching some TV (choose a couple of random news shows on different channels). Fox is not the only right-wing news outlet. Jaldhar also has a conspiracy theory about the Supreme Court and the “need to ensure that the court does not veer leftward again as it will inevitably do during a Democratic administration“, not that the Democratic party is particularly “left” by whatever definition you might apply to the word. I prefer the www.politicalcompass.org analysis which shows that all the serious contenders in the Democratic primary are Authoritarian Right [6]. So the result of the next US presidential election will determine how authoritarian and right-wing the government will be, the fact that it will be authoritarian and right-wing is beyond doubt. That said it would be good if the authoritarian right-wing government that we (*) get at least has some decent policies in regard to IP law (which is the hope for Barak Obama). John Nichols has an interesting analysis of Ann Coulter’s support of Hillary Clinton over John McCain [7]. It would be interesting to see Jaldhar’s comments on this issue. (*) I use the term “we” when talking about US politics to acknowledge the fact that the Australian government will take orders from Washington. Some pundits predict that the Greens will be the second major party in Australian politics in a couple of elections time, so maybe in 8 years time we will have an Australian government that represents Australians instead of Americans. But for the entire duration in office of the next US president (be it 4 years or 8) they will be able to get the Australian government to do almost anything that they desire.

Share This

Biella Coleman: On Confidence, Geekdom, and Desire

So a few folks left some interesting comments in response to my link to the article on the rise of the alpha-girl based on the research of Harvard psychologist, Dan Kindlon. My response to both Joe and Karl is that it is worthy to lower the barriers to entry not because girls will change the cultural ecology of geekdom in positive ways (though they may) or because geekdom is inherently “omg totally awesome!!!!!!!!!!!!!!!!!!!!” but for a much more mundane reason: it may allow girls to follow interests and eventually discover a passion. I think it is a great idea if more women were involved in geekdom and open source development not because it is inherently great but because I am sure there a lot of women out there who would enjoy it (and by extension, I am sure a lot who would not). By highlighting this article I am not advocating some forced hacker-female-labor-migration-policy but pointing to the fact that social conditions outside of geekdom play a role in ensuring more participation among women. But let me explain a few things before I argue my position a little further. First a little clarification, especially now that I have read the article. By alpha the author does not mean dominant, aggressive, and totally jerky, which is what might immediately come to mind when you read/hear alpha (especially if one knows any alpha-geeks or alpha-lawyers for that matter, who are much worse, imho). And I certainly don’t want any more alpha-anything (alpha-women, alpha-dogs, alpha-cats, alpha-hamsters ) given our world is already chock-full of alpha-jerks. By using the term, the author basically means confidence, which apparently is on the rise among girls, leaving them much more likely to be go-getters, undaunted by explicit and implicit barriers and obstacles and thus more likely then ever to enter arenas that have been thought of as traditionally male or in fact totally dominated by males. If this trend is indeed in place, I think it is great and I hope that this eventually translates into more girls/women populating traditionally male areas whether it is neurology, movie directing, hacking, or surfing. What I appreciate about the article is that his research can help us question the idea that women are naturally averse to competition due to their estrogenic hormones while boys, brimming with testosterone, just love it and exude it. I never identified with that conventional well-worn script, in part because I guess I am fairly confident and somewhat competitive. Being in academia for the last 10 years, I have witnessed a lot of really confident women that have helped inspire me and keep me going, especially when times got rough. What is fascinating about the article (and by extension probably his book) is that gender parity and equity and changes in the psychological makeup of women have not changed overnight but have taken a much longer time to settle in place. We are only now bearing the fruits of structural and educational changes first instituted decades ago and that tackled some serious forms of discrimination. As a result, we are seeing girls and women donning a deeper confidence that may help them participate with more success in the arenas they want to whether it be sports or in the workplaces (though there are certainly still major barriers and issues, which are addressed in the article). So why would someone want to participate in the world of free software and hacking in particular? My first answer is there may be any number of reasons why anyone, female or male or transgender or whatever, may want to do so for the love of technology, to feel an intense belonging to a community who share your passions and who work together to create something with a lot of value, to enjoy the challenge of learning, to spread freedom to every corner of the earth, etc… It is not that geekdom is inherently fun and exciting but that it will be interesting to some slice of the population men or women (or other) and what is the harm in lowering the barriers to entry, especially if it brings enjoyment and frankly a lot of economic security too? There are a number of girls/women who already find it worthwhile and I am sure many more who would. And the point is not to create some policy to make sure that we shuttle women into hackerdom but as a society we should equip them with the necessary psychological tools so that if they think this is worthwhile, and discover that they love it, etc, they will dare to venture in there and more importantly, stay if they want to. In terms of Karls point that a lot of people are professional geeks in part because they d have trouble being anything else; like being gay, it s not a choice, I think that is far too narrow of an assessment, not to mention an outdated caricature of geeks. While there is certainly a class of socially awkward geeks (if that is what you mean??), I would say they are in the minority though they may certainly stand out precisely because they are the odd ducks and because the stereotype is so entrenched. Many geeks I have met, while they may pretty darn focused on geekdom, also have full and rich lives/personalities that cannot be easily collapsed into one immutable personality type. Yes they may be obsessed with tech but aren’t doctors, academics, musicians, lawyers sort of fanatical too? I spend like my whole week working on academic stuff (part of necessity, part out of love). And at least hacking has way better conferences and economic perks, which may help explain why people stay :-) Finally, I think you assume a little to strongly that we do what we do because we have a pre-formed existing desired to do so. While I think this is the case for many things (and I knew the minute I learned about anthropology, I wanted to be one, which was odd but it proved to be correct), I think desire is also formed as much through experience and hence the importance of exposure to different worlds and experiences. I know that there are many things I could have never imagined I would have loved–karoake and sailing are two things that come to mind—until I tried them about both took some degree of courage. In the later case, it took a lot of guts to move onto some ship at the age of 18 instead of going to college and I am so glad I did. This is an instance where confidence and an initial curiosity led to discovering a love and passion I never knew I had. So if desire can be formed and not just expressed, I think it is key to make sure people have all sorts of opportunities to cultivate the passions they never knew they had.

Next.

Previous.