Search Results: "will"

15 March 2012

Erich Schubert: ELKI applying for GSoC 2012

I've submitted an organization application to the Google Summer of Code 2012 for ELKI - an open source data mining framework I'm working on.
ELKI
I hope I can get spots for 1-2 students to help implementing additional state of the art methods, to allow for even more broad comparisons. Acceptance notification will be tomorrow. I have no idea how high out chances are of getting accepted. We're open source, as part of the university we have a proven record of educating students (in fact, around two dozen have by now contributed to ELKI, although not all of this has been "released" yet). We're rather small compared to e.g. Gnome, Debian or Apache and just a few years old. But I believe we are trying to fill an important gap in the research and open source interaction: way too many algorithms get published in science, but are never made available as source code to test and compare. This is where we try to step in: make many largely overlooked methods available. Of course we also have k-means (who hasn't?), but there is much more than just k-means! And there are plenty of methods often cited in scientific literature, yet nobody seems to have a working implementation of them ...

While most people equal data mining to prediction and classification (both of which are actually more machine learning topics), ELKI is strong on cluster analysis and outlier detection as well as index structures. Plus, it is much more flexible than other frameworks. For example, we allow almost arbitrary combinations of algorithms and distance functions. So with our tool, you can easily test your own distance function by plugging it into various algorithms. Otherwise we could just have used Weka.
The other reason why we did not extend Weka (or used R) is that we actually wanted to not just implement some algorithms, but actually be able to study the effects of index structures on the algorithms. And in my opinion, this is actually the key difference between true data mining and ML, AI or "statistics". In ML and statistics, the key objective is the result quality. Which is often rather easy to measure, too. For "full" data mining, one also needs to consider all the issues of actually managing the data, indexing it to accelerate computations. Plus, the prime objective is to discover something new, that you did not know before.
Of course these things cannot be completely separated. You can of course discover patterns and rules in the data that will allow you to make good predictions. But data mining is not just prediction and classification!

The next release of ELKI, 0.5, will be released in early april. ELKI 0.4 is already available in Debian testing and Ubuntu. The next release has a focus on comparing clustering results, including an experimental visualization for clustering differences, which will be presented at the ICDE 2012 conference.
On the big to-do list is a lot of stuff. In particular, ELKI is currently mostly useful for research, as it is too difficult to use for the average business guy wanting to do data mining. You can think of it more as a prototyping system: try out what works for you, then implement that within your larger project in an integrated and optimized way. The other big thing that I'm unhappy with is the visualization speed. SVG is great for print export, but Batik can be really slow, and the XML-DOM-API isn't exactly "accessible" to students wanting to add new visualizations. Right now, it is all useful and okay for my kind of experiments, but it could be so much more if we could solve the speed (and memory) issues of the pure SVG based approach. I'd love to see something much faster here, but with a SVG export for editing and printing.

14 March 2012

Stefano Zacchiroli: not a catchy headline

As we're doing trends today, I got curious about Debian. I don't particularly care about popularity contests other than popcon, and Debian choices surely aren't driven by them. But hey, curiosity is curiosity, right? So, here are today's w3techs trends for websites using GNU/Linux, with all lines shown:
Debian: 30%; CentOS: 28,9%; Ubuntu: 18,4%; Red Hat: 12,2%; Fedora: 5%; SuSE: 3%; Gentoo: 1,2%
(As thou shall always read methodologies before stats, here is the technologies overview page.) Debian is the top entrant with 30% of the websites using GNU/Linux. What I find interesting is that Debian has jumped at first place in January 2012, significantly after the release of our current stable release, Squeeze, that dates back to February 2011. It is also interesting to see community distros (Debian and CentOS) starkly ahead of commercially backed distros. That is what I find most fascinating about our projects. Whether that will remain the case or not is, for me, one of the big questions of the decade for the self-determination of Free Software communities.

Erich Schubert: Google Scholar, Mendeley and unreliable sources

Google Scholar and Mendeley need to do more quality control.
Take for example the article
A General Framework for Increasing the Robustness of PCA-Based Correlation Clustering Algorithms
Hans-Peter Kriegel, Peer Kr ger, Erich Schubert, Arthur Zimek
Scientific and Statistical Database Management (SSDBM 2008)
(part of my diploma thesis).
Apparently, someone screwed up entering the data into Mendeley and added the editors to the authors. Now Google happily imported this data into Google Scholar, and keeps on reporting the authors incorrectly, too. Of course, many people will again import this incorrect data into their bibtex files, upload it to Mendeley and others...
Yet, neither Google Scholar nor Mendeley has an option for reporting such an error. They don't even realize that maybe Springerlink - where the DOI points to - is the more reliable source.
On the contrary, Google Scholar just started suggesting to me that the editors are coauthors ...
They really need to add an option to fix such errors. There is nothing wrong with having errors in gathered data, but you need to have a way of fixing them.

Richard Hartmann: Comments

By popular demand, commenting is now enabled. Authentication happens via OpenID only; all comments will be moderated to combat spam. Moderation in ikiwiki seems to happen mainly via the web interface (I know about the RSS) so there may be a certain delay until I approve new comments. If anyone happens to know how to make ikiwiki handle moderation gracefully within a git-based workflow, do leave a comment. That's possible, now :)

Richard Hartmann: FOSDEM 2012, the aftermath

FOSDEM 2012 was very nice, as expected. This year, I decided to stop consuming only and getting more involved. I helped staff the token sale and held two Lightning Talks. This turned out to be an awesome experience. During the beer event, we sold 4600 beer tokens (FOSDEM does not earn anything from this; all proceeds go to Cafe Delirium). The sale was staffed with one to three people at all times which made for a constant but mostly bearable workload. While the others made good-natured fun of me and my German efficiency, I like to think that I helped make things run a tad more smoothly. Along those lines, we will have a sign asking people for exact change and/or to buy stacks that can be paid in bills if possible, next year; that should help streamline the whole thing even more. All in all, it turned out to be quite stressful, but tons of fun. I am looking forward to chuck in again next year. The two Lighting Talks (vcsh on Saturday and git-annex on Sunday) I held very rather well received. Feedback after the talks and online was very positive which is always nice. There's been a slight increase in help requests and generic questions, so I assume people are trying vcsh and git-annex as a consequence of those talks, which is even nicer. Also, I met a lot of people. Most of the meetings were pre-arranged, some by chance. Carrying a name tag at all times, I was approached by several people who knew me online but didn't know my face. I will try to always carry a name tag on conferences now and I suggest others do the same. Finally, a huge thank you to everyone who helped make FOSDEM happen; you rock!

Richard Hartmann: Open Source Days 2012

I will be visiting the Open Source Days 2012 next weekend to talk about how to Gitify your life; if you want to meat (sic) up, do let me know. Compared to FOSDEM, I have 150% of the time for ~250% of the content which means I will need to cut some corners, content-wise. That's a pity, but that's the way it is; we will see how it goes. Never having been there before, I am looking forward to both Copenhagen and the Open Source Days. In somewhat related news, FOSDEM's vcsh talk (slides, video) and git-annex talk (slides, video) have been uploaded completely. Thanks to FOSDEM's/Debian's video team; you're awesome! In less related news, I have had the questionable honour of updating and installing one copy of Windows 7, respectively. I guess it's nice that Intel's driver installation utility tells you it expects a maximum of 12 reboots until a fresh installation is set up and if you would like to enter your username & password to auto-login. That Windows itself only took about half a dozen reboots until the several hundreds of updates were installed was a welcome surprise; Windows XP was even worse. Now, if anyone can tell me why audio drivers need a download of 380 MiB, that would be appreciated. Not that this was an uncommon size for every single driver, mind. Installed Size of linux-image-3.2.0-2-amd64 is 104,820.0 kB, btw. On the plus side, Windows 7 was able to automagically download a printer driver once I told it what the network printer's, which it refused to autodiscover, brand and model were.

Richard Hartmann: Open Source Days 2012, the aftermath

Open Source Days Open Source Days were nice, especially the meating(sic) of and talking to old friends and new people. I got to discuss Mercurial with two of its developers which was very interesting indeed. It's funny how similar git and Mercurial are in some regards and how different in others. Overall, it feels a bit as if Mercurial is not quite as distributed as git. Its local, sliding revision index feels like disaster waiting to happen, to me. On the other hand, Mercurials ability to not check out large files to your local repository sounds very git-annex-ish, which is nice. Choice is good :) My talk about how to gitify your life (slides, no video) went extremely well. Within my time-slot, there were a total of six talks, and lunch(!), in parallel. About 300 people attended OSD on Sunday and of those, many were at booths, stands, eating, etc. My personal turnout was 50-70 listeners so I was well above mathematical average and from what I heard, my talk was the most-visited one during that time. Add the fact that several people asked me to put up the resource links up again after the talk so they could take pictures and I would say this talk really was a success. Ego-stroking? Yes. Vanity? Most likely. Being able to see that people can really want to learn about what you have to say? Priceless. It's a real pity that there was no video recording, but it's obviously too late to do anything about that. Joey Hess asked me to at least make an audio recording on my laptop, but that turned out to be a lot of useless white noise with faint mumbling in between. Maybe I should at least get a small microphone or recorder so I will always have an audio copy. I already did order a presentation clickie with built-in laser pointer so I won't have to rely on possibly non-existent conference infra, any more. If there's any interest, I may re-record the talk as audio, else I will just try to get another speaker slot at another conference, with video recording, and link that. Denmark Barring the airport, central station, bus, hotel, venue, and metro, I did not see anything of Copenhagen. That's a pity, but I didn't want to leave the conference early. To continue my recent tradition of dumping random observations about places I visit onto my blog, here goes:

13 March 2012

Don Armstrong: Debbugs forcemerge support

One of the earliest features I wrote for the Debian bug tracking system (Debbugs) after joining the team was support for forcibly merging bugs. Originally, merging two bugs required that the bugs be in exactly the same state before merging them; forcemerge removed this requirement. Unfortunately, the way I originally implemented this was shortsighted, and merely forced the merge partners to have the same values as the merge master. This meant that owners, blocking bugs, and many other things were silently changed, which meant that people weren't notified of changes, and bugs could end up in an inconsistent state. A while ago, I decided to fix this by calculating the changes required to actually merge the bugs, making those changes, and then merging the bugs normally; thus, doing everything that a maintainer would normally have done for them. This necessitated abstracting out the entire control apparatus into the Debbugs::Control module. Now that it's complete, you can do the following:
> forcemerge 1 2
Bug #1 [foo] new title
Bug #2  Done: foo@bugs.something  [foo] foo
Unset bug forwarded-to-address
Severity set to 'wishlist' from 'grave'
3 was blocked by: 2
3 was not blocking any bugs.
Removed blocking bug(s) of 3: 2
2 was blocked by: 4
2 was not blocking any bugs.
Removed blocking bug(s) of 2: 4
Bug reopened
Removed annotation that bug was owned by bar@baz.com.
Removed indication that 2 affects bleargh
Removed tag(s) unreproducible and moreinfo.
Merged 1 2
> thanks
Stopping processing here.
and bug 2 now is merged with 1 and matches the state of 1. [The above is the control output from the appropriate bit of the 06_mail_handling.t test.] This change also means that I'll be able to finally write support for control@ operations at submit@ time. Also, all of the bug modifications that happen at submit@ or nnn@ time (setting title, found, etc.) will be implemented as calls to Debbugs::Control so we can eventually keep a postgresql database updated in addition to the flatfile database.

Michal Čihař: phpMyAdmin translations status

We're heading towards final phpMyAdmin 3.5, so let's again look at translations status. Here comes fifth round of translation status update. Since last update we one less translation (it was not really complete, the po file missed few strings) at 100%: There are few which are really close to 100% and I hope they will get there soon: As you can see, there is still lot of languages missing, this might be your opportunity to contribute to phpMyAdmin. Also you are welcome to translate phpMyAdmin 3.5 using new translation server.

Filed under: English Phpmyadmin 0 comments Flattr this!

Russell Coker: Airtasker Outsourcing Small Jobs

Airtasker.com is a new company that is dedicated to outsourcing small jobs [1]. It s still in a beta phase and has no way of making money the worker is paid directly by the employer and there s no facility for either of them to pay Airtasker, I would really like to know what the business model will be before considering whether to use it. It also doesn t seem to operate outside of Sydney which seems quite strange, how hard can it be to have separate Google maps instances for each city in Australia? The jobs that are advertised seem to be mostly household repair tasks and from the member list it appears that some people who already run small service businesses are trying to use Airtasker to get more work. There are also a lot of delivery tasks such as buying something from a shop and delivering it and driving a car from Melbourne to Sydney (good for someone who wants a subsidised interstate holiday). But there are some unusual tasks such as pretending to be someone s girlfriend to make his mother stop nagging him. It will be interesting to see whether this takes off. One thing that I might use it for is delivering computer equipment in the CBD. Some of my clients have no good parking near their office so I end up carrying computer gear from a tram stop or an inconvenient parking spot to their office. I d rather have someone paid to do at least half the work in carrying the gear and the rates that are being discussed on Airtasker are well within the range that my clients would be happy to pay. Related posts:
  1. outsourcing bad for corporations but good for the world There is ongoing discussion about whether outsourcing is good or...
  2. Fragmenting Information about Jobs A comment on my previous post about my Linux Jobs...
  3. IT Jobs and Working Conditions Mark Glossop has written about the best designs for offices...

Joachim Breitner: ghc-heap-view: Complete referential opacity

During the last week, I created ghc-heap-view, a library to investigate the actual memory representation of Haskell values. It is inspired by vacuum and the GHCi debugger, but goes beyond them by allowing the user to look inside thunks and functions and see what other values they refer to. Let me demonstrate it by running the included demo: ghc-heap-view-demo Here are a four different lists, where the first three are already evaluated.
The first one, l, was defined as a top level constant as
> l = [1,2,3]
and is now found at 0x00000000006d1750/2 (where the /2 is the pointer tag information) and fully evaluated:
  ConsClosure  info = StgInfoTable  ptrs = 2, nptrs = 0, tipe = CONSTR_STATIC, srtlen = 1 , 
                 ptrArgs = [0x00000000006d16e0/1,0x00000000006d1730/2],
                 dataArgs = [], descr = "ghc-prim:GHC.Types.:" 
The second one, l2, is locally defined
> let l2 = 4:l
and now found at 0x00007fdce19fe4b0/2. See how the cons-cell references l!
  ConsClosure  info = StgInfoTable  ptrs = 2, nptrs = 0, tipe = CONSTR_2_0, srtlen = 1 ,
                 ptrArgs = [0x00000000006dca50/1,0x00000000006d1750/2],
                 dataArgs = [],
                 descr = "ghc-prim:GHC.Types.:" 
And the binding
> args <- map length  fmap  getArgs
evaluates to the one , global empty list at 0x00000000006db640/1:
  ConsClosure  info = StgInfoTable  ptrs = 0, nptrs = 0, tipe = CONSTR_NOCAF_STATIC, srtlen = 0 ,
                 ptrArgs = [],
                 dataArgs = [],
                 descr = "ghc-prim:GHC.Types.[]" 
And now we have, at 0x00007fdce19fe4c8, the concatenation of them, but unevaluated:
> let x = l ++ l2 ++ args
The thunk keeps a reference to l2 and args, but not l, as that is at a static address, unless you are running this in GHCi:
  ThunkClosure  info = StgInfoTable  ptrs = 2, nptrs = 0, tipe = THUNK_2_0, srtlen = 1 ,
                  ptrArgs = [0x00007fdce19fe4b0/2,0x00000000006db640/1],
                  dataArgs = [] 
Now to some more closure types. m and m' locally bound of type the unboxed type Int#, with values 42 resp. 23.
> let f = \x n -> take (I# m + I# x) n ++ args
  t = f m' l2
So here is (0x00007fdce1937d50/2), referencing its free variables args and 42:
  FunClosure  info = StgInfoTable  ptrs = 1, nptrs = 1, tipe = FUN_1_1, srtlen = 65553 ,
                ptrArgs = [0x00000000006db640/1],
                dataArgs = [42] 
And t is a thunk that applies f (also referenced here) to an unboxed value (23) and l2:
  ThunkClosure  info = StgInfoTable  ptrs = 2, nptrs = 1, tipe = THUNK, srtlen = 0 ,
                  ptrArgs = [0x00007fdce19fe4b0/2,0x00007fdce1937d50/2],
                  dataArgs = [23] 
Lastly, here is the standard example for self reference:
> let x = id (:) () x
This is what x (0x00007fdce1947940) looks like, at least without -O:
  ThunkClosure  info = StgInfoTable  ptrs = 0, nptrs = 0, tipe = THUNK, srtlen = 1 ,
                  ptrArgs = [],
                  dataArgs = [] 
So it is unevaluated. Let us evaluate it using seq. Now we have, still at 0x00007fdce1947940:
  IndClosure  info = StgInfoTable  ptrs = 1, nptrs = 0, tipe = BLACKHOLE, srtlen = 0 ,
                indirectee = 0x00007fdce194cc98/2 
The thunk was replaced by an indirection. If we look at the target, 0x00007fdce194cc98/2, we see that it is a newly created cons-cell referencing the original location of x:
  ConsClosure  info = StgInfoTable  ptrs = 2, nptrs = 0, tipe = CONSTR_2_0, srtlen = 1 ,
                 ptrArgs = [0x00000000006db620/1,0x00007fdce1947940],
                 dataArgs = [],
                 descr = "ghc-prim:GHC.Types.:" 
After running the garbage collector (performGC), we find that the address of x is now 0x00007fdce19f30d0/2 and that the self-reference is without indirections:
  ConsClosure  info = StgInfoTable  ptrs = 2, nptrs = 0, tipe = CONSTR_2_0, srtlen = 1 ,
                 ptrArgs = [0x00000000006db620/1,0x00007fdce19f30d0/2],
                 dataArgs = [],
                 descr = "ghc-prim:GHC.Types.:" 
Future plans The output of ghc-heap-view is not really pretty yet; even the indentation in this blog post was added manually by me, so this really needs a pretty printer providing a nicer, possibly more compact representation, including something like what vacuum provides. Maybe vacuum can be ported to use this library, and also include the thunk s and function s references in the output. Maybe also the GHCi debugger can be extended to show more information about unevaluated expressions using this. Internally, the library is not very polished yet either. It only handles those closures types that I have seen so far, and is likely to break horribly if run in a threaded or debugging enabled runtime. How it works Obviously, this is not standard Haskell 98 code, but rather deep trickery involving the GHC API and some C code. Initially I tried to use the API that vacuum and the GHCi debugger rely on, which is an operation
unpackClosure# :: a -> (# Addr#, Array# b, ByteArray# #)
which takes any Haskell value and returns the address to its info table, the pointers and the non-pointer-data in the closure. Unfortunately, it was not complete in that it was meant only for data closures and will for other closure types, e.g. thunks, return no data and no pointers (as can be seen in the code). So I implemented my own version of this operation:
foreign import prim "slurpClosurezh" slurpClosure# :: Any -> (# Addr#, ByteArray#, Array# b #)
where the returned ByteArray# contains the complete closure, including extra information fields such as the arity of a function. The Array# is again the list of pointers in the closure. At first glance, this is a duplication, as the pointers are of course also contained in the ByteArray#. But as soon as the GHC runtime reigns again, a garbage collector run can happen, the referenced values will move somewhere else, and the words that once were pointers in the ByteArray# become useless. But the corresponding entries in the Array# are updated by the garbage collector, as it knows that these are pointers, and not just words. This way, we get both a faithful copy of the closure on the heap and useful references to the contained data. Here is a demonstration of this effect:
$ ghci -XMagicHash -package ghc-heap-view
GHCi, version 7.4.1: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
[..]
Loading package ghc-7.4.1 ... linking ... done.
Loading package ghc-heap-view-0.1 ... linking ... done.
Prelude> let  a = [1,2,3,4]; b = 5:a 
Prelude> :m + GHC.HeapView 
Prelude GHC.HeapView> rawHeapData <- getClosureRaw b
Prelude GHC.HeapView> rawHeapData 
(0x000000004080d658,[1082185320,140040739366568,140040739365928],[0x00007f5dc68626a8,0x00007f5dc6862428])
Prelude GHC.HeapView> System.Mem.performGC
Prelude GHC.HeapView> rawHeapData 
(0x000000004080d658,[1082185320,140040739366568,140040739365928],[0x00007f5dc41b3ad8,0x00007f5dc41b3b28]) 
The function rawHeapData is a thin wrapper around slurpClosure# which turns the primitive array in normal lists. Note that the second component of the triple is unchanged, but the third is updated by the garbage collector. Of course this means that the Show instance for the data type that ghc-heap-view uses to reference values is not referential transparent either. The foreign function import above is of type prim , i.e. does not call a C function but rather a Cmm function. Cmm is a reduced C that GHC uses internally to compile the Haskell code to, and most primitive operations are implemented in this language although I do quickly call regular C from my Cmm code to do the more complicated stuff, mainly figuring out what words of the closure are pointers. The knowledgeable reader might notice that I am passing a boxed value of type Any to the foreign function. This is currently not possible with foreign prim functions, and to actually use that code, you need the patch in GHC ticket #5931. But you can use ghc-heap-view without that as well (and the Cabal package will by default use that path), using the following hack to obtain the pointer to a Haskell value on the Heap as an unboxed type that can pass to the primitive operation:
foreign import prim "slurpClosurezh" slurpClosure'# :: Word#  -> (# Addr#, ByteArray#, Array# b #)
data Ptr' a = Ptr' a
aToWord# :: Any -> Word#
aToWord# a = case Ptr' a of mb@(Ptr' _) -> case unsafeCoerce# mb :: Word of W# addr -> addr
slurpClosure# :: Any -> (# Addr#, ByteArray#, Array# b #)
slurpClosure# a = slurpClosure'# (aToWord# a)
This works because a Word and a Ptr' have the same closure layout, only differing in the fact that one stores an a, and the other stores a Word#. Once we obtained the raw representation of the closure, we do the parsing in Haskell. Using the info table and the raw closure, we have enough information to tell which words have to be replaced by the appropriate pointer (which might already have been updated by the garbage collector) in the pointers list. This work was supported by a scholarship from the Deutsche Telekom Stiftung.

Flattr this post

12 March 2012

Francois Marier: Creating a FreeDOS bootable USB stick to upgrade BIOS

I have an old motherboard that requires creating a DOS boot floppy in order to upgrade its BIOS. Fortunately, it's not too hard to do this with FreeDOS and a USB stick.

The instructions below are based on an FDos wiki article.

Downloading the dependenciesThe first step is to download the required files from your motherboard manufacturer:
and then install the tools you'll need:
apt-get install makebootfat syslinux

Preparing the "floppy" imageStart by collecting all of the files you need to install FreeDOS on the USB stick:
cd /tmp

wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/commandx.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/kernels.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/substx.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/unstablx.zip

for ZIP in *.zip; do unzip $ZIP; done

cp ./source/ukernel/boot/fat16.bin .
cp ./source/ukernel/boot/fat12.bin .
cp ./source/ukernel/boot/fat32lba.bin .

cp /usr/lib/syslinux/mbr.bin .
and then create a directory for the files that will end up in the root directory of the "floppy":
mkdir /tmp/fs-root
cp ./bin/command.com /tmp/fs-root/
cp ./bin/kernel.sys /tmp/fs-root/
and copy the BIOS image and update program into that same directory (/tmp/fs-root/).

Creating a bootable USB stickPlug in a FAT-formatted USB stick and look for the device it uses (/dev/sdb in the example below).

Finally, run makebootfat:
/usr/bin/makebootfat -o /dev/sdb -E 255 -1 fat12.bin -2 fat16.bin -3 fat32lba.bin -m mbr.bin /tmp/fs-root

11 March 2012

Mohammed Sameer: Justin TV without flash!

Justin TV is a streaming service that uses the broken flash concept in order to allow you to view channels.

Unfortunately it's the only source of Arabic movies, series and shows.

I wrote a GStreamer source that retrieves the channel information and uses libRTMP to connect to the Justin streaming server. The rest is handled by GStreamer ;-)

If you want to watch http://www.justin.tv/ammartv for example, you simply do

gst-launch-0.10 playbin uri=jtv://ammartv

The 2nd plugin is a Totem plugin that indirectly utilizes the GStreamer source.
Given a language code (ar or en or ...), it will download the list of channels in that language and shows them in Totem's side panel. One can then easily double click on a channel and simply watch it. I originally wanted to do the Totem plugin in python but the inability to do asynchronous callback based HTTP requests prevented me :-(

Be warned that the code is of course a bit buggy and needs a lot of cleanups, it's been years since I coded anything in Gtk and that Justin might find out and either block the source somehow or even change the protocol but let's wait until that happens ;-) read more

Russell Coker: USB Flash Storage

For some years I have had my Internet gateway/firewall system in a cupboard in my bedroom. While I don t mind some computer noise (I ve slept near a server for most of the last 22 years) it s good to have it as quiet as possible so getting rid of the hard drive is desirable. I considered buying an IDE flash drive, but I d like to continue my trend of not paying for hardware so I chose to use some USB flash devices that HP was giving away at a seminar (thanks HP as an aside the P3 system is an old Compaq Desktop system). So I ve got one 4G USB device for root and one for Squid. For the past few months I ve had /var/spool/squid be a USB flash device. I considered using RAID-0 for that filesystem because the computer is a P3 and only has USB 1.2 and thus a maximum theoretical transfer rate of 1.5MB/s and a maximum real-world rate of about 1MB/s. But my ADSL connection doesn t seem able to sustain much more than 1MB/s and Squid doesn t write data synchronously so in all my tests the USB speed hasn t affected HTTP performance. One issue that has delayed my move to all USB is the difficulty of booting as the P3 system in question doesn t support booting from USB. I considered creating a boot CD that loads the kernel from the USB device, but that seemed a little painful and also relies on the CD-ROM drive working which isn t a great idea for a system that runs 24*7 in a dusty cupboard. I ended up using GRUB on the IDE hard drive to load the kernel and initrd and then mount a USB device as root, this seems to work and the command hdparm -S6 /dev/sda in /etc/rc.local makes the hard drive go to sleep once the system is booted. The only technical parts of the process were putting in the UUIDs of the filesystems in /etc/fstab (because I can t be sure which USB device will be found first) and creating a new initramfs with modules for USB storage listed in /etc/initramfs-tools/modules so that a USB device could be the root filesystem. The firewall system is now a bit quieter and based on my last tests of hard drive power use will probably dissipate about 5-7W less heat. The next thing to do is wait and see if it keeps running or falls over. ;) Related posts:
  1. flash for main storage I was in a discussion about flash on a closed...
  2. Flash Storage and Servers In the comments on my post about the Dell PowerEdge...
  3. IDE DMA and Flash I ve just been working with a Flash device used as...

10 March 2012

Andrew Cater: Raspberry Pi - three distributions and media centre

Fedora 14 Raspberry Pi Remix now available. Debian 6.0.4 available and Arch Linux port also available. OpenELEC media player distribution has also been ported to RaspberryPi - first port to ARM - but not yet tested on final production hardware, though working on the alpha boards.

Arch, Debian and Fedora torrenting - and proving very popular. It should be interesting to see how well the Raspberry Pi works in education - in the interim while the first batches go to developers, there will be a huge amount of use. What will be interesting is that this will mean LOTS of Linux hardware in people's hands at affordable prices.

Steve Kemp: Happy birthday to me

Recently I accidentally flooded Planet Debian with my blog feed. This was an accident caused by some of my older blog entries not having valid "Date:" headers. (I use chronicle which parses simple text files to build a blog, and if there is no Date: header present in entries it uses the CTIME of the file(s).) So why did my CTIMEs get lost? Short version I had a drive failure and a PSU failure which lead to me rebuilding a few things and cloning a fresh copy of my blog to ~/hg/blog/. My host is now once again OK, but during the pain the on-board sound started to die. Horribly crackly and sounding bad. I figure the PSU might have caused some collateral damage, but so far thats the only sign I see. I disabled the on-board sound and ordered a cheap USB sound device which now provides me with perfect sound under the Squeeze release of Debian GNU/Linux. In the past I've ranted about GNU/Linux sound. So I think it is only fair to say this time things worked perfectly - I plugged in the device, it was visible in the output of dmesg, and /proc/asound/cards and suddenly everything just worked. Playing music (mpd + sonata) worked immediately, and when I decided to try playing a movie with xine just for fun sound was mixed appropriately - such that I could hear both "song" + "movie" at the same time. Woo. (I'm not sure if I should try using pulse-audio, or similar black magic. Right now I've just got ALSA running.) Anyway as part of the re-deployment of my desktop I generated and pass-phrased a new SSH key, and then deployed that with my slaughter tool. My various websites all run under their own UID on my remote host, and a reverse-proxy redirects connections. So far example I have a Unix "s-stolen" user for the site stolen-souls.com, a s-tasteful user for the site tasteful.xxx, etc. (Right now I cannot remember why I gave each "webserver user" an "s-" prefix, but it made sense at the time!) Anyway once I'd fixed up SSH keys I went on a spree of tidying up and built a bunch of meta-packages to make it a little more straightforward to re-deploy hosts in the future. I'm quite pleased with the way those turned out to be useful. Finally I decided to do something radical. I installed the bluetile window manager, which allows you to toggle between "tiling" and "normal" modes. This is my first foray into tiling window managers, but it seems to be going well. I've got the hang of resizing via the keyboard and tweaked a couple of virtual desktops so I can work well both at home and on my work machine. (I suspect I will eventually migrate to awesome, or similar, this is very much a deliberate "ease myself into it" step.) ObQuote: "Being Swedish, the walk from the bathroom to her room didn't need to be a modest one. " - Cashback.

9 March 2012

Tanguy Ortolo: JSON License considered harmful

Summary The JSON License may seem interesting, but it a bad license, both non-free and ambiguous: do not use it. If you are the author of a piece of software that uses JSMin or its PHP port, consider dropping that non-free part or at least rendering it optional.The JSON license Do you know the JSON License? It is a software license inspired by the MIT/Expat license with one additional morality clause:
The Software shall be used for Good, not Evil.
Non-free For novices that clause can be perceived as a good idea serving a noble goal. But the problem here is that is would make your software non-free. Indeed, there are two common definitions of free software: the FSF definition, and the Debian Free Software Guidelines, also known as the Open Source Definition. Basically, both definitions indicate that to be free, a piece of software must be usable with no restrictions, and forbidding evil uses is such a restriction. Specifically, it violates: Ambiguous In addition to these incompatibilities to being free, this morality clause implies additional problems, the most important one being that it is ambiguous, which is very bad for a license clause: who exactly defines what is Good and what is Evil? If it is the author, then this clause could be used for arbitrarily forbidding any use; it it is the licensee, then this clause it as good as void. Do not use it For all these reasons, the JSON License with that infamous morality clause is not suitable for free software, so if you plan to use that license for your work, please consider these suggestions: 1. do not; 2. do not, really; 3. if still in doubt, refer to point 1. There are several standard free software licenses, the closest one being the MIT/Expat one, that you should consider instead. Now, if you really want to be considered as non-free, and rejected by Debian, Red Hat/Fedora, Google Code and more generally any free software distributor, this is the way to go. On thing worth noting is that this morality clause only annoys free software distributors, that are certainly not doing Evil, but will not prevent any evil person to do Evil things with your software. To encourage people to do Good rather than Evil, it would be better to formulate that clause as a suggestion rather than an order:
The Software should rather be used for Good, not Evil.
The JSMin case This infamous JSON License is known to be used by JSMin and its PHP port. If you are a Web application author, be careful when embedded external content, since it can render you whole piece of software non-free! If you are using JSMin, I would suggest that you drop it, or at least make it optional by using a conditional (if JSMin is here, use it, otherwise do nothing) so that distributors such as Debian can expurgate that non-free part.

Gunnar Wolf: I am going to DebConf12!

I have just bought our plane tickets to Managua, so I can finally say this: Going-to-banner-180x150-grey Yes, many of you will ask what happened, I was bragging everywhere I wanted to go by land, driving from Mexico City to Managua. I'd love to, and I'm sure it's completely doable... But we have family issues to attend on July 21, in Argentina. So we will have a beautiful flight schedule (and carbon footprint) for this July:
June 30
Mexico San Salvador Managua, 17:35-20:30. Yes, this means I will not be in Mexico to cast my vote on July 1st. Well, I had already accepted this would happen... And the price difference was quite sensible.
July 15
Managua San Jos Mexico, 16:25-22:20
July 16
Mexico Santiago Buenos Aires (AEP), 20:30-09:55
July 23
Buenos Aires (EZE) Lima Mexico, 08:35-19:00
Several people have asked me on the best airline options for this trip. In our case, to Managua, it was with TACA, US$518 total. You can get tickets for ~US$30 less, but the flight goes through Panama instead of San Salvador, for an extra 1000Km And instead of ~3hr it makes slightly over 6. Yes, on our way back we will be routed a bit South to San Jos , but it's not as bad, and it's for a very short layover. For Argentina? Well, we have always found LAN to be the cheapest and most convenient. This time, TACA/Avianca was a very close second, which lost due to almost doubling the flight+layover time Why aren't we taking a Mexico Managua Buenos Aires flight instead? Because it's ~US$150 more expensive per person. Not *that* much, but still some money. And by returning to Mexico and having a night at home, we will save us the hassle of carrying Winter clothes to Nicaragua and Summer clothes to Argentina. Oh, and if you are planning on dropping by home while we are away and robbing all of our stuff: There's not that much to take from there, and we have already arranged for somebody to be there while we are away. But thanks for thinking about us, anyway! [update] And what about DebConf12 registration? When is the system opening for us all to register? Soon, dear friends, we are talking about some related issues, and you will have your registrationi open soon.

Petter Reinholdtsen: Debian Edu interview: Nigel Barker

Inspired by the interview series conducted by Raphael, I started a Norwegian interview series with people involved in the Debian Edu / Skolelinux community. This was so popular that I believe it is time to move to a more international audience. While Debian Edu and Skolelinux originated in France and Norway, and have most users in Europe, there are users all around the globe. One of those far away from me is Nigel Barker, a long time Debian Edu system administrator and contributor. It is thanks to him that Debian Edu is adjusted to work out of the box in Japan. I got him to answer a few questions, and am happy to share the response with you. :) Who are you, and how do you spend your days? My name is Nigel Barker, and I am British. I am married to Yumiko, and we have three lovely children, aged 15, 14 and 4(!) I am the IT Coordinator at Hiroshima International School, Japan. I am also a teacher, and in fact I spend most of my day teaching Mathematics, Science, IT, and Chemistry. I was originally a Chemistry teacher, but I have always had an interest in computers. Another teacher teaches primary school IT, but apart from that I am the only computer person, so that means I am the network manager, technician and webmaster, also, and I help people with their computer problems. I teach python to beginners in an after-school club. I am way too busy, so I really appreciate the simplicity of Skolelinux. How did you get in contact with the Skolelinux/Debian Edu project? In around 2004 or 5 I discovered the ltsp project, and set up a server in the IT lab. I wanted some way to connect it to our central samba server, which I was also quite poor at configuring. I discovered Edubuntu when it came out, but it didn't really improve my setup. I did various desperate searches for things like "school Linux server" and ended up in a document called "Drift" something or other. Reading there it became clear that Skolelinux was going to solve all my problems in one go. I was very excited, but apprehensive, because my previous attempts to install Debian had ended in failure (I used Mandrake for everything - ltsp, samba, apache, mail, ns...). I downloaded a beta version, had some problems, so subscribed to the Debian Edu list for help. I have remained subscribed ever since, and my school has run a Skolelinux network since Sarge. What do you see as the advantages of Skolelinux/Debian Edu? For me the integrated setup. This is not just the server, or the workstation, or the ltsp. Its all of them, and its all configured ready to go. I read somewhere in the early documentation that it is designed to be setup and managed by the Maths or Science teacher, who doesn't necessarily know much about computers, in a small Norwegian school. That describes me perfectly if you replace Norway with Japan. What do you see as the disadvantages of Skolelinux/Debian Edu? The desktop is fairly plain. If you compare it with Edubuntu, who have fun themes for children, or with distributions such as Mint, who make the desktop beautiful. They create a good impression on people who don't need to understand how to use any of it, but who might be important to the school. School administrators or directors, for instance, or parents. Even kids. Debian itself usually has ugly default theme settings. It was my dream a few years back that some kind of integration would allow Edubuntu to do the desktop stuff and Debian Edu the servers, but now I realise how impossible that is. A second disadvantage is that if something goes wrong, or you need to customise something, then suddenly the level of expertise required multiplies. For example, backup wasn't working properly in Lenny. It took me ages to learn how to set up my own server to do rsync backups. I am afraid of anything to do with ldap, but perhaps Gosa will help. Which free software do you use daily? Nowadays I only use Debian on my personal computers. I have one for studio work (I play guitar and write songs), running AV Linux (customised Debian) a netbook running Squeeze, and a bigger laptop still running Skolelinux Lenny workstation. I have a Tjener in my house, that's very useful for the family photos and music. At school the students only use Skolelinux. (Some teachers and the office still have windows). So that means we only use free software all day every day. Open office, The GIMP, Firefox/Iceweasel, VLC and Audacity are installed on every computer in school, irrespective of OS. We also have Koha on Debian for the library, and Apache, Moodle, b2evolution and Etomite on Debian for the www. The firewall is Untangle. Which strategy do you believe is the right one to use to get schools to use free software? Current trends are in our favour. Open source is big in industry, and ordinary people have heard of it. The spread of Android and the popularity of Apple have helped to weaken the impression that you have to have Microsoft on everything. People complain to me much less about file formats and Word than they did 5 years ago. The Edu aspect is also a selling point. This is all customised for schools. Where is the Windows-edu, or the Mac-edu? But of course the main attraction is budget.The trick is to convince people that the quality is not compromised when you stop paying and use free software instead. That is one reason why I say the desktop experience is a weakness. People are not impressed when their USB drive doesn't work, or their browser doesn't play flash, for example.

John Goerzen: Dear FedEx: I Live Here

FedEx has had some issues in my area. There for awhile, they kept returning packages as undeliverable, saying the address was bad. It wasn t. Just because their GPS can t find it or their driver doesn t know it doesn t mean it doesn t exist. A map, a real map, would have done it. I had this kind of conversation with them once. Me: So you re returning my package to the sender. FedEx: Yes sir, the address was wrong. Me: No, it wasn t. [ we confirm we're talking about the same address. ] FedEx: Our driver says it doesn t exist. Me: I ve lived here for [x] years. I do know my address. FedEx: Maybe it s just new. Me: Someone has lived at this spot since the 1890s. FedEx: Hmm. I don t know then. Me: How about I give you directions? FedEx: Well, OK, I guess I could send directions. But they probably won t see them for three days. Me: Where will my package be by then? FedEx: Hard to say. Me: Do you people have cell phones maybe? FedEx: No. Me: Well, OK then Lately we have been noticing that the driver throws the package from the truck without stopping. We find them all over the yard. Unless it s big, in which case he puts it on the porch, puts a completely useless sticker on the door saying We delivered your package to right here! , and doesn t ring the doorbell. And panics due to the muddy road. The FedEx driver once told Terah, I almost got stuck on that muddy road! What s the good one? Terah: Well, which one did you take? [ he describes ] Terah: [ serious ] That IS the good road. FedEx: [ turning pale ] Oh my. I don t know if I can make it back to the highway. Terah: Well, I suppose I could get the boys in the car and drive you into town FedEx: [ embarrassed and more pale ] Uhm that s OK uhm I think A *month* later, we got another FedEx package. The driver called me at work, saying, I just don t know how I m going to get down those roads. Can I deliver the package to you here? (10 miles away). I said yes, and managed to suppress a significant snicker at his obvious relief. FedEx drivers seem to come and go. The UPS driver, on the other hand, knows the names of our boys, our cat, and puts anything small behind the storm door so it won t blow away if we re not home. And he gets a great big laugh when I tell him that the FedEx drivers are afraid of our roads. Haha, he thinks THAT S mud? Obviously he never drives in Butler County.

Next.

Previous.