Search Results: "reg"

1 July 2012

Wouter Verhelst: Heathrow tourist trap buses

So I slept in the Sheraton Heathrow hotel last night. It's a half-an-hour walk away from the airport, or a five minute bus drive. If you go to any service desk and ask for the best way to get to the hotel, they'll offer you a ticket for the 'HotelHoppa' service, which will stop right in front of your hotel entrance, and costs 4 for a one-way trip (even though the bus sides declare a 4.5 charge) However, if you take the regular "London United" bus service (which drives all over London, not just near the airport like the HotelHoppa service) then the same trip is completely free of charge. You may have to walk about a hundred yards more to reach your hotel, since the bus will stop at a regular bus stop rather than at the hotel entrance, but that's hardly a problem. What a ripoff. So, to clarify, don't take this bus: Rip-off bus Instead, take this bus: Free bus This message brought by your friendly neighbourhood anti-ripoff service.

Niels Thykier: Performance bottlenecks in Lintian

Thanks to a heads up from Bastian Blank, I learned that Lintian 2.5.7 and 2.5.8 were horribly slow on the Linux binaries. Bastian had already identified the issue and 2.5.9 fixed the performance regression. But in light of that, I decided to have a look at a couple of other bottlenecks. First, I added a simple benchmark support to Lintian 2.5.10 (enabled with -dd) that prints the approximate run time of a given collection. As an example, when running lintian -dd on lintian 2.5.10, you can see something like: 
N: Collecting info: unpacked for source:lintian/2.5.10 ...
[...]
N: Collection script unpacked for source:lintian/2.5.10 done (0.699s)
When done on linux-image, the slowest 3 things with 2.5.10 are (in order of appearance): 
[...]
N: Collection script strings for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (12.333s)
N: Collection script objdump-info for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (15.915s)
[...]
N: Finished check: binaries (5.911s)
[...]
(The mileage (and order) probably will vary a bit.) These 3 things makes up about 22 seconds of a total running time on approximately 28-30s on my machine. Now if you wondering how 12, 16 and 6 becomes 22 the answer is parallelization . strings and objdump-info are run in parallel so only the most
expensive of the two counts in practise (with multiple processing units). The version of linux-image I have been testing (3.2.20-1, amd64) has over 2800 ELF binaries (kernel modules). That makes the runtime of strings and objdump-info much more dominating than in your average package . For the fun of it I have done a small informal benchmark of various Lintian versions on the binary. I have used the command line: 
# time is the bash shell built-in and not /usr/bin/time
$ time lintian -EvIL +pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null
# This was used with only versions that did not accept -L +pedantic
$ time lintian -EvI --pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null
With older versions of Lintian (<= 2.5.3) Perl starts to emit warnings; these have been manually filtered out. I used lintian from the git repository (i.e. I didn t install the packages, but checked out the relevant git tags). I had libperlio-gzip-perl installed (affects the 2.5.10 run). Most results are only from a single run, though I ran it twice on the first version (hoping my kernel would cache the deb for the next run). The results are: 
2.5.10
real  0m28.836s
user  0m36.982s
sys  0m3.280s
2.5.9
real  1m9.378s
user  0m33.702s
sys  0m11.177s
2.5.8
real  4m54.492s
user  4m0.631s
sys  0m30.466s
2.5.7 (not tested, but probably about same as 2.5.8)
2.5. 0..6 
real  1m20s  - 1m22s
user  0m19.0s - 0m20.7s
sys  0m5.1s  - 0m5.6s
I think Bastian s complaint was warranted for 2.5. 7,8 . :) While it would have been easy to attribute the performance gain in 2.5.10 on the new parallelization improvements, it is simply not the case. These improvements only apply to running collections when checking multiple packages. On my machine, the parallelization limit for a package is effectively determined by the dependencies between the collections on my machine. Instead the improvements comes from reducing the number of system(3) (or fork+exec) calls Lintian does. Mostly through using xargs more, even if it meant slightly more complex code. But also, libperlio-gzip-perl shaved off a couple of seconds on binaries check. But as I said, linux-image is not your average package . Most of the improvements mentioned here are hardly visible on other packages. So let s have a look at some more other bottlenecks. In my experience the following are the worst offenders : But enough Lintian for now time to fix some RC bugs!

Ingo Juergensmann: 100% CPU load due to Leap Second

This morning Gregor Samsa woke up... oh, pardon! This morning I woke up and found myself puzzled, because my home server was eating up all of my 4 cores CPU cycles. Especially mysqld was high on CPU load. 100% CPU load for the mysql-server instance and 100% CPU load for akonadiservers own mysqld instance. Restarting KDE and mysql-server didn't help on my Debian unstable machine. Next step was upgrading the system. Sometimes this helps indeed, but not today. Looking at bugs.debian.org for mysql-server didn't reveal any help as well. So my next logical step was to ask on #debian-devel in IRC. And my question was very quick answered:
11:28 < ij> since tonight I've got two mysqld processes running at 100% CPU, one spawned by akonadi and
the other is the mysqld from mysql-server (unstable that is). is this an already known issue?
haven't found anything on b.d.o for mysql-server, though
11:29 < mrvn> ij: topic
11:29 < mrvn> you need to set the time
11:30 < ij> waaaaah!
11:30 < mrvn> ij: indeed.
The topic was at that time:
100% CPU? Reset leap second http://xrl.us/bnde4w
So, it was caused by the leap second. Although you might suspect mysql doing some nasty things (which, IMHO, is always a good guess ;)), the issue is this time within the Linux kernel itself, as a commit on git.kernel.org clearifies. To fix this issue you need to set the time manually using the following command or just reboot:
date -s " date "
So far I found these applications being hit by this kernel bug:
  • mysql-server
  • akonadi (as it uses own mysql instances)
  • Firefox
  • Openfire Jabber server (because it's using Java, which seems to trigger the problem as mysql does)
  • Virtualbox' VBoxSVC process
  • puppetmaster from package puppet, reported by Michael
  • mythfrontend, reported by pos on #debian-devel
  • Jetty, Hudson, Puppet agent and master, reported by Christian
  • milter-greylist, reported by E. Recio
  • dovecot, reported by Diogo Resende
  • Google Chrome, reported by Erik B. Andersen
  • if you find more apps, please comment and I'll include them here...
So, hope this helps and many thanks to mrvn and infinity on #debian-devel for the help!
Kategorie:
Debian
Tags:
Debian
Linux
Bug
Software

Dirk Eddelbuettel: Getting numpy data into R

The other day, I found myself confronted with a large number of large files. Which were presented in (gzip-)compressed ascii format---which R reads directly via gzfile() connections---as well as (compressed) numpy files. The numpy can be read very efficiently into Python. We can do the same in R via save() and load(), of course. But the trouble is that you need to read them first. And reading hundreds of megabytes from ascii is slow, no matter which language you use. Concerning R, I poked aound scan(), played with the colClasses argument and looked at the recent LaF package written just for this purpose. And all these solutions were still orders of magnitude slower than reading numpy. Which is no surprise as it is really hard to beat binary formats when you have to parse countless ascii tokens. So the obvious next idea was to read the numpy file in Python, and to write a simple binary format. One helpful feature with this data set was that it contained only regular (rectangular) matrices of floats. So we could just store two integers for the dimensions, followed by the total data in either one large binary blob, or a sequence of column vectors. But one minor trouble was that the Intertubes lead to no easy solution to unpack the numpy format. StackOverflow had plenty of question around this topic converned with, say, how to serialize in language-independent way. But no converters. And nobody local knew how to undo the "pickle" format underlying numpy. But a remote friend did: Laurent, well-known for his Rpy2 package, pointed me towards using the struct module and steered me towards the solution shown below. So a shameless plug: if you need a very experienced Python or R consultant for sciece work, consider his consulting firm. Finally, to round out this post, let's show the simple solution we crafted so that the next guy searching the Intertubes will have an easier. Let us start with a minimal Python program writing numpy data to disk: 
#!/usr/bin/env python
#
# simple example for creating numpy data to demonstrate converter
import numpy as np
# simple float array
a = np.arange(15).reshape(3,5) * 1.1
outfile = "/tmp/data.npy"
np.save(outfile, a)
Next, the simple Python converter to create a binary file containing two integers for row and column dimension, followed by row times columns of floats: 
#!/usr/bin/python
#
# read a numpy file, and write a simple binary file containing
#   two integers 'n' and 'k' for rows and columns
#   n times k floats with the actual matrix
# which can be read by any application or language that can read binary
 
import struct
import numpy as np
inputfile = "/tmp/data.npy"
outputfile = "/tmp/data.bin"
# load from the file
mat = np.load(inputfile)
# create a binary file
binfile = file(outputfile, 'wb')
# and write out two integers with the row and column dimension
header = struct.pack('2I', mat.shape[0], mat.shape[1])
binfile.write(header)
# then loop over columns and write each
for i in range(mat.shape[1]):
    data = struct.pack('%id' % mat.shape[0], *mat[:,i])
    binfile.write(data)
binfile.close()
Lastly, a quick littler script showing how R can read the data in a handful of lines: 
#!/usr/bin/r
infile <- "/tmp/data.bin"
con <- file(infile, "rb")
dim <- readBin(con, "integer", 2)
Mat <- matrix( readBin(con, "numeric", prod(dim)), dim[1], dim[2])
close(con)
print(Mat)
That did the job---and I already used to converter to read a few weeks worth of data for further analysis in R. This obviously isn't the last word on possible solutions as the additional temporary file can be wasteful (unless it forms a cache for data read multiple times). If someone has nice solutions, please don't hold back and contact me. Thanks again to Laurent for the winning suggestion concerning struct, and help in getting the examples shown here to work.

30 June 2012

Luca Falavigna: FTP Team stats during Wheezy development

Already chilled by Wheezy freeze? It s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes. Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:
Login Accepts Rejects Comments
ansgar 407 accepts (0.800 per day) 71 rejects (0.139 per day) 53 comments (0.104 per day)
dak 12 accepts (0.024 per day) 1 rejects (0.002 per day) 0 comments (0.000 per day)
dktrkranz 4319 accepts (8.485 per day) 381 rejects (0.749 per day) 104 comments (0.204 per day)
joerg 100 accepts (0.196 per day) 12 rejects (0.024 per day) 1 comments (0.002 per day)
mhy 214 accepts (0.420 per day) 14 rejects (0.028 per day) 5 comments (0.010 per day)
stew 67 accepts (0.132 per day) 16 rejects (0.031 per day) 7 comments (0.014 per day)
tolimar 1480 accepts (2.908 per day) 93 rejects (0.183 per day) 84 comments (0.165 per day)
twerner 278 accepts (0.546 per day) 53 rejects (0.104 per day) 26 comments (0.051 per day)

Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:
  1. Debian Perl Group (559 accepts)
  2. Debian Haskell Group (491 accepts)
  3. Debian Ruby Extras Maintainers (285 accepts)
  4. Debian Java Maintainers (257 accepts)
  5. Debian Med Packaging Team (164 accepts)
  6. Debian Multimedia Maintainers (160 accepts)
  7. Debian Fonts Task Force (156 accepts)
  8. Debian Javascript Maintainers (137 accepts)
  9. Debian Python Modules Team (129 accepts)
  10. Debian Qt/KDE Maintainers (98 accepts)
That doesn t reflect the real developers, though. Here s our Changed-By top ten:
  1. Clint Adams (216 accepts)
  2. Jonas Smedegaard (208 accepts)
  3. Ben Hutchings (203 accepts)
  4. Joachim Breitner (153 accepts)
  5. TANIGUCHI Takaki (112 accepts)
  6. Alessio Treglia (101 accepts)
  7. David Paleino (95 accepts)
  8. Nicholas Bamber (76 accepts)
  9. Mathieu Parent (68 accepts)
  10. Jeff Breidenbach (63 accepts)
Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place ;)

How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not :) liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never ;)

FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:

FTP Team also took care of override changes:

Timo Jyrinki: Where computing takes you...

Do you ever have those afternoons you get a great idea and you've all the evening time for that task. The task is a relaxing one and won't need much attention, and you can watch a movie or something. But then, it happens that the evening turns into night as you realize a couple of little details adding complexity to the idea, and the task turns out to be much more invasive to your evening than you thought?

In this example, I got the great idea to upgrade my Debian running NAS device (thanks Martin for everything!) to use ext4 instead of ext3. The kind of idea that takes a long time for relatively little practical benefit, but it just feels like a nice thing do when you've the extra amounts of nerd time available. It's basically just opening the NAS device up, mounting its hard disk to a laptop via external case, running the tune2fs and fsck then putting the disk back. It just takes a long time for the initial fsck (to make sure everything's intact) and then the required fsck run to get ext4 mountable.

Only in this situation, it would have been beneficial to have the ext4 support in the flashed initramfs before the migration. So... before the photo below, I've already:
  1. done the ext4 migration and fsck:s
  2. screwed the disk back to the NAS case, attached cables and found that it doesn't boot
  3. (on the laptop with the hard disk attached again tried manually unpacking initramfs and adding ext4 module... also had time to bind mount everything and chroot into the ARM system to run update-initramfs manually... also tried booting with those... until remembered the simple fact that the /boot partition is only for show and also the initramfs is loaded directly from flash)
  4. copied the main root filesystem content from the original disk to another external disk with ext3 partition
  5. attached the another disk (with same UUID:s) to the QNAP NAS device, booted, double-checked that I have now ext4 specified under /etc/initramfs-tools/modules, reconfigured the linux image that also regenerates initramfs and flashes it
And in the photo, what's happening is that:
  1. I've again the original disk reattached and system booted with the initramfs generated and flashed from the ext3 disk
  2. the NAS device is hanging in the air, cover open, from the closet where I've things stuffed in (normally secured with cable ties), and I need to support it with a knee or one hand since the 2TB disk is much heavier than the small SSD I used as the ext3 disk so the power cable and RJ-45 cable would have pretty heavy load
  3. Since I've only one hand in use and can't use a laptop, I'm logging in via my Nokia N9 and then reflashing the kernel + initramfs from this original disk, just to make sure everything is now alright and also after that flashing it still boots (it does!). Note that I feel like the setup is secure enough for non-interrupted flashing so that I can indeed support the NAS with a knee, use one hand to keep N9 and another hand to take a photo with a camera.

And so we have had a productive and educating afternoon/evening/night once again. Does this ever happen to you?

Gregor Herrmann: RC bugs 2012/26

wheezy is frozen, the outside temperature in INN at midnight is 30 C, & in 3 hours my trip to MGA starts. & here's the list of RC bugs; this week all but one were sponsored uploads where the BTS already contained a complete debdiff; thanks guys!

Russell Coker: Links June 2012

This Youtube video is an amusing satire of EULAs and copyright law as applied to uploading consciousness [1]. Washington s Blog has an insightful article about the way that the lack of trust in the US is killing their economy [2]. It seems that as long as the 1% are allowed to get away with breaking the law the US economy won t recover. It s something we should all keep in mind at election time, let s not be like the US. AirBnB is an interesting service to allow people to rent a room or an apartment, a quick scan indicates that it s a lot cheaper than hotels and offers many good locations [3]. It s probably of most interest to the more social people though which is a down side for me. Queensland s highest court has ruled that vilification of homosexuals is also vilification of bisexuals because an essential aspect of bisexuality is a sexual feeling of a person of the same sex, that is, homosexuality [4]. Anyone who didn t find that totally obvious could simply consult any dictionary or encyclopedia to find out. But the Australian legal system needed a 46 page ruling. We really need some sanity in the courts. Father Gregory Boyle founded an organisation named Homeboy Industries with the purpose of providing jobs for people with criminal records [5]. It s amazing the way he is helping people turn their lives around and it s apparently a lot cheaper than sending them to jail. Related posts:
  1. Links February 2012 Sociological Images has an interesting article about the attempts to...
  2. Links March 2012 Washington s Blog has an informative summary of recent articles about...
  3. Links April 2012 Karen Tse gave an interesting TED talk about how to...

29 June 2012

Dirk Eddelbuettel: Rcpp 0.9.13

The bug-fix in version 0.9.12 of Rcpp turned out to be incomplete, so a new version 0.9.13 is now on CRAN and will get to Debian shortly. The Rcpp::Enviroment constructor is now properly fixed (using the global environment as a default value). As well, a new #define was created to detect the clang++ compiler before/after version 3.0.0 as one exceptions header changed location. Unit tests files were also once more updated. The complete NEWS entry for 0.9.13 is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page. 
0.9.13  2012-06-28
    o   Truly corrected Rcpp::Environment class by having default constructor
        use the global environment, and removing the default argument of
        global environment from the SEXP constructor
    o   Added tests for clang++ version to include bits/exception_defines.h
        for versions 3.0 or higher (similar to g++ 4.6.0 or later), needed to
        include one particular exceptions header
    o   Made more regression tests conditional on the RunAllRcppTests to come
        closer to the CRAN mandate of running tests in sixty seconds
    o   Updated unit test wrapper tests/doRUnit.R as well as unitTests/runTests.R
Thanks to CRANberries, you can also look at a diff to the previous release 0.9.12. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

28 June 2012

Sylvain Le Gall: OASIS 0.3.0 release

Logo OASIS small OASIS is a tool to integrate a configure, build and install system in your OCaml project. It helps to create standard entry points in your build system and allows external tools to analyse your project easily. It is the building brick of OASIS-DB, a CPAN for OCaml. This release takes almost 18 months to complete. This is too long and I will talk in another blog post on the way I am trying to improve this right now (esp. using continuous integration). This new release fixes a small bug (1 line) that prevents setup.ml to run with OCaml 4.00. If you have projects that was generated with a former release consider upgrading to OASIS 0.3.0. There are several big new stuff that comes with this release. It now supports Pack: field for libraries which allows to pack your library using -for-pack and so on. We are also compiling .cmxs (dynlink object for native) by default and we publish them in the META file. This feature was implemented in order to get more libraries to provide .cmxs and to help project like Ocsigen to take advantage of this. If you want to get rid of this at configure time, you can use ./configure --override native_dynlink false. We introduce two new default flags: --enable-tests and --disable-docs for configure. These are implicit flags that define if we will run Test sections or compile Document sections. They are especially useful to reduce the number of dependencies, because dependencies of Test will be excluded by default. We recommend to set Build$: flag(tests) to any Library or Executable sections that are only useful for tests. This allows you to really cut down the number of dependencies. The last change I want to introduce is about the old setup-dev subcommand which is now deprecated. It has been replaced by 2 different update schemes. I am pretty excited by this feature which in fact comes from OASIS user (esp. Lwt project). The former scheme was to have a big 'setup.ml' that always call the command oasis to update. This was complex and not very useful. We now have 2 mode: dynamic and weak. dynamic allow you to have a small setup.ml and to keep your VCS history clean but you need to install OASIS. weak need a big setup.ml but only need to call the command oasis if someone change something in _oasis. This mode is targeted to project that wants to be able to checkout from VCS an OCaml project without installing OASIS. The difference generated by weak mode doesn't pollute too much the VCS history because most the time they make sense. For example, you upgrade your package version number in _oasis and it produces a change with 6 lines where the version number changes in every META, setup.ml files and so on. This version has been tested on: You can download it here or use your favorite package manager: Debian (UPDATE: pending upload) 
$ apt-get install -t experimental oasis
GODI 
$ godi_console perform -build apps-oasis
odb.ml 
$ ocaml odb.ml oasis
Here is the complete changelog: EXTREMLY IMPORTANT changes (read this) PACKAGES uploaded to oasis-db will be automatically "derived" before OCaml 4.00 release (i.e. oUnit v1.1.1 will be regenerated with this new version as oUnit v1.1.1~oasis1). PACKAGES not uploaded to oasis-db need to be regenerated. In order not to break 3rd party tools that consider a tarball constant, I recommend to create a new version. Thanks to INRIA OCaml team for synchronizing with us on this point. Major changes: You can now have the following example: 
     ...
     Executable test_exec
       Install: false
       Build$: flag(tests)
       MainIs: testExec.ml
       BuildDepends: oUnit
     
     Test main
       Command: $test_exec
       TestTools: test_exec
     ...
The Run$: flag(tests) is implicit for the section Test main. The default value is false for tests. If all the executable for test are flagged correctly (Build$: flag(tests)), you'll get rid of the dependency on oUnit. It works the same for documentation, however the default is true. (Closes: #866) In order to allow interdependent flags, we transform back lazy values into unit -> string functions. This allows to change a flag value on the command line and to update all the dependent values. (Closes: #827, #938) It defines different ways to manage the auto-update of setup.ml: The choice between weak and dynamic depends on your need with regard to VCS and to the presence of oasis. The weak allow to checkout the project from VCS and be able to work on it, without the need of installing 'oasis' as long as you don't change the file _oasis. But it clutter your VCS history with changes to the build system each time you change something in _oasis. The 'dynamic' mode gives you no VCS history pollution but makes mandatory to have installed oasis libraries. Avoid copying executable to their real name. It helps to call ocamlbuild a single time for the whole build rather than calling it n time (n = number of executable sections) and copying resulting exec. This speeds up the build process because ocamlbuild doesn't have to compute/scan dependencies each time. The drawback is that you have to use $foo when you want to call Executable foo, because $foo will be '_build/.../main.byte'. For example: 
CCOpt: -DEXTERNAL_EXP10 -L/sw/lib "-framework vecLib"
Will be parsed correctly and outputed according to target OS. In order to ease building oasis, we have minimize the number of dependencies. You only need to install ocamlmod, ocamlify and ocaml-data-notation for a standard build without tests. Dependencies on pcre, extlib and ocamlgraph has been dropped. The remaining dependencies are hidden behind a flag tests. OASIS now produces .cmxs file by default and add them to META. Now a META looks like: 
     ...
     archive(byte) = "oasis.cma"
     archive(byte, plugin) = "oasis.cma"
     archive(native) = "oasis.cmxa"
     archive(native, plugin) = "oasis.cmxs"
     ...
This will ultimately help to generate automatically .cmxs for all oasis enabled projects. We hope that this new feature will improve dynamic linking use in OCaml (esp. for project like Ocsigen). Other changes Thanks to Anil Madhavapeddy, Pierre Chambart, Christophe Troestler, Jeremie Dimino, Ronan Le Hy, Yaron Minsky and Till Varoquaux for their help with this release. Also thanks to all the testers of the numerous release candidates. This was a long work and each time a tester has downloaded oasis has helped me to know that I was working for someone.

DebConf team: DebConf12 schedule ready! (Posted by Gunnar Wolf)

The DebConf12 schedule is finally available! As usual, the DebConf conference can only be finalized a couple of days before DebConf itself starts. But hey Take a look at it, and start planning your activities! Not final, never final Remember that, even now that the conference schedule is announced, you can still submit talks to DebConf, following what we call ad-hoc talks. What are the steps to do it?
  1. Check the schedule and find the timeslot (day, hour and room) you want to talk in
  2. Register your event in Pentabarf, and note the generated event ID (the last number in the URL after the submission is accepted)
  3. Send a mail to schedule@debconf.org detailing the event ID, title (just to confirm it), and the requested timeslot
  4. All requests _must_ be received at least 24hr before the requested event for the scheduling team to act accordingly and for the video team to be able to plan on resources to cover the requested session
  5. Video team coverage (event recording plus streaming on a best-effort basis) will be given to activities scheduled on the Aula Magna and Roberto Ter n auditoriums.
  6. We will try to grant requests on a first-come-first-serve basis. If two events are requested for the same timeslot, the first will have precedence, and we will schedule the second as close as possible to it. We will notify the submitters if this happens, of course.
Conference tracks In DebConf12, many talks are organized around the following tracks:
DebianDay
This track is meant for the local (Nicaraguan/Central American) community to be attracted to Debian. It does not usually have technically minded talks; the DebConf general audience is, of course, invited to take part in it, as there will surely be interesting angles on how Debian is used in different use-cases, and there will be interesting Q&A session with the audience.
Getting involved in Debian
Central America is a very special region. It has vibrant Free Software user groups and communities, among which there is an important interest to get involved in Debian.
This track will have some talks in Spanish, some talks in English, explaining different Debian processes and tools, aiming at attracting locals to contribute to Debian.
Debian for the cloud
All about virtualization. Management infrastructure, details on virtual machine engines, insight on how to tweak your kernel for optimum virtualized performance, and of course how to get it all humming perfectly with your Debian packages.
Building and porting
With an every day stronger transit to mobile computing, we are seeing a renaissance of porting. At the same time, we finally have multiarch as part of Debian. And were that not to be enough, we finally have more than one compiler that can be used for most of the software in the archive. Put all this together, and you will have a very interesting track.
Social activities
DebConf is as much about technical stuff as it is about building closer ties within the members of its community. Eating cheese+wine, going on a trip to the guts of a volcano, having a nice banquet Do you want to add an activity to this pseudo-track?
Skills exchange
Knowledge can also be built by all of us putting our grain of sand. We first had explicit skills exchange sessions in Bosnia last year, and they were very well received. Feel free to propose similar sessions!
And Well, three days to DebCamp and counting. See you Real Soon Now in Managua!

Sorina Sandu: Status update

Hello!

First of all: I'm going to DebConf :D and I really want to say a big "Thank you!" to every one who made this possible.


So, here's what I've done recently:

I spent some time putting everything in order for the departure to Nicaragua, things like finding a plane ticket through a route that does not require a Visa, making a new Passport, now all is in order.

I've integrated the changes for listing available ESSIDs, in order to close #637128 [0] into a separate branch , which can be found here [1].

The plan is to have only one new template which lists the available networks and which has a option for entering the ESSID manualy. In order to do so, a simple scan is performed on the wireless interface, and the configuration for a particular network resulted after the scan is kept if that one is chosen. The enter manually situation is treated exactly as it was before.

Furthermore, last week (or the week before) I've done some coding style fixes again in a separate branch [2], nothing very fancy, just trimming trailing white spaces and changing some tabs to spaces, where spaces were mostly used. I've also noticed some inconsistencies, regarding white spaces between functions names and the parenthesis. However I think I'm overreacting a little, those matters are not really that relevant, but when I'll have some free time I'll get to this as well.


Next in line: Issues

Testing is still giving me some troubles, the show essids part is not yet fully tested, so I couldn't say for sure if everything works as planned.

Apart from the classical: write the image to an USB and reboot method, I'm using Virtual Box for testing. I've managed to create a passthrough for the wireless interface, but now I have troubles with some firmware. Hopefully I'll solve this as soon as possible since it would really make my life easier.


Future plans:

I've realized that I've underestimated a little what the project implies and the amount of information that is new to me. I feel that I have learned a lot and that there is still a lot to learn and I'm very happy that it is so. I've now started documenting regarding IPv6, beginning with this [3]. We haven't decided yet what the next task will be exactly.


[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637128
[1] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/show_essids
[2] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/coding_style
[3] http://tldp.org/HOWTO/Linux+IPv6-HOWTO/

26 June 2012

Petter Reinholdtsen: Debian Edu interview: Jos Luis Redrejo Rodr guez

I've been too busy at home, but finally I found time to wrap up another interview with the people behind Debian Edu and Skolelinux. This time we get to know Jos Luis Redrejo Rodr guez, one of our great helpers from Spain. His effort was the reason we added support for several desktop types (KDE, Gnome and most recently LXDE) in Debian Edu, and have all of these available in the recently published Debian Edu Squeeze version. Who are you, and how do you spend your days? I'm a father, teacher and engineer who is working for the Education ministry of the Region of Extremadura (Spain) in the implementation of ICT in schools How did you get in contact with the Skolelinux/Debian Edu project? At 2006, I verified that both, we in Extremadura and Skolelinux project, had been working in parallel for some years, doing very similar things, using very similar tools and with similar targets, so I decided it was time to join forces as much as possible. What do you see as the advantages of Skolelinux/Debian Edu? A community of highly skilled experts working together, with a really open schema of collaboration and work. I really love the concepts of Do-ocracy and Merit-ocracy and the way these concepts are been used everyday inside Debian Edu. What do you see as the disadvantages of Skolelinux/Debian Edu? Sometimes the differences in the implementations, laws or economical and technical resources in the different countries don't allow us to agree in the same solution for all of us, and several approaches are needed, what is a waste of effort. Also, there is a lack of more man power to be able to follow the fast evolution of the technologies in school. Which free software do you use daily? Debian, of course, and due to my kind of job I am most of my time between Iceweasel, Geany and Terminator. Which strategy do you believe is the right one to use to get schools to use free software? I think there is not a single strategy because there are very different scenarios: schools with mixed proprietary and free environments, schools using only workstations, other schools using laptops, netbooks, tablets, interactive white-boards, etc. Also the range of ages of the students is very broad and you can not use the same solutions for primary schools and secondary or even universities. So different strategies are needed. But, looking at these differences, and looking back to the things we've done and implemented, and the places were we have spent most of our forces, I think we should focus as much as possible in free multi-platform environments, using only standards tools, and moving more and more to Internet or network solutions that could be deployed using wireless. I think we'll see more and more personal devices in the schools, devices the students and teachers will take home with them, so the solutions must be able to be taken at home and continue working there.

25 June 2012

Rapha&#235;l Hertzog: Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers. I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers. The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API. James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It s now available in Debian and in Ubuntu. First, it doesn t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here s a short sample of test code: 
extern "C"  
#include "timer.h"
#include "timefn.h"
 
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
 
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup()  
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
     
 
    void teardown()  
        timer_free(timer);
        /* [...] */
     
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
     
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
     
 
    void ensure_remaining_is(long sec, long nsec)
     
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
     
 ;
 
TEST(Timer, NewIsNotStarted)
 
    CHECK(!timer->started);
 
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
 
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
To run those tests, you just need this boilerplate code in a main.cpp: 
#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
 
   return CommandLineTestRunner::RunAllTests(argc, argv);
Another interesting feature is its integrated memory leak detection system. Any test that hasn t released allocated memory at the end of the teardown process will be marked as failed. The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest. I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg s code. In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

No comment Liked this article? Click here. My blog is Flattr-enabled.

24 June 2012

Gregor Herrmann: RC bugs 2012/25

good news of the RCBW week: more & more patches from people I haven't seen before in the BTS. keep them coming!

23 June 2012

Richard Darst: The size of DebConf12

Phil blogged about DebConf12 having many fewer attendees than past DebConfs (10 and 11). Dealing with registration for several years, I can say these numbers tell a different story.First and foremost, these numbers tell me that DebConf12 hasn't happened yet, DC10 and DC11 have. People are told to sign up register early and reconfirm, but in reality, only the people who have accommodations with us have a significant motivation to do it. Once DebConf starts, lots of day-attendees will drop by, be told to register, and greatly inflate our numbers without staying for a very long time. The number of advance registrants also depends on how much advertisement is done. In terms of people who have both dates in the registration system (Phil's updated graph), I don't place much weight on this number, either.It's my opinion that number of beds occupied is a better indication of the scale of a DebConf, and especially at this stage. This also roughly correlates to number of Debian developers staying, though for years with cheap rooms or lots of external funding, even this number can get inflated. We need some things like t-shirts and badges for every attendee (possibly registered before a certain time), but for now, the number of rooms is what matters.In fact, I wouldn't be surprised if DebConf13 had lower "reconfirmed" numbers, because it is further away form population centers, yet still was a "bigger" conference because there are more people staying on-site or coming for multiple days.For DebConf12, I see an average of 130 people/night, for DebConf11, I see an average of 240 people/night, and for DebConf10, I see an average of 200 people/night. So, DebConf12 is smaller, but in a different way than it seems, and even this is subject to change or variable interpretation. I'm not really wanting to predict DebConf sizes at this stage.So basically, one does not simply expect DebConf registration data to make sense. One should not simply dismiss any DebConf before it happens. And most importantly, one can not simply use this data to plan a conference. The registration team has been frantically running around trying to correct every possible inconsistency you can imagine.Tags: debconf

Bernd Zeimetz: Report from the Bug Squashing Party in Salzburg

bsp_2012_salzburg photo from salzburg-cityguide.com, Copyright (C) Uwe Brandl

Participation and Results From June 15-17th we held a Debian BugSquashingParty in Salzburg, hosted and sponsored by conova communications GmbH. It was a fun and busy weekend, with 15-17 people from 5 countries being around, mainly working on RC bugs in Testing/Unstable. Gerfried Fuchs (rhonda) also worked on triaging the impact of RC bugs on the version in Squeeze, while Peter Palfrader (weasel) took care of Tor related things and Debian sysadmin work, including starting on the new bugs and udd hosts. Phillip Hug (hug) worked on the debian.ch infrastructure. Together with Miroslav Such from Red Hat Bernd Zeimetz (bzed) worked on the packaging of the necessary libraries and daemons to add (basic) Spacewalk client support to Debian. As soon as the packages passed NEW and #677871 was applied (thanks to the APT guys for working on that already), managing Debian clients with Spacewalk should work out of the box. Of course we also had a little keysigning party :)

Statistics
  • about 68 bugs in unstable/testing were triaged/patched/fixed or at least pinged
  • 54 bugs were tagged to show if they affect Squeeze, several other bugs were pinged to retrieve necessary information or to trigger an update in the next stable pointrelease.
  • 5 packages were introduced into Debian (still in NEW, though) - the Spacewalk client related packages and libapache2-mod-auth-memcookie.

Accomodation Thanks to Debian funds we were able to provide accomodation for four participants in the JUFA youth hostel in Salzburg. We had paid in advance for eight, but changing to rooms with a higher category for only 4 people would have been equally or more expensive.

Press/Media coverage Additionally to being mentioned in the calendars on ProLinux and similar pages, we had some press coverage by the local newspaper and online magazines:

Fun facts We consumed 2kg of Leberkas, a big plate of "Buchteln mit Vanillesosse", about 16000cm^2 of Pizza, about 80 litres of coke, juice, beer and wine and I guess we drank at least the same amount of water. We had coffee made of 1.5kg coffee beans and managed to empty the (formerly well filled) icemaker in the fridge. Also we had successful training sessions of a standard Debconf game (rules won't be explained here obviously). Maybe we even successfully spread the game to the employees of a commercial linux distribution ;)

22 June 2012

Tanguy Ortolo: We need you for the NTP pool!

Ferdinand Berthoud's marine clock #2 You already know NTP, the protocol that allows your computers to keep an accurate clock. Thanks to a global pool of time servers, it is available to everyone on the Internet. This public pool is constituted by volunteers' servers and it needs more.So, if you you should consider joining the pool. It is actually quite easy:
  1. install NTP;
  2. configure it to use five good servers from your region;
  3. register your server.

21 June 2012

Russell Coker: New SE Linux Policy for Wheezy

I ve just uploaded a new SE Linux policy for Debian/Wheezy. It now works correctly with systemd and Chromium, two significant features that I wanted for Wheezy. Now it turns out that we have until the end of the month for Wheezy updates, so I may get another version of the policy uploaded before then. If so it will only be for relatively minor changes, I think that most SE Linux users would be reasonably happy with policy the way it is. Anything that doesn t work now can probably be solved by local configuration changes. execmem The current version of KDE in Debian is 4.8.4, it seems that large parts of the KDE environment depend on execmem access, this includes kwin and plasma-desktop. Basically there is no possibility of having a KDE desktop environment without those programs and therefore KDE depends on execmem access. Debugging this is difficult as the important programs SEGV when denied execmem access and the KDE crash handler really gets in the way of debugging it running /usr/bin/plasma-desktop results in the process forking a child and detaching from the gdb session. The most clear example of an execmem issue in KDE is from the program /usr/lib/kde4/libexec/kwin_opengl_test which gives the following error:
LLVM ERROR: Allocation failed when allocating new memory in the JIT
Can t allocate RWX Memory: Permission denied To make this work you run the command setsebool -P allow_execmem 1 which gives many domains the ability to create writable-executable memory regions. I raised this issue for discussion on the SE Linux mailing list and Hinnerk van Bruinehsen wrote an informative message in response summarising the situation [1]. It seems that it s possible to compile some of the programs in question to not use the JIT and therefore not require such access and there is a build option in Gentoo to allow it. But it s impractically difficult for me to fork KDE in Debian so the only option is to recommend that people enable the allow_execmem boolean for Debian desktop systems running SE Linux. Related posts:
  1. /run and SE Linux Policy Currently Debian/Unstable is going through a transition to using /run...
  2. An Update on DKIM Signing and SE Linux Policy In my previous post about DKIM [1] I forgot to...
  3. New SE Linux Policy for Squeeze I have just uploaded refpolicy version 0.2.20100524-1 to Unstable. This...

20 June 2012

DebConf team: Call for DebConf14 venues! (Posted by Gunnar Wolf)

2000, Bordeaux, France. 2001, Bordeaux, France. 2002, Ottawa, Canada. 2003, Oslo, Norway. 2004, Porto Alegre, Brazil. 2005, Helsinki, Finland. 2006, Oaxtepec, Mexico. 2007, Edinburgh, United Kingdom. 2008, Mar del Plata, Argentina. 2009, C ceres, Spain. 2010, New York, United States. 2011, Banja Luka, Bosnia and Herzegovina. 2012, Managua, Nicaragua. 2013, Vaumarcus, Switzerland. We have gone to all different latitudes (although not yet longitudes!). We have had very hot and very cold weather. Great cities and small villages have hosted us. What will be the next addition to this great list? As we approach DebConf12 (10 days to DebCamp and counting, yay!), we should keep the future in mind. So, as every year in recent history, we will once again have a DebConf session presenting possible venues for the next year. Organizing DebConf in your city means a lot of hard work. It also means one of the greatest personal experiences you can imagine. And it is a great way to contribute to Debian. The decision process for every DebConf venue starts two years before, with presentations during DebConf(n-2). That means, if you consider presenting a bid for DebConf14, now is the moment to act! Do you have to be present at Managua to propose your bid? No. You can proxy via somebody I d suggest to do it via somebody who knows the location you are suggesting, but basically, choose a friend that you trust that trusts you. Of course, you can participate in the presentation session via IRC. Do you have to be a Debian Developer to propose a bid? No. For DebConf9, none of the C ceres guys was a DD; for DebConf10, some of the people most involved from the local New Yorkers were not DDs. For DC11, none of our hosts in Bosnia are DDs. And for DC12, the dear and overworked Nicaraguan crew is also made from people interested in getting closer to the Debian project, but not DDs. Do you have to decide now? No. This is just a call for a first presentation, but the decision regarding DC14 will be taken probably around March 2013. However, giving a nice presentation at DebConf helps a lot, gives you visibility, and will get the ball rolling. Is there a geographical bias? Slight. So far, and since the second DebConf, we have kept the tradition not to repeat continents on two successive DebConfs. This is not a hard condition, however! While there is some probability that DebConf14 will be somewhere in America (the continent, of course), it depends on the proposals more than on any pseudo-rule. What do you need to start thinking about? Go visit our prospective location checklist. You can also look at what other teams have historically presented. And of course, go to the DebConf14 wiki planning page Register there, even if you are just in the early phases of finding data.

Next.

Previous.