Search Results: "tar"

30 June 2012

Leandro Gómez: DebCamp -1

DebCamp starts tomorrow and the local team is relaxing and enjoying some free time with a cup of Debian Coffee. See you soon in Managua! (In case you wonder yes!!! both the Debian Coffee and the DebConf coffee mug will be for sale during DebConf12 in Managua)

Bernhard R. Link: ACPI power button for the rest of us

The acpi-support maintainer unfortunately decided 2012-06-21 that having some script installed by a package to cleanly shut down the computer should not be possible without having consolekit and thus dbus installed. So (assuming this package will migrate to wheezy which it most likely will tomorrow) with wheezy you will either have to write your own event script or install consolekit and dbus everywhere. You need two files. You need one in /etc/acpi/events/, for example a /etc/acpi/events/powerbtn:
event=button[ /]power
action=/etc/acpi/powerbtn.sh
Which causes a power-button even to call a script /etc/acpi/powerbtn.sh, which you of course also need:
#!/bin/sh
/sbin/shutdown -h -P now "Power button pressed"
You can also name it differently, but /etc/acpi/powerbtn.sh has the advantage that the script from acpi-support-base (in case it was only removed and not purged) does not call shutdown itself if it is there. (And do not forget to restart acpid, otherwise it does not know about your event script yet). For those too lazy I've also prepared a package acpi-support-minimal, which only contains those scripts (and a postinst to restart acpid to bring it into effect with installation), which can be get via apt-get using
deb http://people.debian.org/~brlink/acpi-minimal wheezy-acpi-minimal main
deb-src http://people.debian.org/~brlink/acpi-minimal wheezy-acpi-minimal main
or directly from http://people.debian.org/~brlink/acpi-minimal/pool/main/a/acpi-support-minimal/. Sadly the acpi-support maintainer sees no issue at all and ftp-master doesn't like so tiny packages (which is understandable but means the solution is more than a apt-get away).

29 June 2012

Jo Shields: How to build a bungalow begin with the chimney

Whilst nobody was looking, Mono became the most important framework in the world of games. And I mean that with only a smidge more than my usual levels of hyperbole. It really is fantastically important, as far as game development goes. I blogged a long time ago about its use inside EA s multi-million-selling The Sims 3. And the massive performance gains in Second Life when they switched from their in-house scripting language to Mono are well documented. But I m not talking about those, I m talking about the now, the recent stuff. First up is Unity 3D. Not to be confused with the Ubuntu user interface, Unity 3D is a proprietary game development tool which began life for Mac, and has now managed to become the single most popular game engine amongst smartphone developers. It targets a bunch of new platforms not just Android and Windows and iOS, but in the new 4.0 release, Linux too for the first time. And every Unity 3D game is a Mono game the core of Unity is a fork of Mono, which enables it to have the performance and ease of development which it does. A reasonably large percentage of the cross-platform games on Kickstarter, such as Wasteland 2 and Super Retro Squad, are cross-platform because Unity 3D makes it easy and each of those games is powered by Mono. However, I have better things to do today than gush about a proprietary tool. No, far more important are the Free ways to develop games. So today I m going to talk about MonoGame. MonoGame is an implementation of Microsoft XNA a high level framework for developing in .NET languages, originally designed to allow indie developers access to the Xbox 360. XNA covers many of the bases covered by SDL sound, graphics, input, and so on, but can go onto platforms where SDL isn t an option, such as the aforementioned Xbox 360. And with MonoGame, any game written for XNA is a game which can not only run on Microsoft s approved platforms Windows, Xbox 360, Windows Phone but also on other platforms such as Mac OS X, iPhone, iPad, Android, and Linux. Cross-platform is good. More games are good. And MonoGame enables every Xbox 360 indie developer to throw a Linux port out of the door with a relative minimum of fuss. There are 2,521 games on the Xbox Live Indie Games store (games by individuals or small groups, with no major publisher) and each of those can become a game for Debian and Ubuntu with very little work. Not to mention the games with actual publishers (some portion of the 492 games on XBLA), not to mention the mobile games written for Windows Phone, Android and iOS. All of these developers have an easy route to Linux releases, as long as the effort remains low. And with today s publishing in the Debian archive of MonoGame 2.5.1, the effort is as low as can be. Anyone who purchased the recent Humble Indie Bundle 5, and played Bastion, has seen MonoGame in action Bastion began life as an XNA game for Xbox 360, and the Linux port was made possible only by MonoGame. Bastion s 100+ industry awards should indicate the possibilities open to a creative mind with decent developer tools. MonoGame is Free Software, built on top of other Free Software, with an active upstream development team genuinely responsive to the needs of Linux distributions. It s well worth a look not just for porting, but for new projects too. And I m delighted that it will be available to all users of Debian 7.0, as well as Ubuntu 12.10.

Michal Čihař: New job, new challenges

Starting next week, I'll start at new job. I will still stay at SUSE, so it's not that big change, but still it is new challenge for me. I'll be moving out of L3 department, where I've spend almost four years. L3 is really a great team where man can learn basically anything, simply because you can meet there anything from broken swap on S390 up to wrong icons in LibreOffice :-). The new position is inside Security team, still I'll be mostly focused on writing some internal tools, rather than doing security work (at least for first year or so). The current plan is base on work I've done for L3 tools and use Django for the web interface, but that's just a plan for now and might change in the future. Anyway looking forward to new challenges at new job.

Filed under: Django English Suse 0 comments Flattr this!

28 June 2012

Sylvain Le Gall: OASIS 0.3.0 release

Logo OASIS small OASIS is a tool to integrate a configure, build and install system in your OCaml project. It helps to create standard entry points in your build system and allows external tools to analyse your project easily. It is the building brick of OASIS-DB, a CPAN for OCaml. This release takes almost 18 months to complete. This is too long and I will talk in another blog post on the way I am trying to improve this right now (esp. using continuous integration). This new release fixes a small bug (1 line) that prevents setup.ml to run with OCaml 4.00. If you have projects that was generated with a former release consider upgrading to OASIS 0.3.0. There are several big new stuff that comes with this release. It now supports Pack: field for libraries which allows to pack your library using -for-pack and so on. We are also compiling .cmxs (dynlink object for native) by default and we publish them in the META file. This feature was implemented in order to get more libraries to provide .cmxs and to help project like Ocsigen to take advantage of this. If you want to get rid of this at configure time, you can use ./configure --override native_dynlink false. We introduce two new default flags: --enable-tests and --disable-docs for configure. These are implicit flags that define if we will run Test sections or compile Document sections. They are especially useful to reduce the number of dependencies, because dependencies of Test will be excluded by default. We recommend to set Build$: flag(tests) to any Library or Executable sections that are only useful for tests. This allows you to really cut down the number of dependencies. The last change I want to introduce is about the old setup-dev subcommand which is now deprecated. It has been replaced by 2 different update schemes. I am pretty excited by this feature which in fact comes from OASIS user (esp. Lwt project). The former scheme was to have a big 'setup.ml' that always call the command oasis to update. This was complex and not very useful. We now have 2 mode: dynamic and weak. dynamic allow you to have a small setup.ml and to keep your VCS history clean but you need to install OASIS. weak need a big setup.ml but only need to call the command oasis if someone change something in _oasis. This mode is targeted to project that wants to be able to checkout from VCS an OCaml project without installing OASIS. The difference generated by weak mode doesn't pollute too much the VCS history because most the time they make sense. For example, you upgrade your package version number in _oasis and it produces a change with 6 lines where the version number changes in every META, setup.ml files and so on. This version has been tested on: You can download it here or use your favorite package manager: Debian (UPDATE: pending upload)
$ apt-get install -t experimental oasis
GODI
$ godi_console perform -build apps-oasis
odb.ml
$ ocaml odb.ml oasis
Here is the complete changelog: EXTREMLY IMPORTANT changes (read this) PACKAGES uploaded to oasis-db will be automatically "derived" before OCaml 4.00 release (i.e. oUnit v1.1.1 will be regenerated with this new version as oUnit v1.1.1~oasis1). PACKAGES not uploaded to oasis-db need to be regenerated. In order not to break 3rd party tools that consider a tarball constant, I recommend to create a new version. Thanks to INRIA OCaml team for synchronizing with us on this point. Major changes: You can now have the following example:
     ...
     Executable test_exec
       Install: false
       Build$: flag(tests)
       MainIs: testExec.ml
       BuildDepends: oUnit
     
     Test main
       Command: $test_exec
       TestTools: test_exec
     ...
The Run$: flag(tests) is implicit for the section Test main. The default value is false for tests. If all the executable for test are flagged correctly (Build$: flag(tests)), you'll get rid of the dependency on oUnit. It works the same for documentation, however the default is true. (Closes: #866) In order to allow interdependent flags, we transform back lazy values into unit -> string functions. This allows to change a flag value on the command line and to update all the dependent values. (Closes: #827, #938) It defines different ways to manage the auto-update of setup.ml: The choice between weak and dynamic depends on your need with regard to VCS and to the presence of oasis. The weak allow to checkout the project from VCS and be able to work on it, without the need of installing 'oasis' as long as you don't change the file _oasis. But it clutter your VCS history with changes to the build system each time you change something in _oasis. The 'dynamic' mode gives you no VCS history pollution but makes mandatory to have installed oasis libraries. Avoid copying executable to their real name. It helps to call ocamlbuild a single time for the whole build rather than calling it n time (n = number of executable sections) and copying resulting exec. This speeds up the build process because ocamlbuild doesn't have to compute/scan dependencies each time. The drawback is that you have to use $foo when you want to call Executable foo, because $foo will be '_build/.../main.byte'. For example:
CCOpt: -DEXTERNAL_EXP10 -L/sw/lib "-framework vecLib"
Will be parsed correctly and outputed according to target OS. In order to ease building oasis, we have minimize the number of dependencies. You only need to install ocamlmod, ocamlify and ocaml-data-notation for a standard build without tests. Dependencies on pcre, extlib and ocamlgraph has been dropped. The remaining dependencies are hidden behind a flag tests. OASIS now produces .cmxs file by default and add them to META. Now a META looks like:
     ...
     archive(byte) = "oasis.cma"
     archive(byte, plugin) = "oasis.cma"
     archive(native) = "oasis.cmxa"
     archive(native, plugin) = "oasis.cmxs"
     ...
This will ultimately help to generate automatically .cmxs for all oasis enabled projects. We hope that this new feature will improve dynamic linking use in OCaml (esp. for project like Ocsigen). Other changes Thanks to Anil Madhavapeddy, Pierre Chambart, Christophe Troestler, Jeremie Dimino, Ronan Le Hy, Yaron Minsky and Till Varoquaux for their help with this release. Also thanks to all the testers of the numerous release candidates. This was a long work and each time a tester has downloaded oasis has helped me to know that I was working for someone.

Sylvain Le Gall: Installing a new SSD on my laptop

3 years ago I bought a OCZ Core v2 SSD... I was very disappointed by write freeze, price and capacity. Recently a friend of mine, told me that I should try the new ones that has much improved in the meantime. So I decided to give another try to SSD on my laptop. I bought a Crucial M4 128GB SSD, because they seems to be of better quality than OCZ (and probably because of my first failed attempt). The price is correct (~130 ) and after some tests it seems quite good at write and read speed. So from this point of view, I am more convinced than my first attempt. Now, the real problem of the SSD is how to migrate your old data to your new drive. I have always upgraded my HD to a larger one and so a simple dd if=/dev/sda of=/dev/sdb bs=32M was enough... In the case of SSD you have to migrate to a smaller capacity. This includes to juggle a little bit with your partition to get something that fits. In order to improve a little bit the lifetime of my SSD, I have tried to follow various advice you can found on the Internet. One of this advice that seems to make sense, is to align your partition to a number modulo 2048 (rather than the default which is to start at 63). I followed this forum to do that. I end up with all partitions aligned to 2048. The GOOD news: doing data migration for Linux is a breeze. As a matter of fact, Linux support pretty well being move around, ending up on a different partition and so on... It just took me the time to transfer data, update grub and disk UUID and reboot. The BAD news: doing data migration for Windows is a hell. I always keep a Windows partition to test various OCaml software. But Windows doesn't like you like to be moved around. It especially doesn't like that the first sector of its partition get moved (i.e. from sector 63 to sector 2048). I spend a week trying to 'repair windows' but it was a pure waste of time. Thinkpad comes with a special ways to store passwords which is not compatible with the rescue mode of Windows, so you cannot access the repair mode -- because you need the Administrator password -- which is not readable... I decided to get back to the standard first sector on 63... Now Windows complained about 'hal.dll' missing. You know why ! Because the partition number has changed (from /dev/sda1 to /dev/sda3). Boot in Linux, run fdisk, enter expert mode, fix order -- after having remounted read-only the root partition and unmount everything else. Grub complains a little bit, fall into rescue mode. I fix that following these instructions and running 'grub-install --recheck /dev/sda'. So after 2 weeks of fights, I end up with a working windows (honestly, this is a pain, don't try to play with Windows partition, this OS is a lot more sensible sensitive than Linux).

DebConf team: DebConf12 schedule ready! (Posted by Gunnar Wolf)

The DebConf12 schedule is finally available! As usual, the DebConf conference can only be finalized a couple of days before DebConf itself starts. But hey Take a look at it, and start planning your activities! Not final, never final Remember that, even now that the conference schedule is announced, you can still submit talks to DebConf, following what we call ad-hoc talks. What are the steps to do it?
  1. Check the schedule and find the timeslot (day, hour and room) you want to talk in
  2. Register your event in Pentabarf, and note the generated event ID (the last number in the URL after the submission is accepted)
  3. Send a mail to schedule@debconf.org detailing the event ID, title (just to confirm it), and the requested timeslot
  4. All requests _must_ be received at least 24hr before the requested event for the scheduling team to act accordingly and for the video team to be able to plan on resources to cover the requested session
  5. Video team coverage (event recording plus streaming on a best-effort basis) will be given to activities scheduled on the Aula Magna and Roberto Ter n auditoriums.
  6. We will try to grant requests on a first-come-first-serve basis. If two events are requested for the same timeslot, the first will have precedence, and we will schedule the second as close as possible to it. We will notify the submitters if this happens, of course.
Conference tracks In DebConf12, many talks are organized around the following tracks:
DebianDay
This track is meant for the local (Nicaraguan/Central American) community to be attracted to Debian. It does not usually have technically minded talks; the DebConf general audience is, of course, invited to take part in it, as there will surely be interesting angles on how Debian is used in different use-cases, and there will be interesting Q&A session with the audience.
Getting involved in Debian
Central America is a very special region. It has vibrant Free Software user groups and communities, among which there is an important interest to get involved in Debian.
This track will have some talks in Spanish, some talks in English, explaining different Debian processes and tools, aiming at attracting locals to contribute to Debian.
Debian for the cloud
All about virtualization. Management infrastructure, details on virtual machine engines, insight on how to tweak your kernel for optimum virtualized performance, and of course how to get it all humming perfectly with your Debian packages.
Building and porting
With an every day stronger transit to mobile computing, we are seeing a renaissance of porting. At the same time, we finally have multiarch as part of Debian. And were that not to be enough, we finally have more than one compiler that can be used for most of the software in the archive. Put all this together, and you will have a very interesting track.
Social activities
DebConf is as much about technical stuff as it is about building closer ties within the members of its community. Eating cheese+wine, going on a trip to the guts of a volcano, having a nice banquet Do you want to add an activity to this pseudo-track?
Skills exchange
Knowledge can also be built by all of us putting our grain of sand. We first had explicit skills exchange sessions in Bosnia last year, and they were very well received. Feel free to propose similar sessions!
And Well, three days to DebCamp and counting. See you Real Soon Now in Managua!

Sorina Sandu: Status update

Hello!

First of all: I'm going to DebConf :D and I really want to say a big "Thank you!" to every one who made this possible.


So, here's what I've done recently:

I spent some time putting everything in order for the departure to Nicaragua, things like finding a plane ticket through a route that does not require a Visa, making a new Passport, now all is in order.

I've integrated the changes for listing available ESSIDs, in order to close #637128 [0] into a separate branch , which can be found here [1].

The plan is to have only one new template which lists the available networks and which has a option for entering the ESSID manualy. In order to do so, a simple scan is performed on the wireless interface, and the configuration for a particular network resulted after the scan is kept if that one is chosen. The enter manually situation is treated exactly as it was before.

Furthermore, last week (or the week before) I've done some coding style fixes again in a separate branch [2], nothing very fancy, just trimming trailing white spaces and changing some tabs to spaces, where spaces were mostly used. I've also noticed some inconsistencies, regarding white spaces between functions names and the parenthesis. However I think I'm overreacting a little, those matters are not really that relevant, but when I'll have some free time I'll get to this as well.


Next in line: Issues

Testing is still giving me some troubles, the show essids part is not yet fully tested, so I couldn't say for sure if everything works as planned.

Apart from the classical: write the image to an USB and reboot method, I'm using Virtual Box for testing. I've managed to create a passthrough for the wireless interface, but now I have troubles with some firmware. Hopefully I'll solve this as soon as possible since it would really make my life easier.


Future plans:

I've realized that I've underestimated a little what the project implies and the amount of information that is new to me. I feel that I have learned a lot and that there is still a lot to learn and I'm very happy that it is so. I've now started documenting regarding IPv6, beginning with this [3]. We haven't decided yet what the next task will be exactly.


[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637128
[1] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/show_essids
[2] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/coding_style
[3] http://tldp.org/HOWTO/Linux+IPv6-HOWTO/

27 June 2012

Michal Čihař: Weblate - over 20000 translations in 3 months

Today, I wondered how much translations were done using Weblate since it was born. Of course I can give only numbers for services I maintain, the biggest one being translation server for phpMyAdmin, Gammu and Weblate. Looking at the logs, the server l10n.cihar.com helped to contribute over 20000 (actually little bit more as tracked are only changes after releasing Weblate 0.8), what is quite impressive number. Of course biggest amount of contributions came in time of preparing phpMyAdmin 3.5.0 release, where I gave it huge amount of publicity (AKA spamming my blog), but number of people accessing the website still grows since then (still number of submitted translations is slightly lower). Now as 1.0 release proved to be quite stable, it's time to focus on 1.1 development, which will bring offloading of fulltext indexing (it was proven to be unreliable to update index online for bigger sites). Almost all issues targeted for 1.1 are already fixed and all what needs to be done is testing in real life situations.

Filed under: English Phpmyadmin Weblate 0 comments Flattr this!

26 June 2012

Neil McGovern: Digital Economy Act And so it begins

Today, OFCOM issued a press release entitled "New measures to protect online copyright and inform consumers". According to the announcement, internet users will be encouraged to download music and films through legal channels under the measures. Ofcom has published a draft code for consultation that would require large internet service providers (ISPs) to inform customers of allegations that their internet connection has been used to infringe copyright. There does seem to have been a number of concessions made, which is welcome, but I do still have a number of concerns. The code 'only' covers ISPs with more than 400k fixed lines - BT, Everything Everywhere, O2, Sky, TalkTalk Group and Virgin Media. This covers about 93% of the UK retail market. If an allegation of copyright infringement is made, the ISP would need to write to the customer with notice that an infringement has occurred, and in a new change, the number of copyright infringement reports connected to their account. If more than three letters are received in a year, anonymous information may be provided on request to copyright owners showing them which infringement reports are linked to that customer s account. The copyright owner may then seek a court order requiring the ISP to reveal the identity of the customer. Two more changes have been implemented: Obviously the former of these will require further scrutiny, but I'm happier that this is not with the vested interest themselves. As for the latter, this gets a bit interesting. The Act states that
an appeal on any grounds must be determined in favour of the subscriber unless the copyright owner or internet service provider shows that a) the apparent infringement was an infringement of copyright, and b) the report relates to the subscriber s IP address at the time of that infringement.
This is good as it puts the burden of proof on the copyright owner, although it still uses IP addresses which is problematic. For example, New York Judge Gary Brown has ruled IP addresses are insufficient evidence to identify pirates, and has provided a lengthy and thoughtful explanation in his ruling. There has also been a consultation published on the how to share costs costs. The suggestion is fairly simple:
"Copyright Owners should bear all of the costs incurred by Ofcom, the majority of costs incurred by the appeals body, and 75% of the costs efficiently and reasonably incurred by Qualifying ISPs in carrying out their obligations."
This has the risk of driving up internet access prices as ISPs have to recover the costs of these notifications. Additionally:
"Under the Order, subscribers will have to pay a 20 fee to make an appeal against a report of infringement, which will be refunded in the event the appeal succeeds; the remaining costs of determining an appeal will be met by the Copyright Owner who submitted the CIR which has been appealed."
I disagree with the concept of having to pay for an appeal in this way. If accused, someone should be able to refute the argument without cost. This will introduce a barrier to entry, and even if it is refundable is something I believe should be avoided. Finally, the exact costs of implementing this have been set out. It looks like the Copyright Owners would have to pay 60p per letter (with the ISPs shouldering the remaining 20p), a similar split around capital costs - which are the ones that are actually significant. Taking the above, it seems that in total that content owners will have to shell out 15M to use the DEA scheme. The cost seems to be more or less fixed which could encourage content owners to send out frivolous accusations. There's an iterative process which will run to determine the estimated number of notifications, and if eventually the Content Owners don't agree with the pricing, they will have elected not to take advantage of the provisions of the Initial Obligations Code, as they have not been able to make binding commitments to fund CIR processing. So, in summary, this is something to keep an eye on. It certainly hasn't gone away, it's due to start in March 2014. The revised draft code and consultation, which closes on 26 July 2012, can be found here, and the sharing of costs consultation runs until 18th September 2012, found here.

Petter Reinholdtsen: Debian Edu interview: Jos Luis Redrejo Rodr guez

I've been too busy at home, but finally I found time to wrap up another interview with the people behind Debian Edu and Skolelinux. This time we get to know Jos Luis Redrejo Rodr guez, one of our great helpers from Spain. His effort was the reason we added support for several desktop types (KDE, Gnome and most recently LXDE) in Debian Edu, and have all of these available in the recently published Debian Edu Squeeze version. Who are you, and how do you spend your days? I'm a father, teacher and engineer who is working for the Education ministry of the Region of Extremadura (Spain) in the implementation of ICT in schools How did you get in contact with the Skolelinux/Debian Edu project? At 2006, I verified that both, we in Extremadura and Skolelinux project, had been working in parallel for some years, doing very similar things, using very similar tools and with similar targets, so I decided it was time to join forces as much as possible. What do you see as the advantages of Skolelinux/Debian Edu? A community of highly skilled experts working together, with a really open schema of collaboration and work. I really love the concepts of Do-ocracy and Merit-ocracy and the way these concepts are been used everyday inside Debian Edu. What do you see as the disadvantages of Skolelinux/Debian Edu? Sometimes the differences in the implementations, laws or economical and technical resources in the different countries don't allow us to agree in the same solution for all of us, and several approaches are needed, what is a waste of effort. Also, there is a lack of more man power to be able to follow the fast evolution of the technologies in school. Which free software do you use daily? Debian, of course, and due to my kind of job I am most of my time between Iceweasel, Geany and Terminator. Which strategy do you believe is the right one to use to get schools to use free software? I think there is not a single strategy because there are very different scenarios: schools with mixed proprietary and free environments, schools using only workstations, other schools using laptops, netbooks, tablets, interactive white-boards, etc. Also the range of ages of the students is very broad and you can not use the same solutions for primary schools and secondary or even universities. So different strategies are needed. But, looking at these differences, and looking back to the things we've done and implemented, and the places were we have spent most of our forces, I think we should focus as much as possible in free multi-platform environments, using only standards tools, and moving more and more to Internet or network solutions that could be deployed using wireless. I think we'll see more and more personal devices in the schools, devices the students and teachers will take home with them, so the solutions must be able to be taken at home and continue working there.

Ingo Juergensmann: Confusion about mkfs.xfs and log stripe size being too big

Recently I bought some new disks, placed them into my computer, and built a RAID5 on these 3x 4 TB disks. Creating a physical device (PV) with pvcreate, a volume group (VG) with vgcreate and some logical volumes (LV) with lvcreate was as easy and well-known as creating an XFS filesystem on the LVs... but something was strange! I never saw this message before, when creating XFS filesystems with mkfs.xfs:
log stripe unit (524288 bytes) is too large (maximum is 256KiB)
log stripe unit adjusted to 32KiB
Usually I don't mess around with the parameters of mkfs.xfs, because mkfs.xfs is smart enough to find near to optimal parameters for your filesystem. But apparently mkfs.xfs wanted to use a log stripe unit of 512 kiB, although its maximum size for this is 256 kiB. Why? So I started to google and in parallel asked on #xfs@freenode. Erik Sandeen, one of the core developers of XFS, suggested that I write that issue to the mailing list. He did already face this issue himself, but couldn't remember details. So I collected some more information about my setup and wrote to the XFS ML. Of course I included information about my RAID5 setup:
muaddib:/home/ij# mdadm --detail /dev/md7
/dev/md7:
Version : 1.2
Creation Time : Sun Jun 24 14:58:21 2012
Raid Level : raid5
Array Size : 7811261440 (7449.40 GiB 7998.73 GB)
Used Dev Size : 3905630720 (3724.70 GiB 3999.37 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent Update Time : Tue Jun 26 05:13:03 2012
State : active, resyncing
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0 Layout : left-symmetric
Chunk Size : 512K Resync Status : 98% complete Name : muaddib:7 (local to host muaddib)
UUID : b56a714c:d193231e:365e6297:2ca61b65
Events : 16 Number Major Minor RaidDevice State
0 8 52 0 active sync /dev/sdd4
1 8 68 1 active sync /dev/sde4
2 8 84 2 active sync /dev/sdf4
Apparently, mkfs.xfs takes the chunk size of the RAID5 and want to use this for its log stripe size setting. So, that's the explanation why mkfs.xfs wants to use 512 kiB, but why is the chunk size 512 kiB at all? I didn't messed around with chunk sizes when creating the RAID5 either and all of my other RAIDs are using chunk sizes of 64 kiB. The reason was quickly found: the new RAID5 has a 1.2 format superblock, whereas the older ones do have a 0.90 format superblock. So, it seems that somewhen the default setting in mdadm, which superblock format is to be used for its metadata, has been changed. I asked on #debian.de@ircnet and someone answered that this was changed in Debian after release of Squeeze. Even in Squeeze the 0.90 format superblock was obsolete and has been only kept for backward compatibility. Well, ok. There actually was a change of defaults, which explains the behaviour of mkfs.xfs now, wanting to set log stripe size to 512 kiB. But what is the impact of falling back to 32 kiB log stripe size? Dave Chinner, another XFS developer explains:
Best thing in general is to align all log writes to the
underlying stripe unit of the array. That way as multiple frequent
log writes occur, it is guaranteed to form full stripe writes and
basically have no RMW overhead. 32k is chosen by default because
that's the default log buffer size and hence the typical size of
log writes.

If you increase the log stripe unit, you also increase the minimum
log buffer size that the filesystem supports. The filesystem can
support up to 256k log buffers, and hence the limit on maximum log
stripe alignment.
And in another mail, when being asked if it's possible to raise the 256 kiB limit to 512 kiB because of the mdadm defaults to 512 kiB as well:
You can't, simple as that. The maximum supported is 256k. As it is,
a default chunk size of 512k is probably harmful to most workloads -
large chunk sizes mean that just about every write will trigger a
RMW cycle in the RAID because it is pretty much impossible to issue
full stripe writes. Writeback doesn't do any alignment of IO (the
generic page cache writeback path is the problem here), so we will
lamost always be doing unaligned IO to the RAID, and there will be
little opportunity for sequential IOs to merge and form full stripe
writes (24 disks @ 512k each on RAID6 is a 11MB full stripe write).

IOWs, every time you do a small isolated write, the MD RAID volume
will do a RMW cycle, reading 11MB and writing 12MB of data to disk.
Given that most workloads are not doing lots and lots of large
sequential writes this is, IMO, a pretty bad default given typical
RAID5/6 volume configurations we see....
So, reducing the log stripe size is in fact a good thing[TM]. If anyone will benefit from larger log stripe sizes, s/he would be knowledgeable enough to play around with mkfs.xfs parameters and tune them to needs of the workload. Erik Sandeen suggested, though, to remove the warning in mkfs.xfs. Dave objects and maybe it's a good compromise to extend the warning by giving an URL for a FAQ entry explaining this issue in more depth than a short warning can do? Maybe someone else is facing the same issue and searches for information and find this blog entry helpful in the meantime...
Kategorie:

25 June 2012

Raphaël Hertzog: Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers. I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers. The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API. James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It s now available in Debian and in Ubuntu. First, it doesn t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here s a short sample of test code:
extern "C"  
#include "timer.h"
#include "timefn.h"
 
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
 
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup()  
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
     
 
    void teardown()  
        timer_free(timer);
        /* [...] */
     
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
     
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
     
 
    void ensure_remaining_is(long sec, long nsec)
     
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
     
 ;
 
TEST(Timer, NewIsNotStarted)
 
    CHECK(!timer->started);
 
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
 
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
 
To run those tests, you just need this boilerplate code in a main.cpp:
#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
 
   return CommandLineTestRunner::RunAllTests(argc, argv);
 
Another interesting feature is its integrated memory leak detection system. Any test that hasn t released allocated memory at the end of the teardown process will be marked as failed. The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest. I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg s code. In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Jan Wagner: nagios-plugins 1.4.16 is going to be released

Short before Debian is freezing the upcoming release of a new version of nagios-plugins is scheduled for Wednesday. The good news is, that a recent version is available in unstable and testing. Upstream only fixed some check_ping issues, which are not included in nagios-plugins 1.4.16~pre1-1. There is also a package available through squeeze-backports. If you are able to, please test the packages as soon as possible. If there are some quirks, these can be fixed in the next two days upstream.

Robert Collins: Running juju against a private openstack instance.

My laptop has somewhat less than 1/2 the grunt of my desktop at home, but I prefer to work on it as I can go sit in the sun etc, very hard to do that with a mini tower case :) However, running everything through ssh to another machine makes editing and iterating more clumsy; I need to do agent forwarding etc not terribly hard, but not free either, particularly when I travel, I need to remember to sync my source trees back to my laptop. So I prefer to live on my laptop and use my desktop for compute power. I had a couple of Juju charms I wanted to investigate, but I needed enough compute power to make my laptop really quite warm so I thought, its time to update my local cloud provider from Eucalyptus to Openstack. This was easy enough, until I came to run Juju. Turns out that Juju s commands really want to talk to the public DNS name of the instance (in order to SSH tunnel a connection to Zookeeper). But! Openstack returns DNS names like Server-3 , and if you think about a home network, its fairly rare to have a local DNS server *anyway*, so putting a suffix on names like that won t help at all: you either need to use a DNS naming provider (openstack ships with an LDAP provider, which adds even more complexity), and configure your clients to know how to find it, or you need to use the public IP addresses (which default to the FlatNetwork, which is routable within a home LAN by simply adding a route to 10.0.0.0/8 to your wifi interface). Adding to confusion, some wifi routers fail to forward avahi messages, which is a) terrible and b) breaks the only obvious way of doing no-config local DNS :( . So, I did some yak shaving this morning. Turns out other folk have already run into this and filed a Juju bug and a supporting txaws bug. The txaws bug was fixed, but just missed the release of Precise. Clint Byrum is going to SRU it this week though, so we ll have it soon. I ve put a patch up to address the Juju side, which is now pending review. Running the two together works very happily for me. \o/

24 June 2012

Joey Hess: trying obnam

Obnam 1.0 was released during several months when I had no well-connected server to use for backups. Yesterday I installed a terabyte disk in a basement with a fiber optic network connection, so my backupless time is over. Now, granted, I have a very multi-layered approach to backups; all my data is stored in git, most of it with dozens of copies automatically maintained, and with archival data managed by git-annex. But I still like to have a "real" backup system underneath, to catch anything else. And to back up those parts of my user's data that I have not given them tools to put into git yet... My backup server is not my basement, so I need to securely encrypt the backups stored there. Encrypting your offsite backups is such a good idea that I've always been surprised at the paucity of tools to do it. I got by with duplicity for years, but it's increasingly creaky, and the few times I've needed to restore, it's been a terrific pain. So I'm excited to be trying Obnam today. So far I quite like it. The only real problem is that it can be slow, when there's a transatlantic link between the client and the server. Each file backed up requires several TCP round-trips, and the latency kills the bandwidth. Large files are still sent fast, and obnam uses little resources on either the client or server while running. And this mostly only affects the initial, full backup. But the encryption and ease of use more than make up for this. The real killer feature with Obnam's encryption isn't that it's industry-standard encryption with gpg, that can be trivially enabled with a single option (--encrypt-with=DEADBEEF). No, the great thing about it is its key management. I generate a new gpg key for each system I back up. This prevents systems reading each other's backups. But that means you have to backup the backup keys.. or when a system is lost, the backup would be inaccessible. With Obnam, I can instead just grant my personal gpg key access to the repository: obnam add-key --keyid 2512E3C7. Now both the machine's key and my gpg key can access the data. Great system; can't revoke access, but otherwise perfect. I liked this so much I stole the design and used it in git-annex too. :) I'm also pleased I can lock down .ssh/authorized_keys on my backup server, to prevent clients running arbitrary commands. Duplicity runs ad-hoc commands over ssh, which defeated me from ever locking it down. Obnam can be easily locked down, like this: command="/usr/lib/openssh/sftp-server" This could still be improved, since clients can still read the whole filesystem with sftp. I'd like to have something like git-annex's git-annex-shell, which can limit access to only a specific repository. Hmm, if Obnam had its own server-side program like this, it could stream backup data to it using a protocol that avoids the roundtrips needed by the SFTP protocol, and fix the latency issue too. Lars, I know you've been looking for a Haskell starter project ... perhaps this is it? :)

Russell Coker: Targeted Advertising

Don Marti has written another blog post about targeted advertising [1]. His main point is that when a company uses the most targeted adverts (such as Google advertising) everyone knows that they are paying a small number of cents per click and nothing for the people who don t click. This compares to TV adverts which cost a lot of money and for which most viewers either leave the room or use fast-forward. Therefore using Google adverts doesn t send a signal about the amount of money invested in the products. Don also cited an example of a company sponsoring an OK Go film clip, that was a great idea, it shows that the company can do expensive things which are also a bit creative and fans will thank them (watch all the OK Go videos on Youtube, they are great). The next question is how else companies can advertise? One thing I d really like to see is sponsorship of authors. Pick an author and pay them a salary with paid editorial services for releasing a book a year for free in HTML and ebook formats. Having a fixed salary is a significant benefit when it comes time to apply for a mortgage or plan a holiday and being able to freely distribute books would be a significant benefit for an author who hasn t got a large fan base. In the computer industry it seems that there s a lot of potential for sponsoring people who produce free things. That ranges from free software and designs for free hardware to blog posts and documentation. Five years ago Sun had a blogging contest and my friend Dave Hall won a server that was worth $21K [2]. It would be nice if some other companies started doing similar things and if Sun did a repeat so some other people I like could get some free kit. Related posts:
  1. What is Appropriate Advertising? Colin Charles writes about a woman who is selling advertising...
  2. Advertising Free Software Projects Today I just noticed the following advert on one of...
  3. Friends and Adverts For some time I have been running Google Adsense adverts...

Petter Reinholdtsen: Song book for Computer Scientists

Many years ago, while studying Computer Science at the University of Troms , I started collecting computer related songs for use at parties. The original version was written in LaTeX, but a few years ago I got help from H kon W. Lie, one of the inventors of W3C CSS, to convert it to HTML while keeping the ability to create a nice book in PDF format. I have not had time to maintain the book for a while now, and guess I should put it up on some public version control repository where others can help me extend and update the book. If anyone is volunteering to help me with this, send me an email. Also let me know if there are songs missing in my book. I have not mentioned the book on my blog so far, and it occured to me today that I really should let all my readers share the joys of singing out load about programming, computers and computer networks. Especially now that Debconf 12 is about to start (and I am not going). Want to sing? Check out Petter's Computer Science Songbook.

Paul Tagliamonte: debian-desktop/7.0.0~exp2 in experimental

Mirroring my post to debian-desktop here:
Howdy, -desktop,
Hopefully we can contain the flame to the other thread.
desktop-base/7.0.0~exp2 has been accepted into experimental, for testing
by y'all (and generally interested folks) for feedback and testing
before pushing to unstable.
I urge y'all to give it a try and provide feedback on issues that you
may run into.
You can overlay experimental on any unstable install, and install
desktop-base by running:
    $ sudo apt-get -t experimental install desktop-base
For more information on using experimental, check out the wiki[1].
If you would care to test out the plymouth bootsplash, here's the short
version of how to get it running:
 1) Install plymouth via apt ($ sudo apt-get install plymouth)
 2) Set the theme to use  joy' ($ sudo /usr/sbin/plymouth-set-default-theme joy)
 3) Edit the file /etc/default/grub, and add
    "splash" to GRUB_CMDLINE_LINUX_DEFAULT
 4) update your initramfs ($ sudo update-initramfs -u)
Here's the full list of changes from the version in unstable:
    desktop-base (7.0.0~exp2) experimental; urgency=low
      * Remove GConf defaults, they are not used anymore.
      * Remove the GDM3 settings, since the configuration format for GDM
        change again, yay.
      * Add a dconf file for the new-style GDM3 configuration.
      * Add a XML file describing the available resolutions for the
        background (for use with GNOME).
      * postinst: use a higher priority for the 1920x1080 background.
      * Cleanup postinst/prerm/postrm.
      * Add a new alternative: desktop-background.xml.
      * Use it from the gsettings override.
      * Reload gdm3 after installation/removal.
      * Drop old libgnome2-common hack.
      * Clean up an alternative that is no longer available.
     -- Josselin Mouette <joss@debian.org>  Sat, 23 Jun 2012 23:28:15 +0200
    desktop-base (7.0.0~exp1) experimental; urgency=low
      [ Paul Tagliamonte ]
      * Adding myself as a maintainer
      * We've got a new theme --  joy' by Adrien Aubourg. Thanks, Adrien!
         - Theme added to backgrounds.
         - Theme added for Grub
         - Background changed for GDM3
         - Theme added for KDM & changed that to default.
         - Theme added for ksplash & changed to default.
      * Standards bump to 3.9.3
      * Pre-Dependency added for dpkg (>= 1.15.7.2~), since we use dpkg helpers
        in our preinst.
      * Add in Jonathan Carter's Plymouth theme.
      [ Jonathan Carter ]
      * Remove splashy theme since it's no longer available in archives
      [ Yves-Alexis Perez ]
      * Install emblem in the correct folder.
     -- Paul Tagliamonte <paultag@ubuntu.com>  Sat, 23 Jun 2012 13:00:47 +0200
I thank all of you dearly for putting up with this,
  Paul
[1]: http://wiki.debian.org/DebianExperimental

23 June 2012

Richard Darst: The size of DebConf12

Phil blogged about DebConf12 having many fewer attendees than past DebConfs (10 and 11). Dealing with registration for several years, I can say these numbers tell a different story.First and foremost, these numbers tell me that DebConf12 hasn't happened yet, DC10 and DC11 have. People are told to sign up register early and reconfirm, but in reality, only the people who have accommodations with us have a significant motivation to do it. Once DebConf starts, lots of day-attendees will drop by, be told to register, and greatly inflate our numbers without staying for a very long time. The number of advance registrants also depends on how much advertisement is done. In terms of people who have both dates in the registration system (Phil's updated graph), I don't place much weight on this number, either.It's my opinion that number of beds occupied is a better indication of the scale of a DebConf, and especially at this stage. This also roughly correlates to number of Debian developers staying, though for years with cheap rooms or lots of external funding, even this number can get inflated. We need some things like t-shirts and badges for every attendee (possibly registered before a certain time), but for now, the number of rooms is what matters.In fact, I wouldn't be surprised if DebConf13 had lower "reconfirmed" numbers, because it is further away form population centers, yet still was a "bigger" conference because there are more people staying on-site or coming for multiple days.For DebConf12, I see an average of 130 people/night, for DebConf11, I see an average of 240 people/night, and for DebConf10, I see an average of 200 people/night. So, DebConf12 is smaller, but in a different way than it seems, and even this is subject to change or variable interpretation. I'm not really wanting to predict DebConf sizes at this stage.So basically, one does not simply expect DebConf registration data to make sense. One should not simply dismiss any DebConf before it happens. And most importantly, one can not simply use this data to plan a conference. The registration team has been frantically running around trying to correct every possible inconsistency you can imagine.Tags: debconf

Next.

Previous.