Search Results: "ema"

13 August 2014

Riku Voipio: Booting Linaro ARMv8 OE images with Qemu

A quick update - Linaro ARMv8 OpenEmbbeded images work just fine with qemu 2.1 as well:

$ http://releases.linaro.org/14.07/openembedded/aarch64/Image
$ http://releases.linaro.org/14.07/openembedded/aarch64/vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img.gz
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt \
-kernel Image -append 'root=/dev/vda2 rw rootwait mem=1024M console=ttyAMA0,38400n8' \
-drive if=none,id=image,file=vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img \
-netdev user,id=user0 -device virtio-net-device,netdev=user0 -device virtio-blk-device,drive=image
[ 0.000000] Linux version 3.16.0-1-linaro-vexpress64 (buildslave@x86-64-07) (gcc version 4.8.3 20140401 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.04 - Linaro GCC 4.8-2014.04) ) #1ubuntu1~ci+140726114341 SMP PREEMPT Sat Jul 26 11:44:27 UTC 20
[ 0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
root@genericarmv8:~#
Quick benchmarking with age-old ByteMark nbench:
Index Qemu Foundation Host
Memory 4.294 0.712 44.534
Integer 6.270 0.686 41.983
Float 1.463 1.065 59.528
Baseline (LINUX) : AMD K6/233*
Qemu is upto 8x faster than Foundation model on Integers, but only 50% faster on Math. Meanwhile, the Host pc spends 7-40x slower emulating ARMv8 than executing native instructions.

10 August 2014

Cyril Brulebois: Why is my package blocked?

A bit of history: A while ago udeb-producing packages were getting frozen on a regular fashion, when a d-i release was about to be cut. While I wasn t looking at the time, I can easily understand the reasons behind that: d-i is built upon many components, it takes some time to make sure it s basically in shape for a release, and it s very annoying when a regression sneaks in right before the installation images get built. I took over d-i release maintenance in May 2012 and only a few uploads happened before the wheezy freeze. I was only discovering the job at the time, and I basically released whatever was in testing then. The freeze began right after that (end of June), so I started double checking things affecting d-i (in addition to or instead of the review performed by other release team members), and unblocking packages when changes seemed safe, or once they were tested. A few uploads happened after the wheezy release and there s already a Jessie Alpha 1 release. I was about to release Jessie Beta 1 after some fair bits of testing, a debian-installer upload, and the only remaining bits were: building installation images (hello Steve), and of course communication (mail announce and website update). Unfortunately a new upstream release reached testing in the meanwhile, breaking the installer in several ways. I ll give details below, of course not because I want to point finger at the maintainer, but to illustrate the ramifications that a single package s migrating to testing can induce. Since I d like to avoid such experience in the future, I ll probably reintroduce the old method and freeze all udeb-producing packages during next d-i releases. So you know why it might happen. Your next question might be: What to do when your package is getting caught in that net? . In that case, please get in touch with both debian-release@ and debian-boot@ asking for an unblock. I ll then review your particular package, and either let it migrate to testing, or delay it until after the release. Update: official announcement.

8 August 2014

Richard Hartmann: Microsoft Linux: Debian

Huh... Source (Yes, I am on Debian's trademark team and no, I have no idea what that means. Yet.) Update: Thanks to Marcin and Steven Chamberlain for this find: It seems Debian Red is an actual name used by desginers.

7 August 2014

Jordi Mallach: A pile of reasons why GNOME should be Debian jessie s default desktop environment

GNOME has, for some reason or another, always been the default desktop environment in Debian since the installer is able to install a full desktop environment by default. Release after release, Debian has been shipping different versions of GNOME, first based on the venerable 1.2/1.4 series, then moving to the time-based GNOME 2.x series, and finally to the newly designed 3.4 series for the last stable release, Debian 7 wheezy . During the final stages of wheezy s development, it was pointed out that the first install CD image would not longer hold all of the required packages to install a full GNOME desktop environment. There was lots of discussion surrounding this bug or fact, and there were two major reactions to it. The Debian GNOME team rebuilt some key packages so they would be compressed using xz instead of gzip, saving the few megabytes that were needed to squeeze everything in the first CD. In parallel, the tasksel maintainer decided switching to Xfce as default desktop was another obvious fix. This change, unannounced and two days before the freeze, was very contested and spurred the usual massive debian-devel threads. In the end, and after a few default desktop flip flops, it was agreed that GNOME would remain as the default for the already frozen wheezy release, and this issue would be revisited later on during jessie s development. And indeed, some months ago, Xfce was again reinstated as Debian s default desktop for jessie as announced:
Change default desktop to xfce.
This will be re-evaluated before jessie is frozen. The evaluation will
start around the point of DebConf (August 2014). If at that point gnome
looks like a better choice, it ll go back as the default.
Some criteria for that choice will include:
* Popcon numbers for gnome on jessie. If gnome installations continue to
  rise fast enough despite xfce being the default (compared with, say
  kde installations), then we ll know that users prefer gnome.
  Currently we have no data about how many users would choose gnome when
  it s not the default. Part of the reason for switching to xfce now
  is to get such data.
* The state of accessability support, particularly for the blind.
* How well the UI works for both new and existing users. Gnome 3
  seems to be adding back many gnome 2 features that existing users
  expect, as well as making some available via addons. If it feels
  comfortable to gnome 2 (and xfce) users, that would go a long way
  toward switching back to it as the default. Meanwhile, Gnome 3 is also
  breaking new ground in its interface; if the interface seems more
  welcoming to new users, or works better on mobile devices, etc, that
  would again point toward switching back.
* Whatever size constraints exist for CD or other images at the time.
--
Hello to all the tech journalists out there. This is pretty boring.
Why don t you write a story about monads instead?
Joey Hess in dfca406eb694e0ac00ea04b12fc912237e01c9b5.
Suffice to say that the Debian GNOME team participants have never been thrilled about how the whole issue is being handled, and we ve been wondering if we should be doing anything about it, or just move along and enjoy the smaller amount of bug reports against GNOME packages that this change would bring us, if it finally made it through to the final release. During our real life meet-ups in FOSDEM and the systemd+GNOME sprint in Antwerp, most members of the team did feel Debian would not be delivering a graphical environment with the polish we think our users deserve, and decided we at least should try to convince the rest of the Debian project and our users that Debian will be best suited by shipping GNOME 3.12 by default. Power users, of course, can and know how to get around this default and install KDE, Xfce, Cinnamon, MATE or whatever other choice they have. For the average user, though, we think we should be shipping GNOME by default, and tasksel should revert the above commit again. Some of our reasons are: In short, we think defaulting to GNOME is the best option for the Debian release, and in contrast, shipping Xfce as the default desktop could mean delivering a desktop experience that has some incomplete or rough edges, and not on par with Debian quality standards for a stable release. We believe tasksel should again revert the change and be uploaded as soon as possible, in order to get people testing images with GNOME the sooner the better, with the freeze only two months away. We would also like that in the future, changes of this nature will not be announced in a git commit log, but widely discussed in debian-project and the other usual development/decision channels, like the change of init system happened recently. We will, whichever the final decision is, continue to package GNOME with great care to ensure our users get the best possible desktop experience Debian can offer.

Niels Thykier: Recent improvements to Britney2

As mentioned by Rapha l Hertzog, I have been spending some time on improving Britney2. Just the other day I submitted a second branch for review that I expect to merge early next week. I also got another set patches coming up soon. Currently, none of them are really user visible, so unless you are hosting your own version of Britney, these patches are probably not all that interesting to you. The highlights:
  1. Reduce the need for backtracking by finding semantically equivalent packages.
  2. Avoid needing to set up a backtrack point in some cases.
    • This has the side-effect of eliminating some O(e^n) runtime cases.
  3. Optimise installability testing of packages affected by a hinted migration.
    • This has the side-effect of avoiding some O(e^n) runtime cases when the post-hint state does not trigger said behaviour.
    • There is a follow-up patch for this one coming in the third series to fix a possible bug for a corner-case (causing a valid hint to be incorrectly rejected when it removed an uninstallable package).
  4. Reduce the number of affected packages to test when migrating items by using knowledge about semantically equivalent packages.
    • In some cases, Britney can now do free migrations when all binaries being updated replace semantically equivalent packages.
  5. (Merge pending) Avoid many redundant calls to sort_actions() , which exhibits at least O(n^2) runtime in some cases.
    • For the dataset Rapha l submitted, this patch shaves off over 30 minutes runtime. In the particular case, each call to sort_actions takes 3+ minutes and it was called at least 10 times, where it was not needed.
    • That said, sort_actions have a vastly lower runtime in the runs for Debian (and presumably also Ubuntu, since no one complained from their side so far).
The results so far: After the first patch series was merged, the Kali dataset (from Rapha l) could be processed in only ~2 hours. With the second patch series merged, the dataset will drop by another 30-50 minutes (most of which are thanks to the change mentioned in highlight #5). The third patch series currently do not have any mention-worthy performance related changes. It will probably be limited to bug fixes and some refactoring. Reflections: The 3 first highlights only affects the new installability tester meaning that the Britney2 instances at Ubuntu and Tanglu should be mostly unaffected by the O(n^2) runtime. Although those cases will probably just fail with several AIEEE s. :) The 5th highlight should equally interesting to all Britney2 instances though. For me, the most interesting part is that we have never observed the O(n^2) behaviour in a daily sid -> testing run. The dataset from Rapha l was basically a stable -> testing/sid run, which is a case I do not think we have ever done before. Despite our current updates, there is still room for improvements on that particular use case. In particular, I was a bit disheartened at how poorly our auto hinter(s) performed on this dataset. Combined they only assisted with the migration of something like 28 items . For comparison, the main run migrated ~7100 items and 9220 items were unable to migrate. Furthermore, the Original auto hinter spend the better part of 15 minutes computing hints at least it results in 10 items migrating. Links to the patches:

Lars Wirzenius: On ticketing systems

I don't really like any of the ticketing systems I've ever needed to use, whether they've been used as bug tracking systems, user support issue management systems, or something else. Some are not too bad. I currently rely most on debbugs and ikiwiki. debbugs is the Debian bug tracking system. See https://www.debian.org/Bugs/ for an entry point. It's mostly mail based, with a read-only web interface. You report a bug by sending an email to submission address, and (preferably) include a few magic "pseudo-headers" at the top of your message body ot identify the package and version. There's tools to make this easier, but mostly it's just about sending an e-mail. All replies are via e-mails as well. Effectively, each bug becomes is own little dedicated mailing list. This is important. A ticket, whether it is a bug report or a support request, is all about the discussion. "Hey I have this problem..." followed by "Have you tried..." and so forth. Anything that makes that discussion easier and faster to have is better. It is my very strong opinion, and long experience, that the best way to have such a discussion is over e-mail. A lot of modern ticketing systems are web based. They might have an e-mail mode, perhaps read-only, but that's mostly an afterthought. It's a thing bolted onto the side of the system because people like me whinge otherwise. I like e-mail for this for several reasons. For these reasons, I strongly prefer ticketing systems in which e-mails are the primary form of discussions, and e-mail is a first class citizen. I don't mind if there's other ways to participate in the discussion, but if I have to use something else than e-mail, I tend not to be happy. I use ikiwiki to provide a distributed, shared notebook on bugs. It's a bit cumbersome, and doesn't work well for discussions. I think we can improve on the way debbugs works, however. I've been thinking about ticketing systems for Obnam (my backup program), since it gaining enough users that it's getting hard to keep track of discussions with just an e-mail client. Here's what I want: I will eventually have this. I'm not saying I'm working on this, since I don't have enough free time to do that, but there's a git repository, and some code, and it imports e-mails automatically now. Some day there may even be a web interface. (This has been a teaser.)

Craig Small: WordPress 3.9.2 for Debian

WordPress released today a security release 3.9.2 which they fix several security issues, including a denial of service issue around XML. The corresponding Debian package 3.9.2+dfsg-1 is currently being uploaded to the Debian ftp-master server as I write this and should be available on the mirrors soon. Unfortunately at the time of writing, there are no CVE identifiers to match these problems up with, but refer to the wordpress page for details about these bugs. Andrew Nacin from WordPress has kindly outlined what versions are susceptible and it looks like the Debian squeeze (3.6.1+dfsg-1~deb6u4) and wheezy (3.6.1+dfsg-1~deb7u3) are vulnerable to at least some of these bugs which means for me its patch reading and back-porting time

5 August 2014

Simon Josefsson: Replicant 4.2 0002 and NFC on I9300

I m using Replicant on my Samsung SIII (i9300) phone (see my earlier posts). During my vacation the Replicant project released version 4.2-0002 as a minor update to their initial 4.2 release. I didn t anticipate any significant differences, so I followed the installation instructions but instead of wipe data/factory reset I chose wipe cache partition and rebooted. Everything appeared to work fine, but I soon discovered that NFC was not working. Using adb logcat I could get some error messages:
E/NFC-HCI ( 7022): HCI Timeout - Exception raised - Force restart of NFC service
F/libc    ( 7022): Fatal signal 11 (SIGSEGV) at 0xdeadbaad (code=1), thread 7046 (message)
I/DEBUG   ( 1900): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1900): Build fingerprint: 'samsung/m0xx/m0:4.1.1/JRO03C/I9300XXDLIB:user/release-keys'
I/DEBUG   ( 1900): Revision: '12'
I/DEBUG   ( 1900): pid: 7022, tid: 7046, name: message  >>> com.android.nfc <<<
The phone would loop trying to start NFC and having the NFC sub-system die over and over. Talking on #replicant channel, paulk quickly realized and fixed the bug. I had to rebuild the images to get things to work, so I took the time to create a new virtual machine based on Debian 7.5 for building Replicant on. As a side note, the only thing not covered by Replicant build dependency documentation was that I needed the Debian xmllint package to avoid a build failure and the Debian xsltproc package to avoid a error message being printed in the beginning of every build. Soon I had my own fresh images and installed them and NFC was working again, after installing the non-free libpn544_fw.so file. During this, I noticed that there are multiple libpn544_fw.so files floating around. I have the following files:
version string source
libpn544_fw_C3_1_26_SP.so internet
libpn544_fw_C3_1_34_SP.so stock ROM on S3 bought in Sweden during 2013 and 2014 (two phones)
libpn544_fw_C3_1_39_SP.so internet
(For reference the md5sum's of these files are 682e50666effa919d557688c276edc48, b9364ba59de1947d4588f588229bae20 and 18b4e634d357849edbe139b04c939593 respectively.) If you do not have any of these files available as /vendor/firmware/libpn544_fw.so you will get the following error message:
I/NfcService( 2488): Enabling NFC
D/NFCJNI  ( 2488): Start Initialization
E/NFC-HCI ( 2488): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
E/NFCJNI  ( 2488): phLibNfc_Mgt_Initialize() returned 0x00ff[NFCSTATUS_FAILED]
E/NFC-HCI ( 2488): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
W/NFCJNI  ( 2488): Firmware update FAILED
E/NFC-HCI ( 2488): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
W/NFCJNI  ( 2488): Firmware update FAILED
E/NFC-HCI ( 2488): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
W/NFCJNI  ( 2488): Firmware update FAILED
E/NFCJNI  ( 2488): Unable to update firmware, giving up
D/NFCJNI  ( 2488): phLibNfc_Mgt_UnConfigureDriver() returned 0x0000[NFCSTATUS_SUCCESS]
D/NFCJNI  ( 2488): Terminating client thread...
W/NfcService( 2488): Error enabling NFC
Using the first (26) file or the last (39) file does not appear to be working on my phone, I get the following error messages. Note that the line starting with 'NFC capabilities' has 'Rev = 34' in it, possibly indicating that I need the version 34 file.
I/NfcService( 5735): Enabling NFC
D/NFCJNI  ( 5735): Start Initialization
D/NFCJNI  ( 5735): NFC capabilities: HAL = 8150100, FW = b10122, HW = 620003, Model = 12, HCI = 1, Full_FW = 1, Rev = 34, FW Update Info = 8
D/NFCJNI  ( 5735): Download new Firmware
W/NFCJNI  ( 5735): Firmware update FAILED
D/NFCJNI  ( 5735): Download new Firmware
W/NFCJNI  ( 5735): Firmware update FAILED
D/NFCJNI  ( 5735): Download new Firmware
W/NFCJNI  ( 5735): Firmware update FAILED
E/NFCJNI  ( 5735): Unable to update firmware, giving up
D/NFCJNI  ( 5735): phLibNfc_Mgt_UnConfigureDriver() returned 0x0000[NFCSTATUS_SUCCESS]
D/NFCJNI  ( 5735): Terminating client thread...
W/NfcService( 5735): Error enabling NFC
Loading the 34 works fine.
I/NfcService( 2501): Enabling NFC
D/NFCJNI  ( 2501): Start Initialization
D/NFCJNI  ( 2501): NFC capabilities: HAL = 8150100, FW = b10122, HW = 620003, Model = 12, HCI = 1, Full_FW = 1, Rev = 34, FW Update Info = 0
D/NFCJNI  ( 2501): phLibNfc_SE_GetSecureElementList()
D/NFCJNI  ( 2501): 
D/NFCJNI  ( 2501): > Number of Secure Element(s) : 1
D/NFCJNI  ( 2501): phLibNfc_SE_GetSecureElementList(): SMX detected, handle=0xabcdef
D/NFCJNI  ( 2501): phLibNfc_SE_SetMode() returned 0x000d[NFCSTATUS_PENDING]
I/NFCJNI  ( 2501): NFC Initialized
D/NdefPushServer( 2501): start, thread = null
D/NdefPushServer( 2501): starting new server thread
D/NdefPushServer( 2501): about create LLCP service socket
D/NdefPushServer( 2501): created LLCP service socket
D/NdefPushServer( 2501): about to accept
D/NfcService( 2501): NFC-EE OFF
D/NfcService( 2501): NFC-C ON
What is interesting is, that my other S3 running CyanogenMod does not have the libpn544_fw.so file but still NFC works. The messages are:
I/NfcService( 2619): Enabling NFC
D/NFCJNI  ( 2619): Start Initialization
E/NFC-HCI ( 2619): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
W/NFC     ( 2619): Firmware image not available: this device might be running old NFC firmware!
D/NFCJNI  ( 2619): NFC capabilities: HAL = 8150100, FW = b10122, HW = 620003, Model = 12, HCI = 1, Full_FW = 1, Rev = 34, FW Update Info = 0
D/NFCJNI  ( 2619): phLibNfc_SE_GetSecureElementList()
D/NFCJNI  ( 2619): 
D/NFCJNI  ( 2619): > Number of Secure Element(s) : 1
D/NFCJNI  ( 2619): phLibNfc_SE_GetSecureElementList(): SMX detected, handle=0xabcdef
D/NFCJNI  ( 2619): phLibNfc_SE_SetMode() returned 0x000d[NFCSTATUS_PENDING]
I/NFCJNI  ( 2619): NFC Initialized
D/NdefPushServer( 2619): start, thread = null
D/NdefPushServer( 2619): starting new server thread
D/NdefPushServer( 2619): about create LLCP service socket
D/NdefPushServer( 2619): created LLCP service socket
D/NdefPushServer( 2619): about to accept
D/NfcService( 2619): NFC-EE OFF
D/NfcService( 2619): NFC-C ON
Diffing the two NFC-relevant repositories between Replicant (external_libnfc-nxp and packages_apps_nfc) and CyanogenMod (android_external_libnfc-nxp and android_packages_apps_Nfc) I found a commit in Replicant that changes a soft-fail on missing firmware to a hard-fail. I manually reverted that patch in my build tree, and rebuilt and booted a new image. Enabling NFC now prints this on my Replicant phone:
I/NfcService( 2508): Enabling NFC
D/NFCJNI  ( 2508): Start Initialization
E/NFC-HCI ( 2508): Could not open /system/vendor/firmware/libpn544_fw.so or /system/lib/libpn544_fw.so
W/NFC     ( 2508): Firmware image not available: this device might be running old NFC firmware!
D/NFCJNI  ( 2508): NFC capabilities: HAL = 8150100, FW = b10122, HW = 620003, Model = 12, HCI = 1, Full_FW = 1, Rev = 34, FW Update Info = 0
D/NFCJNI  ( 2508): phLibNfc_SE_GetSecureElementList()
D/NFCJNI  ( 2508): 
D/NFCJNI  ( 2508): > Number of Secure Element(s) : 1
D/NFCJNI  ( 2508): phLibNfc_SE_GetSecureElementList(): SMX detected, handle=0xabcdef
D/NFCJNI  ( 2508): phLibNfc_SE_SetMode() returned 0x000d[NFCSTATUS_PENDING]
I/NFCJNI  ( 2508): NFC Initialized
D/NdefPushServer( 2508): start, thread = null
D/NdefPushServer( 2508): starting new server thread
D/NdefPushServer( 2508): about create LLCP service socket
D/NdefPushServer( 2508): created LLCP service socket
D/NdefPushServer( 2508): about to accept
D/NfcService( 2508): NFC-EE OFF
D/NfcService( 2508): NFC-C ON
And NFC works! At least YubiKey NEO with the Yubico Authenticator app. One less non-free blob on my phone. I have double-checked that power-cycling the phone (even removing battery for a while) does not affect anything, so it seems the NFC chip has firmware loaded from the factory. Question remains why that commit was added. Is it necessary on some other phone? I have no idea, other than if the patch is reverted, S3 owners will have NFC working with Replicant without non-free software added. Alternatively, make the patch apply only on the platform where it was needed, or even to all non-S3 builds. flattr this!

Steve Kemp: Free (orange) SMS alerts

In the past I used to pay for an email->SMS gateway, which was used to alert me about some urgent things. That was nice because it was bi-directional, and at one point I could restart particular services via sending SMS messages. These days I get it for free, and for my own reference here is how you get to receive free SMS alerts via Orange, which is my mobile phone company. If you don't use Orange/EE this will probably not help you. The first step is to register an Orange email-account, which can be done here: Once you've done that you'll have an email address of the form example@orange.net, which is kinda-sorta linked to your mobile number. You'll sign in and be shown something that looks like webmail from the early 90s. The thing that makes this interesting is that you can look in the left-hand menu and see a link called "SMS Alerts". Visit it. That will let you do things like set the number of SMSs you wish to receive a month (I chose "1000"), and the hours during which delivery will be made (I chose "All the time"). Anyway if you go through this dance you'll end up with an email address example@orange.net, and when an email arrives at that destination an SMS will be sent to your phone. The content of the SMS will be the subject of the mail, truncated if necessary, so you can send a hello message to yourself like this:
echo "nop"   mail -s "Hello, urgent message is present" username@orange.net
Delivery seems pretty reliable, and I've scheduled the mailbox to be purged every week, to avoid it getting full:
Hostnamepop.orange.net
UsernameYour mobile number
PasswordYour password
If you wished to send mail from this you can use smtp.orange.net, but I pity the fool who used their mobile phone company for their primary email address.

4 August 2014

Russ Allbery: Review: Parasite

Review: Parasite, by Mira Grant
Series: Parasitology #1
Publisher: Orbit
Copyright: October 2013
ISBN: 0-316-21893-6
Format: Kindle
Pages: 504
It's 2027, and a company named Symbogen has revolutionized medicine and health. They manufacture a genetically-engineered parasite based on a tapeworm that can stabilize and protect the health of just about anyone. It can synthesize medication, fix chronic medical problems, and be adapted to different conditions. (Yes, I know biological systems don't work this way. That's not going to be the only suspension of disbelief problem.) This has made Symbogen one of the most powerful corporations in the country, aided by the skill at marketing and self-promotion shown by one of the founders. Sal, the protagonist of the novel, is one of Symbogen's most famous success stories. Sally was the victim of a horrible car crash that put her into an apparently irreversible coma. But as her family was debating whether to turn off life support, she woke up. She had no memory of her previous life at all, and had to relearn fine motor control, reading, and many other skills. She was essentially a new person. But she was alive; her symbiont had saved her. When the story proper starts, Sal is still a ward of her parents. She has generally adult skills despite still struggling with reading, but she still has occasional attacks and is under intensive monitoring by Symbogen. That means periodic mandatory appointments with Symbogen, which she hates, but she's otherwise started building a life for herself: a job in an animal shelter, an interest in exotic predatory plants, and, most notably, a boyfriend. There are things about her life she doesn't like, and she wants to be free of Symbogen, but she doesn't have a bad life. But then a mysterious illness begins sweeping through the population, causing people to go blank, apparently lose their minds, and then start attacking those near them. Some of you have doubtless already figured out the key plot revelation. It's not hard; even if you didn't from the summary, you will probably figure it out shortly into the book. And therein lies a large problem with this novel: it's hopelessly predictable. Creepy evil corporation that supposedly has your best interests at heart, check. Plucky mad scientist opposition who understands exactly what's going on, check. Well-meaning but heavy-handed government agents who try to get involved but mostly make everything worse, check. (Although it's unusual to have those agents as part of the protagonist's family, and I thought that added some additional depth.) Mostly clueless protagonist sucked into the plot and becoming critical to its resolution, check. Very few readers are going to be surprised by this story. This is not, by itself, a fatal flaw. Predictable story structures can carry satisfying variations, or introduce the reader to enjoyable characters. And I think Grant manages both here. Seanan McGuire, both as herself and under her Mira Grant pseudonym, tends to write damaged and struggling characters. Both her Newsflesh and October Daye series feature protagonists that have been hurt badly, but are coping and muddling through in their own ways. In Parasite, I think she takes a more daring and intriguing approach: a protagonist that other people in the story perceive as damaged and struggling, but who actually isn't. Sal is not a badly injured Sally, and she's quite a bit healthier than those around her think she is. Her thought processes don't work quite the same as those around her, but that's not because she's hurt. That's because she's a different person. This makes Parasite partly a novel about identity, about Sal claiming ownership of her own life. Grant drags this out longer than I wish she had, but I liked the idea. In Sal, she strikes a good balance between gratitude and genuine affection for her family and the need to become her own person unconstrained by other people's expectations. As with the Newsflesh series, Grant uses quotes and excerpts from interviews to fill in the world background: a few at the start of each chapter, and more around each part boundary. I like this technique, and Grant uses it well. By the end of the book, the Rolling Stone interview with the head of Symbogen has added a lot of insight into how Symbogen manages its public relations. Grant also throws in a few of her trademark dangerously off-beat characters: hyper-competent, wise-cracking, but eerily skewed. I loved those in Blackout and I loved Tansy and Dr. Cale here. (Adam was much less successful.) A whole book from Tansy's perspective wouldn't work, since she needs Sal as a straight woman, but I thought she stole every scene she was in. However, I agree wholeheartedly with Tansy on another point: Sal is remarkably, irritatingly dim about what is apparently intended to be the critical revelation of the book. I won't state it outright; given its significant presence in the final scene, apparently it is intended to be a spoiler. But I figured it out about 50 pages into the book. Grant telegraphs this revelation heavily, and Tansy considers it painfully obvious (with quite a bit of justification). But Sal doesn't figure it out for the entire book, ignores all the signs, and is apparently willfully blind. In a book written from the first-person perspective by an otherwise-reliable narrator, this is highly annoying. It significantly undermined my enjoyment of the book. I spent much of the novel ahead of the narrator in my understanding of the plot and waiting, in vain, for her to get on with it already. That unfortunately makes Parasite a mixed bag. I really liked many of the characters, and I think Grant did some interesting things with family dynamics and with claiming one's own identity. But this is undermined by a very predictable plot, the protagonist deciding to be dumber than a sack of hammers about a critical plot point, and some rather dubious world logic. (For example, why is Sal terrified of bad driving? It makes sense as a post-traumatic stress reaction... except it's a critical point to her characterization that she never went through that stress.) Sometimes I wanted to like this book and sometimes I wanted to shake it, and sometimes I felt both reactions at the same time. I like Grant's writing and characterization well enough that I will probably read the sequel, but this is more like the later books in the Newsflesh series than it is like the spectacular Feed. Worth reading, at least for me, but it could have been better. Rating: 7 out of 10

3 August 2014

Bits from Debian: DebConf14 - schedule available

Debconf14 will be held in three weeks in Portland, OR, USA and we're happy to announce that the schedule is already available. Of course, it is still possible for some minor changes to happen! DebConf will open on Saturday, August 23 with the Welcome talk followed by two highlighted talks: There will also be also a plethora of social events, such as our traditional cheese and wine party, our group photo and our day trip. The complete schedule can be found at: https://summit.debconf.org/debconf14/ DebConf talks will be broadcast live on the Internet when possible, and videos of the talks will be published on the web along with the presentation slides.

2 August 2014

Don Armstrong: ErgoDox keyboard assembly

I routinely use a Kinesis Advantage Pro keyboard, which is a split, ergonomic keyboard with thumb clusters that uses brown cherryMX switches. Over the thirteen years that I've been using it, I've become a huge fan of this style of keyboard. However, I have two major annoyances with the Kinesis. First, while the firmware is good, remapping the keys is complicated and producing more complicated keyboard layouts with layers and keycodes that are not present in the original layout is not possible. Secondly, the interconnect between the main key wells and the controller board in the middle occasionally fails, and requires disassembly and occasional re-tinning of the circuit board interconnect connector. 1 About a year ago, I became aware of the ErgoDox keyboard, which is a keyboard design which mimics the kinesis to some degree, but with completely separated key halves (useful, because I'm substantially bigger than the average human), programmable firmware (so I can finally have the layers and missing keys) and with slightly more elegant interconnects (TRRS cables). Unfortunately, at the time I first heard about it (and other custom keyboards), making it required sourcing circuit boards, parts, and finding someone to cut a case for the keyboard. Then, a few months ago, I learned about MassDrop, a company who puts together groups of people to do buys of products at near-wholesale level prices, and their offer of all of the parts to build an ErgoDox. After waiting for a group buy of the keyboard to become available, I put in an order, and received the parts two months later. Over a few hours yesterday, I learned how to do surface mount soldering of the 78 diodes (one for each key), and finished assembling and flashing the firmware. This morning, I fixed up the few key bindings that I needed to be productive, and viola, my laptop at home now has a brand new ergonomic keyboard.

1 August 2014

Russell Coker: More BTRFS Fun

I wrote a BTRFS status report yesterday commenting on the uneventful use of BTRFS recently [1]. Early this morning the server that stores my email (which had 93 days uptime) had a filesystem related problem. The root filesystem became read-only and then the kernel message log filled with unrelated messages so there was no record of the problem. I m now considering setting up rsyslogd to log the kernel messages to a tmpfs filesystem to cover such problems in future. As RAM is so cheap it wouldn t matter if a few megs of RAM were wasted by that in normal operation if it allowed me to extract useful data when something goes really wrong. It s really annoying to have a system in a state where I can login as root but not find out what went wrong. After that I tried 2 kernels in the 3.14 series, both of which had kernel BUG assertions related to Xen networking and failed to network correctly, I filed Debian Bug #756714. Fortunately they at least had enough uptime for me to run a filesystem scrub which reported no errors. Then I reverted to kernel 3.13.10 but the reboot to apply that kernel change failed. Systemd was unable to umount the root filesystem (maybe because of a problem with Xen) and then hung the system instead of rebooting, I filed Debian Bug #756725. I believe that if asked to reboot a system there is no benefit in hanging the system with no user space processes accessible. Here are some useful things that systemd could have done:
  1. Just reboot without umounting (like reboot -nf does).
  2. Pause for some reasonable amount of time to give the sysadmin a possibility of seeing the error and then rebooting.
  3. Go back to a regular runlevel, starting daemons like sshd.
  4. Offer a login prompt to allow the sysadmin to login as root and diagnose the problem.
Options 1, 2, and 3 would have saved me a bit of driving. Option 4 would have allowed me to at least diagnose the problem (which might be worth the drive). Having a system on the other side of the city which has no remote console access just hang after a reboot command is not useful, it would be near the top of the list of things I don t want to happen in that situation. The best thing I can say about systemd s operation in this regard is that it didn t make the server catch fire. Now all I really know is that 3.14 kernels won t work for my server, 3.13 will cause problems that no-one can diagnose due to lack of data, and I m now going to wait for it to fail again. As an aside the server has ECC RAM and it s hardware is known to be good, so I m sure that BTRFS is at fault.

31 July 2014

Craig Small: Linux Capabilities

I was recently updating some code that uses fping. Initially it used exec() that was redirected to a temporary file but I changed it to use popen. While it had been a while since I ve done this sort of thing, I do recall there was an issue with running popen on setuid binary. A later found it is mainly around setuid scripts which are very problematic and there are good reasons why you don t do this. Anyhow, the program worked fine which surprised me. Was fping setuid root to get the raw socket?
$ ls -l /usr/bin/fping
-rwxr-xr-x 1 root root 31464 May  6 21:42 /usr/bin/fping
It wasn t which at first all I thought ok, so that s why popen is happy . The way that fping and other programs work is they bind to a raw socket. This socket sits below the normal type sockets such as the ones used for TCP and UDP and normal users cannot use them by default. So how did fping work it s magic and get access to this socket? It used Capabilities. Previously getting privileged features had a big problem; it was an all or nothing thing. You want access to a raw socket? Sure, be setuid but that means you also could, for example, read any file on the system or set passwords. Capabilites provide a way of giving programs some better level of access, but not a blank cheque. The tool getcap is the way of determining what capabilities are found on a file. These capabilities are attributes on the file which, when the file is run, turn into capabilities or extra permissions. fping has the capability cap_net_raw+ep applied to it. This gives access to the RAW and PACKET sockets which is what fping needs. The +ep after the capability name means it is an Effective and Permitted capability, which describes what happens with child processes and dropping privileges. I hadn t seen these Capabilities before. They are a nice way to give your programs the access they need, but limiting the risk of something going wrong and having a rouge program running as root.

Russell Coker: BTRFS Status July 2014

My last BTRFS status report was in April [1], it wasn t the most positive report with data corruption and system hangs. Hacker News has a brief discussion of BTRFS which includes the statement Russell Coker s reports of his experiences with BTRFS give me the screaming heebie-jeebies, no matter how up-beat and positive he stays about it [2] (that s one of my favorite comments about my blog). Since April things have worked better. Linux kernel 3.14 solves the worst problems I had with 3.13 and it s generally doing everything I want it to do. I now have cron jobs making snapshots as often as I wish (as frequently as every 15 minutes on some systems), automatically removing snapshots (removing 500+ snapshots at once doesn t hang the system), balancing, and scrubbing. The fact that I can now expect that a filesystem balance (which is a type of defragment operation for BTRFS that frees some chunks ) from a cron job and expect the system not to hang means that I haven t run out of metadata chunk space. I expect that running out of metadata space can still cause filesystem deadlocks given a lack of reports on the BTRFS mailing list of fixes in that regard, but as long as balance works well we can work around that. My main workstation now has 35 days of uptime and my home server has 90 days of uptime. Also the server that stores my email now has 93 days uptime even though it s running Linux kernel 3.13.10. I am rather nervous about the server running 3.13.10 because in my experience every kernel before 3.14.1 had BTRFS problems that would cause system hangs. I don t want a server that s an hour s drive away to hang The server that runs my email is using kernel 3.13.10 because when I briefly tried a 3.14 kernel it didn t work reliably with the Xen kernel 4.1 from Debian/Wheezy and I had a choice of using the Xen kernel 4.3 from Debian/Unstable to match the Linux kernel or use an earlier Linux kernel. I have a couple of Xen servers running Debian/Unstable for test purposes which are working well so I may upgrade my mail server to the latest Xen and Linux kernels from Unstable in the near future. But for the moment I m just not doing many snapshots and never running a filesystem scrub on that server. Scrubbing In kernel 3.14 scrub is working reliably for me and I have cron jobs to scrub filesystems on every system running that kernel. So far I ve never seen it report an error on a system that matters to me but I expect that it will happen eventually. The paper An Analysis of Data Corruption in the Storage Stack from the University of Wisconsin (based on NetApp data) [3] shows that nearline disks (IE any disks I can afford) have an incidence of checksum errors (occasions when the disk returns bad data but claims it to be good) of about 0.42%. There are 18 disks running in systems I personally care about (as opposed to systems where I am paid to care) so with a 0.42% probability of a disk experiencing data corruption per year that would give a 7.3% probability of having such corruption on one disk in any year and a greater than 50% chance that it s already happened over the last 10 years. Of the 18 disks in question 15 are currently running BTRFS. Of the 15 running BTRFS 10 are scrubbed regularly (the other 5 are systems that don t run 24*7 and the system running kernel 3.13.10). Newer Kernels The discussion on the BTRFS mailing list about kernel 3.15 is mostly about hangs. This is correlated with some changes to improve performance so I presume that it has exposed race conditions. Based on those discussions I haven t felt inclined to run a 3.15 kernel. As the developers already have some good bug reports I don t think that I could provide any benefit by doing more testing at this time. I think that there would be no benefit to me personally or the Linux community in testing 3.15. I don t have a personal interest in RAID-5 or RAID-6. The only systems I run that have more data than will fit on a RAID-1 array of cheap SATA disks are ones that I am paid to run and they are running ZFS. So the ongoing development of RAID-5 and RAID-6 code isn t an incentive for me to run newer kernels. Eventually I ll test out RAID-6 code, but at the moment I don t think they need more bug reports in this area. I don t have a great personal interest in filesystem performance at this time. There are some serious BTRFS performance issues. One problem is that a filesystem balance and subtree removal seem to take excessive amounts of CPU time. Another is that there isn t much support for balancing IO to multiple devices (in RAID-1 every process has all it s read requests sent to one device). For large-scale use of a filesystem these are significant problems. But when you have basic requirements (such as a mail server for dozens of users or a personal workstation with a quad-core CPU and fast SSD storage) it doesn t make much difference. Currently all of my systems which use BTRFS have storage hardware that exceeds the system performance requirements by such a large margin that nothing other than installing Debian packages can slow the system down. So while there are performance improvements in newer versions of the BTRFS kernel code that isn t an incentive for me to upgrade. It s just been announced that Debian/Jessie will use Linux 3.16, so I guess I ll have to test that a bit for the benefit of Debian users. I am concerned that 3.16 won t be stable enough for typical users at the time that Jessie is released.

29 July 2014

Gunnar Wolf: Editorial process starting in 3... 2... 1...

Yay! Today I finally submitted our book, Fundamentos de Sistemas Operativos, for the Editorial Department of our institute. Of course, I'm not na ve enough to assume there won't be a heavy editorial phase, but I'm more than eager to dive into it... And have the book printed in maybe two months time! Of course, this book is to be published under a free license (CC-BY-SA). And I'm talking with the coauthors, we are about to push the Git repository to a public location, as we believe the source for the text and figures can also be of interest to others. The book itself (as I've already boasted about here :- ) is available (somewhat as a preprint) for download. [update] Talked it over with the coauthors, and we finally have a public repository! Clone it from: https://github.com/gwolf/sistop.git Or just browse it from Github's web interface.

Christian Perrier: Developers per country (July 2014)

This is time again for my annual report about the number of developers per country. This is now the sixth edition of this report. Former editions: So, here we are with the July 2014 version, sorted by the ratio of *active* developers per million population for each country.
Act: number of active developers
Dev: total number of developers
A/M: number of active devels per million pop.
D/M: number of devels per million pop.
2009: rank in 2009
2010: rank in 2010
2011: rank in 2011 (June)
2012: rank in 2012 (June)
2013: rank in 2012 (July)
2014: rank now
Code Name Population Act Dev Dev Act/Million Dev/Million 2009 2010 June 2011 June 2012 July 2013 July 2014
fi Finland 5259250 19 31 3,61 5,89 1 1 1 1 1 1
ie Ireland 4670976 13 17 2,78 3,64 13 9 6 2 2 2
nz New Zealand 4331600 11 15 2,54 3,46 4 3 5 7 7 3 *
mq Martinique 396404 1 1 2,52 2,52

3 4 4 4
se Sweden 9088728 22 37 2,42 4,07 3 6 7 5 5 5
ch Switzerland 7870134 19 29 2,41 3,68 2 2 2 3 3 6 *
no Norway 4973029 11 14 2,21 2,82 5 4 4 6 6 7 *
at Austria 8217280 18 29 2,19 3,53 6 8 10 10 10 8 *
de Germany 81471834 164 235 2,01 2,88 7 7 9 9 8 9 *
lu Luxemburg 503302 1 1 1,99 1,99 8 5 8 8 9 10 *
fr France 65350000 101 131 1,55 2 12 12 11 11 11 11
au Australia 22607571 32 60 1,42 2,65 9 10 12 12 12 12
be Belgium 11071483 14 17 1,26 1,54 10 11 13 13 13 13
uk United-Kingdom 62698362 77 118 1,23 1,88 14 14 14 14 14 14
nl Netherlands 16728091 18 40 1,08 2,39 11 13 15 15 15 15
ca Canada 33476688 34 63 1,02 1,88 15 15 17 16 16 16
dk Denmark 5529888 5 10 0,9 1,81 17 17 16 17 17 17
es Spain 46754784 34 56 0,73 1,2 16 16 19 18 18 18
it Italy 59464644 36 52 0,61 0,87 23 22 22 19 19 19
hu Hungary 10076062 6 12 0,6 1,19 18 25 26 20 24 20 *
cz Czech Rep 10190213 6 6 0,59 0,59 21 20 21 21 20 21 *
us USA 313232044 175 382 0,56 1,22 19 21 25 24 22 22
il Israel 7740900 4 6 0,52 0,78 24 24 24 25 23 23
hr Croatia 4290612 2 2 0,47 0,47 20 18 18 26 25 24 *
lv Latvia 2204708 1 1 0,45 0,45 26 26 27 27 26 25 *
bg Bulgaria 7364570 3 3 0,41 0,41 25 23 23 23 27 26 *
sg Singapore 5183700 2 2 0,39 0,39


33 33 27 *
uy Uruguay 3477778 1 2 0,29 0,58 22 27 28 28 28 28
pl Poland 38441588 11 15 0,29 0,39 29 29 30 30 30 29 *
jp Japan 127078679 36 52 0,28 0,41 30 28 29 29 29 30 *
lt Lithuania 3535547 1 1 0,28 0,28 28 19 20 22 21 31 *
gr Greece 10787690 3 4 0,28 0,37 33 38 34 35 35 32 *
cr Costa Rica 4301712 1 1 0,23 0,23 31 30 31 31 31 33 *
by Belarus 9577552 2 2 0,21 0,21 35 36 39 39 32 34 *
ar Argentina 40677348 8 10 0,2 0,25 34 33 35 32 37 35 *
pt Portugal 10561614 2 4 0,19 0,38 27 32 32 34 34 36 *
sk Slovakia 5477038 1 1 0,18 0,18 32 31 33 36 36 37 *
rs Serbia 7186862 1 1 0,14 0,14



38 38
tw Taiwan 23040040 3 3 0,13 0,13 37 34 37 37 39 39
br Brazil 192376496 18 21 0,09 0,11 36 35 38 38 40 40
cu Cuba 11241161 1 1 0,09 0,09
38 41 41 41 41
co Colombia 45566856 4 5 0,09 0,11 41 44 46 47 46 42 *
kr South Korea 48754657 4 6 0,08 0,12 39 39 42 42 42 43 *
gt Guatemala 13824463 1 1 0,07 0,07



43 44 *
ec Ecuador 15007343 1 1 0,07 0,07
40 43 43 45 45
cl Chile 16746491 1 2 0,06 0,12 42 41 44 44 47 46 *
za South Africa 50590000 3 10 0,06 0,2 38 48 48 48 48 47 *
ru Russia 143030106 8 9 0,06 0,06 43 42 47 45 49 48 *
mg Madagascar 21281844 1 1 0,05 0,05 44 37 40 40 50 49 *
ro Romania 21904551 1 2 0,05 0,09 45 43 45 46 51 50 *
ve Venezuela 28047938 1 1 0,04 0,04 40 45 50 49 44 51 *
my Malaysia 28250000 1 1 0,04 0,04

49 50 52 52
pe Peru 29907003 1 1 0,03 0,03 46 46 51 51 53 53
tr Turkey 74724269 2 2 0,03 0,03 47 47 52 52 54 54
ua Ukraine 45134707 1 1 0,02 0,02 48 53 58 59 55 55
th Thailand 66720153 1 2 0,01 0,03 50 50 54 54 56 56
eg Egypt 80081093 1 3 0,01 0,04 51 51 55 55 57 57
mx Mexico 112336538 1 1 0,01 0,01 49 49 53 53 58 58
cn China 1344413526 10 14 0,01 0,01 53 53 57 56 59 59
in India 1210193422 8 9 0,01 0,01 52 52 56 57 60 60
sv El Salvador 7066403 0 1 0 0,14

36 58 61 61































969 1561 62,08%







A few interesting facts:

28 July 2014

Daniel Pocock: Secure that Dictaphone

2014 has been a big year for dictaphones so far. First, it was France and the secret recordings made by Patrick Buisson during the reign of President Sarkozy. Then, a US court ordered the release of the confidential Boston College tapes, part of an oral history project. Originally, each participant had agreed their recording would only be released after their death. Sinn Fein leader Gerry Adams was arrested and questioned over a period of 100 hours and released without charge. Now Australia is taking its turn. In #dictagate down under, a senior political correspondent from a respected newspaper recorded (most likely with consent) some off-the-record comments of former conservative leader Ted Baillieu. Unfortunately, this journalist misplaced the dictaphone at the state conference of Baillieu's arch-rivals, the ALP. A scandal quickly errupted. Secure recording technology There is no question that electronic voice recordings can be helpful for people, including journalists, researchers, call centers and many other purposes. However, the ease with which they can now be distributed is only dawning on people. Twenty years ago, you would need to get the assistance of a radio or TV producer to disseminate such recordings so widely. Today there is email and social media. The Baillieu tapes were emailed directly to 400 people in a matter of minutes. Just as technology brings new problems, it also brings solutions. Encryption is one of them. Is encryption worthwhile? Coverage of the Snowden revelations has revealed that many popular security technologies are not one hundred percent safe. In each of these dictaphone cases, however, NSA-level expertise was not a factor. Even the most simplistic encryption would have caused endless frustration to the offenders who distributed the Baillieu tape. How can anybody be sure encryption is reliable? Part of the problem is education. Everybody using the technology needs to be aware of the basic concepts, for example, public key cryptography. Another big question mark is back doors. There is ongoing criticism of Apple iPhone/iPod devices and the many ways that their encryption can be easily disabled by Apple engineers and presumably many former staff, security personnel and others. The message is clear: proprietary, closed-source solutions should be avoided. Free and open source technologies are the alternative. If a company does not give you the source code, how can anybody independently audit their code for security? With encryption software, what use is it if nobody has verified it? What are the options? However, given that the majority of people don't have a PhD in computer science or mathematics, are there convenient ways to get started with encryption? Reading is a good start. The Code Book by Simon Singh (author of other popular science books like Fermat's Last Theorem) is very accessible, not classified and assumes no formal training in mathematics. Even for people who do know these topics inside out, it is a good book to share with friends and family. The Guardian Project (no connection with Guardian Media of Edward Snowden fame) aims to provide a secure and easy to use selection of apps for pocket devices. This project has practical applications in business, journalism and politics alike. How should a secure dictaphone app work? Dictaphone users typically need to take their dictaphones in the field, so there is a risk of losing it or having it stolen. A strong security solution in this situation may involve creating an RSA key pair on a home/office computer, keeping the private key on the home computer and putting the public key on the dictaphone device. Configured this way, the dictaphone will not be able to play back any of the recordings itself - the user will always have to copy them to the computer for decryption.

27 July 2014

Christian Perrier: Developers per country (July 2014)

This is time again for my annual report about the number of developers per country. This is now the sixth edition of this report. Former editions: So, here we are with the July 2014 version, sorted by the ratio of *active* developers per million population for each country.
Act: number of active developers
Dev: total number of developers
A/M: number of active devels per million pop.
D/M: number of devels per million pop.
2009: rank in 2009
2010: rank in 2010
2011: rank in 2011 (June)
2012: rank in 2012 (June)
2013: rank in 2012 (July)
2014: rank now
Code Name Population Act Dev Dev Act/Million Dev/Million 2009 2010 June 2011 June 2012 July 2013 July 2014
fi Finland 5259250 19 31 3,61 5,89 1 1 1 1 1 1
ie Ireland 4670976 13 17 2,78 3,64 13 9 6 2 2 2
nz New Zealand 4331600 11 15 2,54 3,46 4 3 5 7 7 3 *
mq Martinique 396404 1 1 2,52 2,52

3 4 4 4
se Sweden 9088728 22 37 2,42 4,07 3 6 7 5 5 5
ch Switzerland 7870134 19 29 2,41 3,68 2 2 2 3 3 6 *
no Norway 4973029 11 14 2,21 2,82 5 4 4 6 6 7 *
at Austria 8217280 18 29 2,19 3,53 6 8 10 10 10 8 *
de Germany 81471834 164 235 2,01 2,88 7 7 9 9 8 9 *
lu Luxemburg 503302 1 1 1,99 1,99 8 5 8 8 9 10 *
fr France 65350000 101 131 1,55 2 12 12 11 11 11 11
au Australia 22607571 32 60 1,42 2,65 9 10 12 12 12 12
be Belgium 11071483 14 17 1,26 1,54 10 11 13 13 13 13
uk United-Kingdom 62698362 77 118 1,23 1,88 14 14 14 14 14 14
nl Netherlands 16728091 18 40 1,08 2,39 11 13 15 15 15 15
ca Canada 33476688 34 63 1,02 1,88 15 15 17 16 16 16
dk Denmark 5529888 5 10 0,9 1,81 17 17 16 17 17 17
es Spain 46754784 34 56 0,73 1,2 16 16 19 18 18 18
it Italy 59464644 36 52 0,61 0,87 23 22 22 19 19 19
hu Hungary 10076062 6 12 0,6 1,19 18 25 26 20 24 20 *
cz Czech Rep 10190213 6 6 0,59 0,59 21 20 21 21 20 21 *
us USA 313232044 175 382 0,56 1,22 19 21 25 24 22 22
il Israel 7740900 4 6 0,52 0,78 24 24 24 25 23 23
hr Croatia 4290612 2 2 0,47 0,47 20 18 18 26 25 24 *
lv Latvia 2204708 1 1 0,45 0,45 26 26 27 27 26 25 *
bg Bulgaria 7364570 3 3 0,41 0,41 25 23 23 23 27 26 *
sg Singapore 5183700 2 2 0,39 0,39


33 33 27 *
uy Uruguay 3477778 1 2 0,29 0,58 22 27 28 28 28 28
pl Poland 38441588 11 15 0,29 0,39 29 29 30 30 30 29 *
jp Japan 127078679 36 52 0,28 0,41 30 28 29 29 29 30 *
lt Lithuania 3535547 1 1 0,28 0,28 28 19 20 22 21 31 *
gr Greece 10787690 3 4 0,28 0,37 33 38 34 35 35 32 *
cr Costa Rica 4301712 1 1 0,23 0,23 31 30 31 31 31 33 *
by Belarus 9577552 2 2 0,21 0,21 35 36 39 39 32 34 *
ar Argentina 40677348 8 10 0,2 0,25 34 33 35 32 37 35 *
pt Portugal 10561614 2 4 0,19 0,38 27 32 32 34 34 36 *
sk Slovakia 5477038 1 1 0,18 0,18 32 31 33 36 36 37 *
rs Serbia 7186862 1 1 0,14 0,14



38 38
tw Taiwan 23040040 3 3 0,13 0,13 37 34 37 37 39 39
br Brazil 192376496 18 21 0,09 0,11 36 35 38 38 40 40
cu Cuba 11241161 1 1 0,09 0,09
38 41 41 41 41
co Colombia 45566856 4 5 0,09 0,11 41 44 46 47 46 42 *
kr South Korea 48754657 4 6 0,08 0,12 39 39 42 42 42 43 *
gt Guatemala 13824463 1 1 0,07 0,07



43 44 *
ec Ecuador 15007343 1 1 0,07 0,07
40 43 43 45 45
cl Chile 16746491 1 2 0,06 0,12 42 41 44 44 47 46 *
za South Africa 50590000 3 10 0,06 0,2 38 48 48 48 48 47 *
ru Russia 143030106 8 9 0,06 0,06 43 42 47 45 49 48 *
mg Madagascar 21281844 1 1 0,05 0,05 44 37 40 40 50 49 *
ro Romania 21904551 1 2 0,05 0,09 45 43 45 46 51 50 *
ve Venezuela 28047938 1 1 0,04 0,04 40 45 50 49 44 51 *
my Malaysia 28250000 1 1 0,04 0,04

49 50 52 52
pe Peru 29907003 1 1 0,03 0,03 46 46 51 51 53 53
tr Turkey 74724269 2 2 0,03 0,03 47 47 52 52 54 54
ua Ukraine 45134707 1 1 0,02 0,02 48 53 58 59 55 55
th Thailand 66720153 1 2 0,01 0,03 50 50 54 54 56 56
eg Egypt 80081093 1 3 0,01 0,04 51 51 55 55 57 57
mx Mexico 112336538 1 1 0,01 0,01 49 49 53 53 58 58
cn China 1344413526 10 14 0,01 0,01 53 53 57 56 59 59
in India 1210193422 8 9 0,01 0,01 52 52 56 57 60 60
sv El Salvador 7066403 0 1 0 0,14

36 58 61 61































969 1561 62,08%







A few interesting facts:

25 July 2014

Juliana Louback: Extending an xTuple Business Object

xTuple is in my opinion incredibly well designed; the code is clean and the architecture ahderent to a standardized structure. All this makes working with xTuple software quite a breeze. I wanted to integrate JSCommunicator into the web-based xTuple version. JSCommunicator is a SIP communication tool, so my first step was to create an extension for the SIP account data. Luckily for me, the xTuple development team published an awesome tutorial for writing an xTuple extension. xTuple cleverly uses model based business objects for the various features available. This makes customizing xTuple very straightforward. I used the tutorial mentioned above for writing my extension, but soon noticed my goals were a little different. A SIP account has 3 data fields, these being the SIP URI, the account password and an optional display name. xTuple currently has a business object in the core code for a User Account and it would make a lot more sense to simply add my 3 fields to this existing business object rather than create another business object. The tutorial very clearly shows how to extend a business object with another business object, but not how to extend a business object with only new fields (not a whole new object). Now maybe I m just a whole lot slower than most people, but I had a ridiculously had time figuring this out. Mind you, this is because I m slow, because the xTuple documentation and code is understandable and as self-explanatory as it gets. I think it just takes a bit to get used to. Either way, I thought this just might be useful to others so here is how I went about it. Setup First you ll have to set up your xTuple development environment and fork the xtuple and xtuple-extesions repositories as shown in this handy tutorial. A footnote I d like to add is please verify that your version of Vagrant (and anything else you install) is the one listed in the tutorial. I think I spent like two entire days or more on a wild goose (bug) chase trying to set up my environment when the cause of all the errors was that I somehow installed an older version of Vagrant - 1.5.4 instead of 1.6.3. Please don t make the same mistake I did. Actually if for some reason you get the following error when you try using node:
<<ERROR 2014-07-10T23:52:46.948Z>> Unrecoverable exception. Cannot call method 'extend' of undefined
    at /home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:37:39
    at Object.<anonymous> (/home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:1364:3)
    ...
chances are, you have the wrong version. That s what happened to me. The Vagrant Virtual Development Environment automatically installs and configures everything you need, it s ready to go. So if you find yourself installing and updating and apt-gets and etc, you probably did something wrong. Coding So by now we should have the Vagrant Virtual Development Environment set up and the web app up and running and accessible at localhost:8443. So far so good. Disclaimer: You will note that much of this is similar - or rather, nearly identical - to xTuple s tutorial but there are some small but important differences and a few observations I think might be useful. Other Disclaimer: I m describing how I did it, which may or may not be up to snuff . Works for me though. Schema First let s make a schema for the table we will create with the new custom fields. Be sure to create the correct directory stucture, aka /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>/database/source or in my case /path/to/xtuple-extensions/source/sip_account/database/source, and create the file create_sa_schema.sql, sa is the name of my schema. This file will contain the following lines:
do $$
  /* Only create the schema if it hasn't been created already */
  var res, sql = "select schema_name from information_schema.schemata where schema_name = 'sa'",
  res = plv8.execute(sql);
  if (!res.length)  
    sql = "create schema sa; grant all on schema sa to group xtrole;"
    plv8.execute(sql);
   
$$ language plv8;
Of course, feel free to replace sa with your schema name of choice. All the code described here can be found in my xtuple-extensions fork, on the sip_ext branch. Table We ll create a table containing your custom fields and a link to an existing table - the table for the existing business object you want to extend. If you re wondering why make a whole new table for a few extra fields, here s a good explanation, the case in question is adding fields to the Contact business object. You need to first figure out what table you want to link to. This might not be uber easy. I think the best way to go about it is to look at the ORMs. The xTuple ORMs are a JSON mapping between the SQL tables and the object-oriented world above the database, they re .json files found at path/to/xtuple/node_modules/xtuple/enyo-client/database/orm/models for the core business objects and at path/to/xtuplenyo-client/extensions/source/<EXTENSION NAME>/database/orm/models for exension business objects. I ll give two examples. If you look at contact.json you will see that the Contact business object refers to the table cntct . Look for the type : Contact on the line above, so we know it s the Contact business object. In my case, I wanted to extend the UserAccount and UserAccountRelation business objects, so check out user_account.json. The table listed for UserAccount is xt.usrinfo and the table listed for UserAccountRelation is xt.usrlite. A closer look at the sql files for these tables (usrinfo.sql and usrlite.sql) revealed that usrinfo is in fact a view and usrlite is A light weight table of user information used to avoid punishingly heavy queries on the public usr view . I chose to refer to xt.usrlite - that or I received error messages when trying the other table names. Now I ll make the file /path/to/xtuple-extensions/source/sip_account/database/source/usrlitesip.sql, to create a table with my custom fields plus the link to the urslite table. Don t quote me on this, but I m under the impression that this is the norm for naming the sql file joining tables: the name of the table you are referring to ( usrlite in this case) and your extension s name. Content of usrlitesip.sql:
select xt.create_table('usrlitesip', 'sa');
select xt.add_column('usrlitesip','usrlitesip_id', 'serial', 'primary key', 'sa');
select xt.add_column('usrlitesip','usrlitesip_usr_username', 'text', 'references xt.usrlite (usr_username)', 'sa');
select xt.add_column('usrlitesip','usrlitesip_uri', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_name', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_password', 'text', '', 'sa');
comment on table sa.usrlitesip is 'Joins User with SIP account';
Breaking it down, line 1 creates the table named usrlitesip (no duh), line 2 is for the primary key (self-explanatory). You can then add any columns you like, just be sure to add one that references the table you want to link to. I checked usrlite.sql and saw the primary key is usr_username, be sure to use the primary key of the table you are referencing. You can check what you made by executing the .sql files like so:
$ cd /path/to/xtuple-extensions/source/sip_account/database/source
$ psql -U admin -d dev -f create_sa_schema.sql
$ psql -U admin -d dev -f usrlitesip.sql
After which you will see the table with the columns you created if you enter:
$ psql -U admin -d dev -c "select * from sa.usrlitesip;"
Now create the file /path/to/xtuple-extensions/source/sip_account/database/source/manifest.js to put the files together and in the right order. It should contain:
 
  "name": "sip_account",
  "version": "1.4.1",
  "comment": "Sip Account extension",
  "loadOrder": 999,
  "dependencies": ["crm"],
  "databaseScripts": [
    "create_sa_schema.sql",
    "usrlitesip.sql",
    "register.sql"
  ]
 
I think the name has to be the same you named your extension directory as in /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>. I think the comment can be anything you like and you want your loadOrder to be high so it s the last thing installed (as it s an add on.) So far we are doing exactly what s instructed in the xTuple tutorial. It s repetitive, but I think you can never have too many examples to compare to. In databaseScripts you will list the two .sql files you just created for the schema and the table, plus another file to be made in the same directory named register.sql. I m not sure why you have to make the register.sql or even if you indeed have to. If you leave the file empty, there will be a build error, so put a ; in the register.sql or remove the line register.sql from manifest.js as I think for now we are good without it. Now let s update the database with our new extension:
$ cd /path/to/xtuple
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ psql -U admin -d dev -c "select * from xt.ext;"
That last command should display a table with a list of extensions; the ones already in xtuple like crm and billing and some others plus your new extension, in this case sip_account . When you run build_app.js you ll probably see a message along the lines of <Extension name> has no client code, not building client code and that s fine because yeah, we haven t worked on the client code yet. ORM Here s where things start getting different. So ORMs link your object to an SQL table. But we DON T want to make a new business object, we want to extend an existing business object, so the ORM we will make will be a little different than the xTuple tutorial. Steve Hackbarth kindly explained this new business object/existing business object ORM concept here. First we ll create the directory /path/to/xtuple-extensions/source/sip_account/database/orm/ext, according to xTuple convention. ORMs for new business objects would be put in /path/to/xtuple-extensions/source/sip_account/database/orm/models. Now we ll create the .json file /path/to/xtuple-extensions/source/sip_account/database/orm/ext/user_account.jscon for our ORM. Once again, don t quote me on this, but I think the name of the file should be the name of the business object you are extending, as is done in the turorial example extending the Contact object. In our case, UserAccount is defined in user_account.json and that s what I named my extension ORM too. Here s what you should place in it:
 1 [
 2    
 3     "context": "sip_account",
 4     "nameSpace": "XM",
 5     "type": "UserAccount",
 6     "table": "sa.usrlitesip",
 7     "isExtension": true,
 8     "isChild": false,
 9     "comment": "Extended by Sip",
10     "relations": [
11        
12         "column": "usrlitesip_usr_username",
13         "inverse": "username"
14        
15     ],
16     "properties": [
17        
18         "name": "uri",
19         "attr":  
20           "type": "String",
21           "column": "usrlitesip_uri",
22           "isNaturalKey": true
23          
24        ,
25        
26         "name": "displayName",
27         "attr":  
28           "type": "String",
29           "column": "usrlitesip_name"
30          
31        ,
32        
33         "name": "sipPassword",
34         "attr":  
35           "type": "String",
36           "column": "usrlitesip_password"
37          
38        
39     ],
40     "isSystem": true
41    ,
42    
43     "context": "sip_account",
44     "nameSpace": "XM",
45     "type": "UserAccountRelation",
46     "table": "sa.usrlitesip",
47     "isExtension": true,
48     "isChild": false,
49     "comment": "Extended by Sip",
50     "relations": [
51        
52         "column": "usrlitesip_usr_username",
53         "inverse": "username"
54        
55     ],
56     "properties": [
57        
58         "name": "uri",
59         "attr":  
60           "type": "String",
61           "column": "usrlitesip_uri",
62           "isNaturalKey": true
63          
64        ,
65        
66         "name": "displayName",
67         "attr":  
68           "type": "String",
69           "column": "usrlitesip_name"
70          
71        ,
72        
73         "name": "sipPassword",
74         "attr":  
75           "type": "String",
76           "column": "usrlitesip_password"
77          
78        
79     ],
80     "isSystem": true
81    
82 ]
Note the context is my extension name, because the context + nameSpace + type combo has to be unique. We already have a UserAccount and UserAccountRelation object in the XM namespace in the xtuple context in the original user_account.json, now we will have a UserAccount and UserAccountRelation object in the XM namespace in the sip_account conext. What else is important? Note that isExtension is true on lines 7 and 47 and the relations item contains the column of the foreign key we referenced. This is something you might want to verify: column (lines 12 and 52) is the name of the attribute on your table. When we made a reference to the primary key usr_usrname from the xt.usrlite table we named that column usrlitesip_usr_usrname. But the inverse is the attribute name associated with the original sql column in the original ORM. Did I lose you? I had a lot of trouble with this silly thing. In the original ORM that created a new UserAccount business object, the primary key attribute is named username , as can be seen here. That is what should be used for the inverse value. Not the sql column name (usr_username) but the object attribute name (username). I m emphasizing this because I made that mistake and if I can spare you the pain I will. If we rebuild our extension everything should come along nicely, but you won t see any changes just yet in the web app because we haven t created the client code. Client Create the directory /path/to/xtuple-extensions/source/sip_account/client which is where we ll keep all the client code. Extend Workspace View I want the fields I added to show up on the form to create a new User Account, so I need to extend the view for the User Account workspace. I ll start by creating a directory /path/to/xtuple-extensions/source/sip_account/client/views and in it creating a file named workspace.js containing this code:
XT.extensions.sip_account.initWorkspace = function ()  
	var extensions = [
  	 kind: "onyx.GroupboxHeader", container: "mainGroup", content: "_sipAccount".loc() ,
  	 kind: "XV.InputWidget", container: "mainGroup", attr: "uri"  ,
  	 kind: "XV.InputWidget", container: "mainGroup", attr: "displayName"  ,
  	 kind: "XV.InputWidget", container: "mainGroup", type:"password", attr: "sipPassword"  
	];
	XV.appendExtension("XV.UserAccountWorkspace", extensions);
 ;
So I m initializing my workspace and creating an array of items to add (append) to view XV.UserAccountWorkspace. The first item is this onyx.GroupboxHeader which is a pretty divider for my new form fields, the kind you find in the web app at Setup > User Accounts, like Overview . I have no idea what other options there are for container other than mainGroup , so let s stick to that. I ll explain content: _sipAccount .loc() in a bit. Next I created three input fields of the XV.InputWidget kind. This also confused me a bit as there are different kinds of input to be used, like dropdowns and checkboxes. The only advice I can give is snoop around the webapp, find an input you like and look up the corresponding workspace.js file to see what was used. What we just did is (should be) enough for the new fields to show up on the User Account form. But before we see things change, we have to package the client. Create the file /path/to/xtuple-extensions/source/sip_account/client/views/package.js. This file is needed to package groups of files and indicates the order the files should be loaded (for more on that, see this). For now, all the file will contain is:
enyo.depends(
"workspace.js"
);
You also need to package the views directory containing workspace.js, so create the file Create the file /path/to/xtuple-extensions/source/sip_account/client/package.js and in it show that the directory views and its contents must be part of the higher level package:
enyo.depends(
"views"
);
I like to think of it as a box full of smaller boxes. This will sound terrible, but apparently you also need to create the file /path/to/xtuple-extensions/source/sip_account/client/core.js containing this line:
XT.extensions.icecream =  ;
I don t know why. As soon as I find out I ll be sure to inform you. As we ve added a file to the client directory, be sure to update /path/to/xtuple-extensions/source/sip_account/client/package.js so it included the new file:
enyo.depends(
"core.js",
"views"
);
Translations Remember _sipAccount .loc() in our workspace.js file? xTuple has great internationalization support and it s easy to use. Just create the directory and file /path/to/xtuple-extensions/source/sip_account/client/en/strings.js and in it put key-value pairs for labels and their translation, like this:
(function ()  
  "use strict";
  var lang = XT.stringsFor("en_US",  
    "_sipAccount": "Sip Account",
    "_uri": "Sip URI",
    "_displayName": "Display Name",
    "_sipPassword": "Password"
   );
  if (typeof exports !== 'undefined')  
    exports.language = lang;
   
 ());
So far I included all the labels I used in my Sip Account form. If you write the wrong label (key) or forget to include a corresponding key-value pair in strings.js, xTuple will simply name your lable _lableName , underscore and all. Now build your extension and start up the server:
$ cd /path/to/xtuple 
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ node node-datasource/main.js
If the server is already running, just stop it and restart it to reflect your changes. Now if you go to Setup > User Accounts and click the + button, you should see a nice little addition to the form with a Sip Account divider and three new fields. Nice, eh? Extend Parameters Currently you can search your User Accounts list using any of the User Account fields. It would be nice to be able to search with the Sip account fields we added as well. To do that, let s create the directory /path/to/xtuple-extensions/source/sip_account/client/widgets and there create the file parameter.js to extend XV.UserAccountListParameters. One again, you ll have to look this up. In the xTuple code you ll find the application s parameter.js in /path/to/xtuple/enyo-client/application/source/widgets. Search for the business object you are extending (for example, XV.UserAccount) and look for some combination of the business object name and Parameters . If there s more than one, try different ones. Not a very refined method, but it worked for me. Here s the content of our parameter.js:
XT.extensions.sip_account.initParameterWidget = function ()  
  var extensions = [
     kind: "onyx.GroupboxHeader", content: "_sipAccount".loc() ,
     name: "uri", label: "_uri".loc(), attr: "uri", defaultKind: "XV.InputWidget" ,
     name: "displayName", label: "_displayName".loc(), attr: "displayName", defaultKind: "XV.InputWidget" 
  ];
  XV.appendExtension("XV.UserAccountListParameters", extensions);
 ;
Node that I didn t include a search field for the password attribute for obvious reasons. Now once again, we package this new code addition by creating a /path/to/xtuple-extensions/source/sip_account/client/widgets/package.js file:
enyo.depends(
"parameter.js"
);
We also have to update /path/to/xtuple-extensions/source/sip_account/client/package.js:
enyo.depends(
"core.js",
"widgets"
"views"
);
Rebuild the extension (and restart the server) and go to Setup > User Accounts. Press the magnifying glass button on the upper left side of the screen and you ll see many options for filtering the User Accounts, among them the SIP Uri and Display Name. Extend List View You might want your new fields to show up on the list of User Accounts. There s a bit of an issue here because unlike what we did in workspace.js and parameter.js, we can t append new things to the list of UserAccounts with the funciton XV.appendExtension(args). First I tried overwriting the original UserAccountList, which works but it s far from ideal as this could result in a loss of data from the core implementation. After some discussion with the xTuple dev community, now there s a better alternative: Create the file /path/to/xtuple-extensions/source/sip_account/client/views/list.js and add the following:
1 var oldUserAccountListCreate = XV.UserAccountList.prototype.create;
2 
3 XV.UserAccountList.prototype.create = function ()  
4   oldUserAccountListCreate.apply(this, arguments);
5   this.createComponent(
6    kind: "XV.ListColumn", container: this.$.listItem, components: [
7         kind: "XV.ListAttr", attr: "uri" 
8    ] )
9  ;
To understand what I m doing, check out the XV.UserAccountList implementation in /path/to/xtuple/enyo-client/application/source/views/list.js the entire highlighted part. What we are doing is extending XV.UserAccountList through prototype-chaining ; this is how inheritance works with Enyo. In line 1 we create a prototype and in line 4 we inherit the features including original components array which the list is based on. We then create an additional component immitating the setup shown in XV.UserAccountList: An XV.ListColumn containing an XV.ListAttr, which should be placed in the XV.ListItem components array as is done with the existing columns (refer to implementation). Components can or should (?) have names which are used to access said components. You d refer to a specific component by the this.$.componentName hash. The components in XV.UserAccountList don t have names, so Enyo automatically names them (apparently) based on the kind name, for example something of the kind ListItem is named listItem. I found this at random after a lot of trial and error and it s not a bullet proof solution. Can be bettered. It s strange because if you encapsulate that code with
XT.extensions.sip_account.initList = function ()  
 //Code here
 ;
as is done with parameter.js and workspace.js (and in the xTuple tutorial you are supposed to do that with a new business object), it doesn t work. I have no idea why. This might be wrong or against xTuple coding norms; I will find out and update this post ASAP. But it does work this way. * shrugs * That said, as we ve created the list.js file, we need to ad it to our package by editing /path/to/xtuple-extensions/source/sip_account/client/views/package.js:
enyo.depends(
"list.js",
"workspace.js"
);
That s all. Rebuild the app and restart your server and when you select Setup > User Accounts in the web app you should see the Sip URI displayed on the User Accounts that have the Sip Account data. Add a new User Account to try this out.

Next.

Previous.