Search Results: "ag"

14 March 2014

Andrew Pollock: [life] Day 45, BJJ, Science Friday and baking cookies

We had a pretty busy day today. Sarah dropped Zoe off at around 8:30am. I reminded Zoe that she'd forgotten to give Sarah her crown that she'd made at Playgroup the previous day, so she gave that to her. After Sarah left, I let Zoe watch a little bit of TV while I finished getting myself organised. We drove to Brazilian Jiu Jitsu class today, because I wanted to go directly to the Planetarium afterwards. Zoe's BFF Megan came to BJJ class today for a trial class. Zoe was very excited. Megan picked everything up really quickly, and the girls had a good time. She didn't want to wear her uniform today though. I managed to get out of her later that she felt it made her sweaty. I told her it was probably all the running around that made her sweaty, not the uniform. We went out for a coffee with Megan, her Dad and her little sister afterwards. We found a little take away shop around the corner. Not quite the place that did babyccinos unfortunately, so Megan's Dad very cleverly got a chocolate milk and split it three ways with the girls and called it a "milkshake". I've realised that for Science Friday, I've pretty much been doing solely chemistry so far, so I wanted to throw in a bit of astronomy this week. The Sir Thomas Brisbane Planetarium had just recently reopened after getting a spanky new projector because 1978 called and wanted theirs back. Unfortunately, they weren't showing Perfect Little Planet today, but the slightly more specific Black Holes: Journey into the Unknown. I wanted Zoe to see something in the auditorium, so I thought we'd give it a shot anyway. I think we got to the Planetarium at around 11am, and the next show was at 1:30pm, so we had a bit of a look around the foyer for a bit, which Zoe quite enjoyed. They had a life-size model of Neil Armstrong's spacesuit, and a scale model of a Saturn V rocket. Zoe quite enjoyed both of them. I'll have to dig up some Youtube footage of a Saturn V launch. We then headed over to the cafe in the adjacent Botantic Gardens, and had a reasonable lunch. After lunch we wandered around the Botanic Gardens to kill some time, so you could say that today's Science Friday also included a goodly amount of botany as well. We went into the Tropical Dome and looked at the fish in the lily pond, and toured the fern garden, before heading back. We timed it pretty well, and got back to the Planetarium with about 10 minutes to kill. I'd prepped Zoe for the presentation, and told her that she needed to keep quiet, and if we had to leave we wouldn't be able to go back in again, and she could whisper to me if she needed to. I have to say, she was so well behaved. She sat through the whole 45 minute presentation and the projected tour of the night sky. I guess the presentation was sufficiently visually interesting to hold her attention for the duration. There was certainly a lot of "wow" out of her. I was so proud of how she went that I had to go and buy her an ice cream as a reward. After that, we went for a drive around Mount Coot-tha. I pointed out the TV towers that we can see from our balcony, and then she fell asleep on the way home. She didn't wake up particularly happy from her late, short nap, when we got home, but once she got herself sorted out, we baked the cookies that we'd made yesterday. They came out very nicely indeed. Once they were all baked, we dropped off a dozen to Megan's place, and then came home to have dinner. Bedtime went pretty well. I hope she has a good night's sleep, as we have a big day tomorrow.

Richard Hartmann: Git prize: Outstanding Contribution to Open Source/Linux/Free Software

In February, Linux Magazine contacted me, asking if I would be willing to accept the Linux New Media Award 2014 in the main category "Outstanding Contribution to Open Source/Linux/Free Software" on behalf of the Git community due to my involvement with evangelizing and vcsh. Needless to say, I was thrilled. I managed to poke Junio via someone at Google and he agreed. We also reached out within the German Git community and two maintainers of git submodule, Jens Lehmann and Heiko Voigt, joined in as well. While we didn't manage to hammer out interoperability issues of vcsh and git submodule due to time constraints and too much beer, we are planning to follow up on that. Git beat OpenStack, Python, and Ubuntu by a huge margin; sadly I don't have exact numbers (yet). More details and a rather crummy photo can be found in Linux Magazine's article. A video of the whole thing will uploaded to this page soonish. If it appears that we kept our "speech" very short, that was deliberate after the somewhat prolonged speeches beforehand ;) The aftershow event was nice even though the DJ refused to turn down the music down to tolerable levels; his reaction to people moving father away, and asking him to turn down the volume a bit, was to turn it up... Anyway, given the mix of people present during the award ceremony, very interesting discussions ensued. While I failed to convert Klaus Knopper to zsh and git, at least there's a chance that Cornelius Schuhmacher will start using vcsh and maybe even push for a separation of configuration and state in KDE. The most interesting tidbits of the evening were shared by Abhisek Devkota of cyanogenmod fame. Without spilling any secrets it's safe to say that the future of cyanogenmod is looking extremely bright and that there are surprises in the works which will have quite the impact. Last but not least, here's the physical prize: Glass trophy held by your's truly

Jan Wagner: Chemnitzer Linuxtage 2014 ahead

As Jan has previously announced, the Debian project is maintaining a booth at Chemnitzer Linux-Tage 2014, which is also organized by him. This year we will have merchandising at the booth, which is provided by Alexander Wirt and of course a demo system with Debian wheezy BabelBox as the past years. I'll drop it tomorrow, as I have a conflicting appointment on Saterday, maybe I can attend later on Sunday. In case you have spare time at the weekend ahead, it may be worth to spend it with great lectures and meet nice people over there.

Neil Williams: LAVA server on an arndale

(ARMv7 A15 dual core) I ve covered how I set up my home LAVA lab and how I got two iMX.53 boards running, the next step was to sort out the LAVA server. My initial setup relied on using my Thinkpad as the server. This was somewhat convenient as that s how I develop the code, but didn t help me run jobs like health checks because the laptop was usually somewhere other than on the home LAN. There had been a long-standing problem with using LAVA on ARM the uwsgi support didn t compile but the code being used was old. As the same code has been in Debian for a long time, I suspected that this was fixed with a newer version. So as part of migrating LAVA to django1.6, apache2.4, postgres9.3, django-tables2 0.13 and a few other updated dependencies, I have been using the version of uwsgi from Debian unstable for quite a while without problems on x86. A simple tweak to the LAVA setup scripts and a newer version of uwsgi became available and it builds just fine on armhf. (I haven t tried armel because Linaro is ARMv7 or newer and we don t test with ARMv5/ARMv6 boards, so there s no point in trying armel it s not likely to be an issue for this fix though.) (Only the current lava-deployment-tool methods need to actually compile uwsgi directly from upstream without any patches the packaging which will replace l-d-t uses precompiled binaries, just as any admin would expect.) That initial install was on one of the iMX.53 boards, which ran as slowly as anyone who has used an iMX.53 would expect. I haven t run test jobs on that install, it had a hard enough time serving the frontend. Clearly some more cores would be a good step and when an arndale became available for the home lab, the next step was to get a SATA drive onto the arndale, put Debian unstable on that and then deploy LAVA to the arndale using the Debian packages. As a bonus, these packages are built for django1.6 (using an unreleased branch). Sounds simple enough. It works on iMX.53, it should work on an arndale. Well, it does but only after some unexpected faff. arndale, u-boot and SATA I hate u-boot every single time I ever touch it, the little I know about any other u-boot device is completely useless. Uniformity is a good thing infinite permutations are just rope to hang ourselves with cute embedded nonsense hacks and I m bored of that whole scene. So, boot the arndale, interrupt the bootloader to get to the prompt. What s the first thing most people would want? help. Denied this particular u-boot has been compiled without the ability to display any help messages. u-boot has just sunk further into the depths of please can we have something else which is uniform across devices . OK, well maybe I can get by without help. sata init unknown command sata . (Go away, do something else for a bit.) The arndale isn t one of the newest boards in the ARMv7 world, LAVA has been using them for a while, so someone has probably done some work to fix this . nope. There s a workaround a nice detailed guide by Phil Endecott which details that the kernel needs to continue to be loaded from SD and then the SATA can be used for just the rootfs. Mildly annoyed (I was hoping to be able to update the arndale kernel without the hassle of mounting the SD card at all), the rest was straight-forward. Preparing the SATA and installing Debian Due to the complexity of some of the transitions currently ongoing in Debian ahead of the Jessie release freeze, uwsgi is not in Debian testing right now so this install has to be Debian unstable. That s OK, I ve been running Debian unstable on all my machines for several years now usually updating ~5 times a week.

# parted /dev/sda -s unit mb mktable msdos
# parted /dev/sda -s unit mb mkpart primary 1 -- "-0"
# mkfs.ext4 /dev/sda1
# mount /dev/sda1 /mnt/root/
(note the in the mkpart command, that catches me out most times I try this. Without it, the special -0 gets interpreted as an option to parted but parted doesn t then tell you how to fix it, it just complains about an unknown option.) The SD image I am using is a LAVA master image, based on Ubuntu, so it was easy to put debootstrap onto it.

# apt-get install debootstrap
# debootstrap --arch=armhf sid /mnt/root/ http://ftp.uk.debian.org/debian
# chroot /mnt/root/
Inside the chroot, it s Debian as usual:

apt-get update
apt-get install emdebian-archive-keyring
passwd
emdebian-archive-keyring is a package I (ab)use to sign my LAVA packaging as it has the advantage of being in Debian and Ubuntu going back to the time of Lenny, so it is always available from a repository which has already been authenticated to apt and doesn t involve downloading some random key from a website. I use passwd manually because this rootfs won t be using auto-login, it will have regular users. root passwords can be automated, if necessary. Thinking about login, remember to give the rootfs a usable tty. (The LAVA master image has a nasty habit of mangling editors like nano and vi when used over serial, so I m using echo.) For the arndale you need:

# echo T0:23:respawn:/sbin/getty -L ttySAC2 115200 vt102 >> ./etc/inittab
# echo >> ./etc/securetty
# echo ttySAC2 >> ./etc/securetty
Then to make the hostname useful, adapt this to your needs:

echo 127.0.0.1 arndale.codehelp arndale >> ./etc/hosts
Finally, get the network running at boot and add the LAVA repository:

# echo auto lo eth0 > ./etc/network/interfaces.d/arndale
# echo iface lo inet loopback >> ./etc/network/interfaces.d/arndale
# echo iface eth0 inet dhcp >> ./etc/network/interfaces.d/arndale
# echo deb http://people.linaro.org/~neil.williams/lava sid main >> ./etc/apt/sources.list.d/lava.list
# apt-get update
# exit
Once the LAVA packaging is finalised, the packages will be uploaded to a normal Linaro PPA, I ll update the packages to use an official download location and we ll move to using sane version string handling. For now, the packages are still in development. Code for the packaging is in Alioth but there are issues with anonymous access to the pkg-linaro-lava repositories currently, so I ve also pushed to http://github.com/Linaro if you are interested in building the packages yourself. See also my packaging guide which is also linked from the Links page on my own site. With that done, time to reboot, set the SATA drive as the rootfs using the horrid u-boot and install LAVA.

# reboot
> setenv bootargs console=tty0 console=ttySAC2,115200n8 drm_kms_helper.edid_firmware=edid-1920x1080.fw root=/dev/sda1
> boot
login
# apt-get install postgresql
After this operation, 63.0 MB of additional disk space will be used.

# apt-get install lava-server
After this operation, 231 MB of additional disk space will be used.

# a2dissite 000-default
# a2ensite lava-server
# apache2ctl restart
# service lava-server restart
Take note of the IP address of your new ARM LAVA server and sign in with your regular details, e.g. using OpenID before creating a temporary superuser:

# lava-server manage createsuperuser --username default --email=user@example.com
Choose a temporary password, log out, log in as the temporary superuser, promote your regular user to superuser and default owner of unrestricted devices in the django admin interface. Finally, log out and login as regular user to delete the temporary user. (Yes, that probably could do with some improvement remind me later and I ll see if a known user can be promoted from the command line.) See also Senthil s recent post on LAVA unit tests. Getting those to run without so many steps is also on the TODO list. Results
IP Address: 127.0.0.1
Is Master? : True
Uptime: 6:02:23.700000
Architecture: armv7l
Platform: Linux-3.10.1.0-1-linaro-arndale-armv7l-with-debian-jessie-sid
Insignal Arndale evaluation board based on EXYNOS5250
System memory: 1998MiB
Next! I m working on some code upstream to make it easier to add devices from the command line. The aim is that a set of simple commands will set up an initial device, run a job on that device and submit a result bundle to the LAVA server as job #1. This gives some assurance that everything has gone well. Now I can finally add health checks for the iMX.53 boards and start regular tests on those boards. The cubie boards will wait for a later blog post, there are interesting problems with the bootloader and serial connections. That s a whole different rant.

Steve Kemp: So load-balancers are awesome

When I was recently talking about load-balancers, and automatically adding back-ends, not just removing bad ones that go offline, I obviously spent a while looking over some. There are several dedicated load-balancers packaged for Debian GNU/Linux, including: In addition to actual dedicated load-balancers there are things that can be coerced into running in that way: apache2, varnish, squid, nginx, & etc. Of the load-balancers I was immediately drawn to both pen and pound, because they have command line tools ("penctl" and "poundctl" respectively) for adding/removing/updating the running configuration. Pen I've been using for a couple of days now, and although it suffers from some security issues I'm confident they will be resolved in the near future. (#741370) My only outstanding task is to juggle some hosts around and stress-test the pair of them a little more before deciding on a winner. In other news I kinda regret the whole blogspam.net API. I'd have had a far simpler life if I'd just ran the damn thing as a DNSBL in the first place. (That's essentially how it operates on the whole anyway. Submit spammy comments for long enough and you're just blacklisted, thereafter.)

Michal Čihař: phpMyAdmin participating in GSoC 2014

phpMyAdmin has been accepted for Google Summer of Code 2014. So if you are a student and thinking about how to spend this summer, you might want to join us. As usual, we have prepared dozen of ideas, so in case you are interested, it's really the time to start to work on your application. We require you to contribute before GSoC, so that we can see you can handle the code and our tools. All details you might need are available in our applicant guide. Our requirements might sound strict, but without them, we would drown in hundredths of applications with no clue how to decide, so do your homework and prepare perfect application. If you have any questions, get in touch with us on mailing list and submit the application to to GSoC website.

Filed under: English phpMyAdmin 0 comments Flattr this!

Dirk Eddelbuettel: Rcpp 0.11.1

A new minor release 0.11.1 of Rcpp is now on the CRAN network for GNU R; binaries for Debian have also been uploaded. The release fixes a number of bugs that have come up since the 0.11.0 release in January, but also brings some extensions. See the NEWS file section below for details, or the ChangeLog file in the package and on the Rcpp Changelog page Once again, we tested this release by building against all CRAN packages which depend upon Rcpp. In short, three packages are blacklisted from tests, and three came up with something we noted --- but the remaining 177 packages all build and test cleanly. Detailed results of those tests (and the scripts for it) are on GitHub. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page.
Changes in Rcpp version 0.11.1 (2014-03-13)
  • Changes in Rcpp API:
    • Preserve backwards compatibility with Rcpp 0.10.* by allowing RObject extraction from vectors (or lists) of Rcpp objects
    • Add missing default constructor to Reference class that was omitted in the header-only rewrite
    • Fixes for NA and NaN handling of the IndexHash class, as well as the vector .sort() method. These fixes ensure that sugar functions depending on IndexHash (i.e. unique(), sort_unique(), match()) will now properly handle NA and NaN values for numeric vectors.
    • DataFrame::nrows now more accurately mimics R's internal behavior (checks the row.names attribute)
    • Numerous changes to permit compilation on the Solaris OS
    • Rcpp vectors gain a subsetting method it is now possible to subset an Rcpp vector using CharacterVectors (subset a by name), LogicalVectors (logical subsetting), and IntegerVectors (0-based index subsetting). Such subsetting will also work with Rcpp sugar expressions, enabling expressions such as x[ x > 0].
    • Comma initialization (e.g. CharacterVector x = "a", "b", "c";, has been disabled, as it causes problems with the behavior of the = operator with Rcpp::Lists. Users who want to re-enable this functionality can use #define RCPP_COMMA_INITIALIZATION, but be aware of the above caveat. The more verbose CharacterVector x = CharacterVector::create("a", "b", "c") is preferred.
  • Changes in Rcpp Attributes
    • Fix issue preventing packages with Rcpp::interfaces attribute from compiling.
    • Fix behavior with attributes parsing of ::create for default arguments, and also allow constructors of a given size (e.g. NumericVector v = NumericVector(10)) gives a default value of numeric(10) at the R level). Also make NAs preserve type when exported to R (e.g. NA_STRING as a default argument maps to NA_character_ at the R level)
  • Changes in Rcpp modules
    • Corrected the un_pointer implementation for object
Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Petter Reinholdtsen: Freedombox on Dreamplug, Raspberry Pi and virtual x86 machine

The Freedombox project is working on providing the software and hardware for making it easy for non-technical people to host their data and communication at home, and being able to communicate with their friends and family encrypted and away from prying eyes. It has been going on for a while, and is slowly progressing towards a new test release (0.2). And what day could be better than the Pi day to announce that the new version will provide "hard drive" / SD card / USB stick images for Dreamplug, Raspberry Pi and VirtualBox (or any other virtualization system), and can also be installed using a Debian installer preseed file. The Debian based Freedombox is now based on Debian Jessie, where most of the needed packages used are already present. Only one, the freedombox-setup package, is missing. To try to build your own boot image to test the current status, fetch the freedom-maker scripts and build using vmdebootstrap with a user with sudo access to become root:
git clone http://anonscm.debian.org/git/freedombox/freedom-maker.git \
  freedom-maker
sudo apt-get install git vmdebootstrap mercurial python-docutils \
  mktorrent extlinux virtualbox qemu-user-static binfmt-support \
  u-boot-tools
make -C freedom-maker dreamplug-image raspberry-image virtualbox-image
Root access is needed to run debootstrap and mount loopback devices. See the README for more details on the build. If you do not want all three images, trim the make line. But note that thanks to a race condition in vmdebootstrap, the build might fail without the patch to the kpartx call. If you instead want to install using a Debian CD and the preseed method, boot a Debian Wheezy ISO and use this boot argument to load the preseed values:
url=http://www.reinholdtsen.name/freedombox/preseed-jessie.dat
But note that due to a recently introduced bug in apt in Jessie, the installer will currently hang while setting up APT sources. Killing the 'apt-cdrom ident' process when it hang a few times during the installation will get the installation going. This affect all installations in Jessie, and I expect it will be fixed soon. Give it a go and let us know how it goes on the mailing list, and help us get the new release published. :) Please join us on IRC (#freedombox on irc.debian.org) and the mailing list if you want to help make this vision come true.

Andrew Pollock: [life] Day 44, Playgroup, laksa and baking

Zoe had Playgroup this morning. We had a little bit of time to kill between when Sarah dropped her off and it was time to bike over to Playgroup, so we went for a little walk to the post box to mail a letter. It was a beautiful morning, so the walk was nice. We biked over to Playgroup. Today I learned from the mother of the boy who was causing issues last week, that he was extremely allergic to everything. They couldn't use egg cartons for their craft activities, for example, because he was allergic to eggs, and even the empty cartons would cause a problem, apparently. That's quite the allergic sensitivity. I'm glad Zoe outgrew her egg allergy. Zoe wanted to do the craft activity first up, so she made another crown, this time for Mummy. I helped roll scraps of crepe paper into balls, and she very carefully glued them onto the points of the crown. She seems to really enjoy mucking around with Clag glue pots. I'll have to get a bottle for home. It reminds of my early primary school days. After that, we just generally mucked around for the rest of Playgroup. She loved the dress up pink lacy skirt again, and wanted to fool around on stage for a bit. One of my ex-co-workers has an occasional yum cha or laksa lunch with a bunch of his ex-co-workers, and I was invited along to laksa at Super Bowl. I figured that since Zoe loves any opportunity to use her chopsticks, and generally loves a good noodle soup, that this could be something I could bring her along to. So straight after Playgroup, we biked over to the Bulimba ferry terminal, locked up the bike, took the ferry across to Teneriffe and jumped on a City Glider bus to Fortitude Valley. We ended up getting there about 30 minutes earlier than everyone else, so I just ordered Zoe some wonton noodle soup and let her have at it while we waited for everyone else to arrive. We all had a BBQ pork laksa, which I personally found a little bit too spicy for my liking, but it was doable. Lunch went well. Zoe was really well behaved, so I'll definitely bring her along again. The next one is supposed to be yum cha, so I think she'll enjoy the selection. Zoe was looking pretty tired by the end of lunch, and we reversed our way back, and biked home from Bulimba. Once we got home, she practically put herself to bed for a nap. Since she was napping pretty late, I didn't let her nap for too long, and she woke up happy. After her nap, I thought we'd do some baking. I wanted to try and make some oatmeal cookies like our friend Bri used to make, so I found a recipe and adapted it to the Thermomix, and we got stuck into it. The joys of having a small child help is that not always do the ingredients wind up where they're supposed to. We had to mill a cup of raw sugar down to white sugar, and I had the cup's worth of sugar in a bowl and asked Zoe to tip the sugar into the Thermomix bowl. Unfortunately, she didn't line up very well, so some of the sugar made it into the Thermomix, but about as much went all over the Thermomix, the counter, the floor, the Guidecraft kitchen helper, everywhere. I've become quite good at taking these sorts of things in my stride, and didn't bat an eyelid. I was actually more bothered by Zoe asking me if I still loved her. After I reassured her I still loved her, even if she accidentally spilled sugar everywhere, and we cleaned up the sugar together, we continued to the point where I realised the dough was supposed to chill in the fridge for at least an hour. This would have meant the cookies wouldn't be ready to hit the oven until right around when Sarah was due to pick up Zoe, so I figured we'd just leave the dough in the fridge and bake the cookies tomorrow. Zoe watched some TV while I cleaned up the kitchen, and then Sarah arrived to pick her up.

13 March 2014

Mario Lang: Living in RAM

I switched to SSD on my home server a few years ago, at a time when SSH was still new and sort of hyped. However, I immediately knew that this type of technology is for me: it is fast and rather silent. SSD storage has this air of being fragile though. You are supposed to not wear it out too much. It's not like in the good old days where you would write to your hard disk without any extra thought.
tmpfs Before I switched to SSD I was already using tmpfs to store /tmp in RAM, for performance reasons mostly. /etc/default/tmpfs:
RAMTMP=yes
Since RAM is relatively cheap these days I have lots of it in my home server.
/var/cache/apt/archives When I switched to SSD, I extended this concept by mounting /var/cache/apt/archives via tmpfs. This is to save disk I/O for package updates. And for a machine running sid which is more or less regularily updated, this saves quite some useless writes to disk. Roughly 100MB per update.
/tmp/src Nowadays, I have almost all of my source code checkouts and build-trees in /tmp/src. myrepos makes this very easy to handle. Simply clone all repositories you are interested in, and remember their details with mr register. After a reboot, all you need to do is:
$ mkdir /tmp/src
$ cd !$
$ mr up
Your repository locations have (likely) been recorded in ~/.mrconfig. Of course there is the danger of accidentally loosing data due to a power outage. However, I feel this leads to a rather clean workflow. Changes need to be pushed anyway. So since you can not trust your local repository to really persist, you are forced to push your changes regularily, which is a good thing! Besides, I have a UPS at home which gives me roughly one hour of backup power. This was enough to sustain every power outage I have witnessed in the past years. Typically, outages are rather short lived in my area, somewhere between a few seconds and 5 minutes. It is rather rare that electricity goes missing for more than half an hour. The only downside of this approach is that from time to time, you will use more bandwidth by re-cloning repositories after a reboot. But really, who needs reboots on Linux? Last time I had half a year uptime, and only rebooted because I wanted to jump from linux 3.8 to 3.13. This approach only works on long-running machines though. It is probably not very useful for a laptop.

12 March 2014

Matthew Garrett: Dealing with Apple ACPI issues

I wrote about Thunderbolt on Apple hardware a while ago. Since then Andreas Noever has somehow managed to write a working Thunderbolt stack, which awesome! But there was still the problem I mentioned of the device not appearing unless you passed acpi_osi="Darwin" on the kernel command line, and a further problem that if you suspended and then resumed it vanished again.

The ACPI _OSI interface is a mechanism for the firmware to determine the OS that the system is running. It turns out that this works fine for operating systems that export fairly static interfaces (Windows, which adds a new _OSI per release) and poorly for operating systems that don't even guarantee any kind of interface stability in security releases (Linux, which claimed to be "Linux" regardless of version until we turned that off). OS X claims to be Darwin and nothing else. As I mentioned before, claiming to be Darwin in addition to Windows was enough to get the Thunderbolt hardware to stay alive after boot, but it wasn't enough to get it powered up again after suspend.

It turns out that there's two sections of ACPI code where this Mac checks _OSI. The first is something like:

if (_OSI("Darwin")) Store 0x2710 OSYS; else if(_OSI("Windows 2009") Store 0x7D9 OSYS; else

ie, if the OS claims to be Darwin, all other strings are ignored. This is called from \_SB._INI(), which is the first ACPI method the kernel executes. The check for whether to power down the Thunderbolt controller occurs after this and then works correctly.

The second version is less helpful. It's more like:

if (_OSI("Darwin")) Store 0x2710 OSYS; if (_OSI("Windows 2009")) Store 0x7D9 OSYS; if

ie, if the OS claims to be both Darwin and Windows 2009 (which Linux will if you pass acpi_osi="Darwin"), the OSYS variable gets set to the Windows 2009 value. This version gets called during PCI initialisation, and once it's run all the other Thunderbolt ACPI calls stop doing anything and the controller gets powered down after suspend/resume. That can be fixed easily enough by special casing Darwin. If the platform requests Darwin before anything else, we'll just stop claiming to be Windows.

Phew. Working Thunderbolt! (Well, almost - _OSC fails and so we disable PCIe hotplug, but that's easy to work around). But boo, no working battery. Apple do something very strange with their ACPI battery interface. If you're running anything that doesn't claim to be Darwin, Apple expose an ACPI Control Method battery. Control Method interfaces abstract the complexity away into either ACPI bytecode or system management traps - the OS simply calls an ACPI method, magic happens and it gets an answer back. If you claim to be Darwin, Apple remove that interface and instead expose the raw ACPI Smart Battery System interface. This provides an i2c bus over which the OS must then speak the Smart Battery System protocol, allowing it to directly communicate with the battery.

Linux has support for this, but it seems that this wasn't working so well and hadn't been for years. Loading the driver resulted in modprobe hanging until a timeout occurred, and most accesses to the battery would (a) take forever and (b) probably fail. It also had the nasty habit of breaking suspend and resume, which was unfortunate since getting Thunderbolt working over suspend and resume was the whole point of this exercise.

So. I modified the sbs driver to dump every command it sent over the i2c bus and every response it got. Pretty quickly I found that the failing operation was a write - specifically, a write used to select which battery should be connected to the bus. Interestingly, Apple implemented their Control Method interface by just using ACPI bytecode to speak the SBS protocol. Looking at the code in question showed that they never issued any writes, and the battery worked fine anyway. So why were we writing? SBS provides a command to tell you the current state of the battery subsystem, including which battery (out of a maximum of 4) is currently selected. Unsurprisingly, calling this showed that the battery we wanted to talk to was already selected. We then asked the SBS manager to select it anyway, and the manager promptly fell off the bus and stopped talking to us. In keeping with the maxim of "If hardware complains when we do something, and if we don't really need to do that, don't do that", this makes it work.

Working Thunderbolt and working battery. We're even getting close to getting switchable GPU support working reasonably, which is probably just going to involve rewriting the entirety of fbcon or something similarly amusing.

comment count unavailable comments

Francesca Ciceri: All hail the Spam Reviewers!

With the help of Enrico and the Debian Listmaster Team -and in particular Alexander Wirt: thank you very much! - the spam reviewers of the Debian mailing lists have been added to the list of Debian Contributors.
While some statistics about the reviewing job already existed, adding these data to the Debian Contributors list is another little step to map all kind of Debian contributions. Wondering what exactly is the job of the spam reviewers? It consists in checking all the messages of the Debian mailing lists reported as spam: if a message reported as spam gets three reviews as spam (and none as ham) it is removed from the archive.
Only DDs can do this: if you are one check here how to do it.
But there's one - and probably more important job - that everyone can do: report spam. When a message is reported as spam by 5 or more persons, it will make to the reviewers' queue. There are some organized spam cleaning efforts you can join: here there's a list of them, while on this page you can find more information on the whole process. Wondering what exactly is the Debian Contributors list? It's a brilliant effort started by Enrico to map all kind of contributions in Debian.
You can read more about it here and here.

Vasudev Kamath: Working around type system of Go with unsafe

Go language has a strong type system unlike C, and some times this will be head ache when we want to interact with C data types with Cgo or just to convert a Go type to lets say byte slice. I recently faced the same problem and after poking around things, I learned the Go provides unsafe package which can be used to work around the Go's type system.
Problem Recently I started using Cgo to use some C library I had to write some tools for development and testing. The reason I chose Go for this was prototyping and writing some quick tools is much easier in Go than done in C. The C library had some function which takes pointer to array types and fills it with some values. The problem I was facing here was how to create a C array in Go. Go does have array but its largely used as internal representation for much efficient type called slice and I can't directly cast a byte slice into an C array. Second problem I had was I had to store arbitrary Go types like float (float32,float64) and int (int32, int64) etc. into C array. So in brief, the problems that needed to solve are
  1. Find a way to convert byte slice from Go into C array and vice versa.
  2. Find a way to convert and store Go types into a C array.
Solution Basically C array are sequence of memory location which can be statically or dynamically allocated. In Cgo its possible to access C standard library functions for memory allocation, so why not use it. The memory allocation function then returns the pointer to starting of allocated memory, we can use this pointer to write Go's bytes into the memory location and bytes from memory location into Go's slice. The pointer returned by C allocation functions are not directly usable for memory dereferencing in Go, here is where unsafe package kicks in. We will cast the return of C allocation function as unsafe.Pointer type and from the documentation of unsafe package,
  1. A pointer value of any type can be converted to a Pointer.
  2. A Pointer can be converted to a pointer value of any type.
  3. A uintptr can be converted to a Pointer.
  4. A Pointer can be converted to a uintptr.
So we can then cast unsafe.Pointer to uintptr which is the Go type which is large enough to hold any memory address in Go and can be used for pointer arithmetic just like we do in C (of course with some more castings). Below I'm pasting a simplified code in C which I wrote for this post.
#ifndef __BYTETEST_H__
#define __BYTETEST_H__
typedef unsigned char UBYTE;
extern void ArrayReadFunc(UBYTE *arrayout);
extern void ArrayWriteFunc(UBYTE *arrayin);
#endif
#include "bytetest.h"
#include <stdio.h>
#include <string.h>
void ArrayReadFunc(UBYTE *arrayout)
 
     UBYTE array[20] =  1, 2, 3, 4,5, 6, 7, 8, 9, 10,
                        11, 12, 13, 14, 15, 16, 17,
                        18, 19, 20 ;
     memcpy(arrayout, array, 20);
 
void ArrayWriteFunc(UBYTE *arrayin)
 
     UBYTE array[20];
     memcpy(array, arrayin, 20);
     printf("Byte slice array received from Go:\n");
     for(int i = 0; i < 20; i ++) 
             printf("%d ", array[i]);
      
     printf("\n");
 
Functions are written just for this post and they don't really do anything. As you can see ArrayReadFunc takes a pointer to array and fills it with content of another array using memcpy. Function ArrayWriteFunc on other hand takes pointer to array and copies its content to internal array. I've added print logic to ArrayWriteFunc just to show that values passed from Go are making it here. Below is the Go code which uses the above C files passes byte slice to get value out of C code and array made of byte slice to C function to send values in.
package main
/*
#cgo CFLAGS: -std=c99
#include "bytetest.h"
#include <stdlib.h>
*/
import "C"
import (
     "fmt"
     "unsafe"
)
func ReadArray() unsafe.Pointer  
       var outArray = unsafe.Pointer (C.calloc(20,1))
       C.ArrayReadFunc((*C.UBYTE)(outArray))
       return outArray
 
func WriteArray(inArray unsafe.Pointer)  
       C.ArrayWriteFunc((*C.UBYTE)(inArray))
 
func CArrayToByteSlice(array unsafe.Pointer, size int) []byte  
       var arrayptr = uintptr(array)
       var byteSlice = make([]byte, size)
       for i := 0; i < len(byteSlice); i ++  
               byteSlice[i] = byte(*(*C.UBYTE)(unsafe.Pointer(arrayptr)))
               arrayptr ++
        
       return byteSlice
 
func ByteSliceToCArray (byteSlice []byte) unsafe.Pointer  
       var array = unsafe.Pointer(C.calloc(C.size_t(len(byteSlice)), 1))
       var arrayptr = uintptr(array)
       for i := 0; i < len(byteSlice); i ++  
              *(*C.UBYTE)(unsafe.Pointer(arrayptr)) = C.UBYTE(byteSlice[i])
              arrayptr ++
        
       return array
 
func main() 
        carray := ReadArray()
        defer C.free(carray)
        carraybytes := CArrayToByteSlice(carray, 20)
        fmt.Println("C array converted to byte slice:")
        for i := 0; i < len(carraybytes); i ++  
                fmt.Printf("%d ", carraybytes[i])
         
        fmt.Println()
        gobytes := []byte 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
                31, 32, 33, 34, 35, 36, 37, 38, 39, 40 
        gobytesarray := ByteSliceToCArray(gobytes)
        defer C.free(gobytesarray)
        WriteArray(gobytesarray)
 
Functions ReadArray and WriteArray are just wrapper to the calls to C counter parts ArrayReadFunc and ArrayWriteFunc. ReadArray returns unsafe.Pointer which is allocated C array and should be freed by caller. WriteArray takes unsafe.Pointer which is pointing to memory location containing C array. Now the functions of interest are CArrayToByteSlice and ByteSliceToCArray. It should be pretty clear from the above code to understand what is happening in these functions. Still I will just put explain them briefly. ByteSliceToCArray allocates a C array using calloc from C standard library. It then creates a uintptr, a pointer type in Go which is used to dereference the each memory location and store bytes from the input byte slice in them. CArrayToByteSlice on other hands creates a uintptr type by casting input unsafe.Pointer type and then uses this pointer type to dereference values from memoy and store it in byte slice with suitable casting. So lets build the code and run it and see the output:
C array converted to byte slice:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Byte slice array received from Go:
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
So yes it actually works and values are moving across C and Go. This solves first problem in hand next is converting arbitrary Go types into byte slices. There are many cases where we would like to convert an arbitrary Go types like (int, float) into bytes. One such use case I found was when writing a TCP client for communicating with a Server written using C speaking custom protocol. Here I'm just going to show how to convert types like float, int to byte slice, I've not tried converting structures but it is certainly possible. Below is the function which can convert int32,float32 into byte slice it can also be extended for other types.
func CopyValueToByte(value interface ) []byte  
     var valptr uintptr
     var slice []byte
     switch t := value.(type)  
     case int32:
             i := value.(int32)
             valptr = uintptr(unsafe.Pointer(&i))
             slice = make([]byte, unsafe.Sizeof(i))
     case float32:
             f := value.(float32)
             valptr = uintptr(unsafe.Pointer(&f))
             slice = make([]byte, unsafe.Sizeof(f))
     default:
             fmt.Fprintf(os.Stderr,"Unsupported type: %T\n", t)
             os.Exit(1)
      
     for i := 0; i < len(slice); i++  
             slice[i] = byte(*(*byte)(unsafe.Pointer(valptr)))
             valptr++
      
     return slice
 
This function is generic which can take various types of value. First we will use Go's type assertion to determine the type and creates a uintptr pointer for the value and allocates byte slice depending on the size of the value as calculated using unsafe.Sizeof. Later it uses the pointer to dereference value from memory location and copies each byte into byte slice. The idea used here is every type is represented as certain number of bytes in the memory. Below is the entire program.
package main
import (
     "fmt"
     "unsafe"
     "os"
)
func CopyValueToByte(value interface ) []byte  
     var valptr uintptr
     var slice []byte
     switch t := value.(type)  
     case int32:
             i := value.(int32)
             valptr = uintptr(unsafe.Pointer(&i))
             slice = make([]byte, unsafe.Sizeof(i))
     case float32:
             f := value.(float32)
             valptr = uintptr(unsafe.Pointer(&f))
             slice = make([]byte, unsafe.Sizeof(f))
     default:
             fmt.Fprintf(os.Stderr,"Unsupported type: %T\n", t)
             os.Exit(1)
      
     for i := 0; i < len(slice); i++  
             slice[i] = byte(*(*byte)(unsafe.Pointer(valptr)))
             valptr++
      
     return slice
 
func main()  
     a := float32(-10.3)
     floatbytes := CopyValueToByte(a)
     fmt.Println("Float value as byte slice:")
     for i := 0; i < len(floatbytes); i++  
             fmt.Printf("%x ", floatbytes[i])
      
     fmt.Println()
     b := new(float32)
     bptr := uintptr(unsafe.Pointer(b))
     for i := 0; i < len(floatbytes); i++  
             *(*byte)(unsafe.Pointer(bptr)) = floatbytes[i]
             bptr++
      
     fmt.Printf("Byte value copied to float var: %f\n", *b)
 
The above conversion can also be achieved using encoding/binary package provided by Go. But its been told to me that it makes things pretty slow.
Conclusion So goes unsafe.Pointer is really powerful thing which allows us to work around the Go's type system but as package documentation says it should be used with care
PS: I'm not really sure if its recommended to use allocation functions from C standard library, I will wait for expert gophers to comment on that.

John Goerzen: Agile Is Dead (Long Live Agility)

In an intriguing post, PragDave laments how empty the word agile has become. To paraphrase, I might say he s put words to a nagging feeling I ve had: that there are entire books about agile, conferences about agile, hallway conversations I ve heard about whether somebody is doing this-or-that agile practice correctly. Which, when it comes down to it, means that they re not being agile. If process and tools, even if they re labeled as agile processes and tools, are king, then we ve simply replaced one productivity-impairing dictator with another. And he makes this bold statement:
Here is how to do something in an agile fashion: What to do:
  • Find out where you are
  • Take a small step towards your goal
  • Adjust your understanding based on what you learned
  • Repeat
How to do it: When faced with two or more alternatives that deliver roughly the same value, take the path that makes future change easier. Those four lines and one practice encompass everything there is to know about effective software development.
He goes on to dive into that a bit, of course, but I think this man has a rare gift of expressing something complicated so succinctly. I am inclined to believe he is right.

Neil McGovern: Entering the fray 2014 DPL election

After a number of years in Debian, in various roles, I decided to put my name forward for election to Debian Project Leader. My platform has now been published, and campaigning is now under way!
Featured image is CC-BY by State Library of Victoria Collections

Petter Reinholdtsen: How to add extra storage servers in Debian Edu / Skolelinux

On larger sites, it is useful to use a dedicated storage server for storing user home directories and data. The design for handling this in Debian Edu / Skolelinux, is to update the automount rules in LDAP and let the automount daemon on the clients take care of the rest. I was reminded about the need to document this better when one of the customers of Skolelinux Drift AS, where I am on the board of directors, asked about how to do this. The steps to get this working are the following:
  1. Add new storage server in DNS. I use nas-server.intern as the example host here.
  2. Add automoun LDAP information about this server in LDAP, to allow all clients to automatically mount it on reqeust.
  3. Add the relevant entries in tjener.intern:/etc/fstab, because tjener.intern do not use automount to avoid mounting loops.
DNS entries are added in GOsa , and not described here. Follow the instructions in the manual (Machine Management with GOsa in section Getting started). Ensure that the NFS export points on the server are exported to the relevant subnets or machines:
root@tjener:~# showmount -e nas-server
Export list for nas-server:
/storage         10.0.0.0/8
root@tjener:~#
Here everything on the backbone network is granted access to the /storage export. With NFSv3 it is slightly better to limit it to netgroup membership or single IP addresses to have some limits on the NFS access. The next step is to update LDAP. This can not be done using GOsa , because it lack a module for automount. Instead, use ldapvi and add the required LDAP objects using an editor.
ldapvi --ldap-conf -ZD '(cn=admin)' -b ou=automount,dc=skole,dc=skolelinux,dc=no
When the editor show up, add the following LDAP objects at the bottom of the document. The "/&" part in the last LDAP object is a wild card matching everything the nas-server exports, removing the need to list individual mount points in LDAP.
add cn=nas-server,ou=auto.skole,ou=automount,dc=skole,dc=skolelinux,dc=no
objectClass: automount
cn: nas-server
automountInformation: -fstype=autofs --timeout=60 ldap:ou=auto.nas-server,ou=automount,dc=skole,dc=skolelinux,dc=no
add ou=auto.nas-server,ou=automount,dc=skole,dc=skolelinux,dc=no
objectClass: top
objectClass: automountMap
ou: auto.nas-server
add cn=/,ou=auto.nas-server,ou=automount,dc=skole,dc=skolelinux,dc=no
objectClass: automount
cn: /
automountInformation: -fstype=nfs,tcp,rsize=32768,wsize=32768,rw,intr,hard,nodev,nosuid,noatime nas-server.intern:/&
The last step to remember is to mount the relevant mount points in tjener.intern by adding them to /etc/fstab, creating the mount directories using mkdir and running "mount -a" to mount them. When this is done, your users should be able to access the files on the storage server directly by just visiting the /tjener/nas-server/storage/ directory using any application on any workstation, LTSP client or LTSP server.

Andrew Pollock: [life] Day 43, a lovely lazy day, until we got on the bike

Zoe slept through the night again last night. Two nights of uninterrupted sleep has been totally amazing! We had a slower start to the day because we didn't have to be anywhere. I've been meaning to do a massive cull of Zoe's wardrobe for a while, and this morning seemed like as good an opportunity as any, so after TV, I managed to get Zoe to cooperate with trying on clothes for about an hour. She really enjoyed pulling everything out of her wardrobe and putting it all back again neatly. After that, she asked me if I could make her a bus out of the last remaining moving box, so I managed to make something approximating a bus. The port-a-robe boxes are really rather handy for this sort of thing. The balcony is starting to look like a cardboard box shanty town, with a cubby house, a rocket and now a bus. I think we need to host a few more play dates so they all get a good work out. It was time for lunch after that. I'd had a bucket of caramel popcorn, which I used for the journey to Coochiemudlo the other week. Zoe had asked me how popcorn was made, so I had resolved to make popcorn with her sometime soon. Yesterday at the health food shop, I opportunistically noticed some popping corn, so I grabbed a small bag of it. It turned out that buying peanut oil was good for more than just making peanut butter, as it came in handy today for popping corn. I've never popped corn before, aside from occasionally getting a bag of the microwave stuff, so it was a fun experience to do it "from scratch". It turned out pretty well, with only a few burned ones and a few unpopped ones. It'd be fun to try and dry out a corn cob and totally do it from scratch. After that I tried to get Zoe to take a nap, but it didn't work out so well, so after her quiet time we cleaned up her bedroom and generally fooled around for a bit. Throughout the day, Zoe did lots of little bouts of self-directed play. She dredged up this little wooden xylophone that we'd made in the US from a Kiwi Crate and had a great time banging away on it until she stood on it and broke it. The doctor's kit got a good workout. It was just a really nice, relaxing day at home. Zoe's Kindergarten was having a play date meet-and-greet in the park, for all the parents to meet each other. It was a fair way from home, but there was a nice, albeit circuitous, bike path to get there, and I decided to try and bike there, without really properly estimating the distance or length of time it would take. The play date was due to start at 3:30pm, and I was planning on leaving at 3pm. We were about 5 minutes late getting out of the door. I think it ended up taking us about 50 minutes to get there. It was a beautiful ride though, once we got onto the Bulimba Creek bikeway, through the Minippi Parklands. It would have been the longest ride Zoe's had on the bike, and she wasn't fazed at all. The play date itself was nice. For some reason, I'm a bit shy about going and introducing myself to the mothers (I kept wanting to write "other mothers"), but I sucked it up and made a few introductions. I'm terrible about remembering names though. At least I've got a few faces I can say hi to at drop off and pick up now. Sarah met us at the park and hung out for a bit as well, and then Zoe and her left together and I biked back home. I was feeling pretty good, and thought I'd try and bang out a run when I got home, but I didn't end up having the fuel in the tank, and blew up after about 1.5km.

Dirk Eddelbuettel: QuantLib 1.4 packages also available for Ubuntu and Windoze

QuantLib release 1.4 came out a couple of days ago. Sticking with standard practice, I immediately updated the Debian packages I have been maintaining for over a dozen years now, and corresponding binaries are available for Debian flavors unstable ("sid") and testing ("jessie"). I also updated my RQuantLib package accordingly (though it needed no changes -- yay to stable APIs). However, I also run Ubuntu on a few machines at work and home, and proceeded to create local packages for both 32-bit ("i386") and 64-bit ("amd64") variants of the current release 13.10 ("saucy"). As these may be of use to others, I am now making these available here from my box so that folks can run a current QuantLib version on those machines. Lastly, Brian Ripley was kind enough to also fire up his cross-compilation setup on his Linux box to create similar Windows binaries, These are e.g. used on CRAN and by the win-builder service to compile the RQuantLib CRAN Windows builds of RQuantLib. That corresponding archive file can be obtained as Quantlib14.zip see this short page for details. My thanks to Brian for helping with this. I hope this may be useful to the few people wishing to compile and/or extend RQuantLib and QuantLib.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 March 2014

Wouter Verhelst: The apt-get install game

"Given a Debian system with just the set of essential packages installed, find a single package name that, together with its dependencies, will increase the used diskspace of the system by as much as possible." My entry: openarena. Off to bed, now.

Steve Kemp: Discovering back-end servers automatically?

Recently I've been pondering how to do service discovery. Pretend you have a load balancer which accepts traffic and routes incoming requests to different back-ends. The loadbalancer might be pound, varnish, haproxy, nginx, or similar. The back-ends might be node applications, apache, or similar. The typical configuration of the load-balancer will read:
# forward
# backends
backend web1    .host = "10.0.0.4";  
backend web2    .host = "10.0.0.6";  
backend web3    .host = "10.0.0.5";  
#  afterword
I've seen this same setup in many situations, and while it can easily be imagined that there might be "random HTTP servers" on your (V)LAN which shouldn't receive connections it seems like a pain to keep updating the backends. Using UDP/multicast broadcasts it is trivial to announce "Hey I'm a HTTP-server with the name 'foo'", and it seems to me that this should allow seamless HTTP load-balancing. To be more explicit - this is normal: What I'd like to propose is another step: i.e. This allows backends to be removed from service when they go offline but also to be added when they come online. Without the load-balancer needing its configuration to be updated. This means you'd not give a static list of back-ends to your load-balancer, instead you'd say "Route traffic to any service that adfvertises itself with the tag 'foo'.". VLANS, firewalls, multicast, udp, all come into play, but in theory this strikes me as being useful, obvious, and simple. (Failure cases? Well if the "announcer" dies then the backend won't get traffic routed to it. Just like if the backend were offline. And clearly if a backend is announced, but not receiving HTTP-requests it would be dropped as normal.) If I get the time this evening I'll sit down and look at some load-balancer source code to see if any are written in such a way that I could add this "broadcast discovery" as a plugin/minor change.

Next.

Previous.