Search Results: "ron"

5 May 2012

Ritesh Raj Sarraf: Cancer cure drugs now more affordable

Now these are the kind of moves that needs to happen more often. Whether this will cause a negative impact on the overall market, and the further invention of drugs (including patent control), but the affordability of the medicatoin to an average citizen is a great move.The typical Chemotherapy can be on an average of 22 times. When summed up with the dosage (somewhere around 250 mg IIRC), the cost comes to approx: (22/4) * 15000 = 82k, which now, will be affordable at 27k.I guess the price slash is only for India and am not sure what the impact to the global market will look like.Quoting the article: http://www.moneycontrol.com/news/business/cipla-drug-price-cut-not-to-hurt-revenue-much-say-analysts_700380.html

Cipla cut price of its kidney cancer drug Sorafenib, which is sold under brand name Nexavar by multi-national Bayer to Rs 6,840 for a month's supply, from around Rs 28,000 earlier. Its lung cancer drug Gestinib, which is sold under brand name Iressa by AstraZeneca will cost Rs 4,250, versus Rs 10,200 earlier, and price of Temozolamide used to treat brain tumour, has been reduced by Rs 15,000 to Rs 5,000.

India's Patent Office recently issued a compulsory licence allowing Natco to make a generic copy of Sorafenib, on the payment of a royalty to Bayer, which sells the drug at around Rs 2 lakh.

Domestic sales account for 46-47% of Cipla's total sales and of that the cancer drugs portfolio is a very small portion, so these price cuts are unlikely to have any major impact on its revenue, Hitesh Mahida of Fortune Equity Brokers told moneycontrol.com

"Cipla's idea seems to be to create disruption in the market, increase its market share..." the analyst says.

Swiss pharma major Roche had earlier this year signed a manufacturing deal with India's Emcure Pharma so that its anti-cancer drugs Herceptin and MabThera could be made in India at affordable prices. Analysts say Cipla's move to slash prices could in future deter some MNCs from launching their drugs in India at all, but some may also look at doing deals like the one struck by Roche.

Meanwhile, shares of pharma major Cipla surged over 3% on Friday after brokerage CLSA upgraded the stock to "outperform" from "underperform," saying, Cipla would be strongest beneficiaries of a weakening rupee.

The rupee has been sliding sharply against the US dollar in recent days and hit over four month low of around Rs 53.78 earlier in trade.

"We expect improving margins over the coming quarters on back of a weak rupee and a low base. We expect strong operating profit growth over coming quarters led by margin expansion and high margin product supplies," CLSA's Hemant Bakhru said.

The US Food and Drugs Administration has approved Meda's drug Dymista for allergic rhinitis and the product is widely expected to reach USD 300-500 million in annual sales over the coming years. The analyst says Cipla being a partner, will benefit through product supplies over a longer term.

"Apart from approval (outside North America) related milestone payment (US$5m), we expect gradual increase in Cipla s sales from product related supplies to Meda. Assuming Cipla supplies product at 10-15% of sales, it could earn US$50-75m at peak sales," Bakhru said.

Additionally, a low base in domestic formulations could result in reasonable India growth, he adds.

Cipla shares were up 2.8% at Rs 326.60 on NSE in noon trade.

Keywords:

4 May 2012

John Goerzen: Geeky plan: Solar power supply IN THE CAR

From the might be geeky dept For an upcoming car trip, we have to take my wife s car (long story). I will be taking two large lead-acid batteries in the trunk hooked up to a solar panel in the rear window area and, of course, some electronics in the front seat. Why, you ask? While she drives, I will be working via tethered 3G (conveniently reducing the number of vacation days the trip requires). This is Kansas, so signals are weak sometimes. I have an external antenna that hooks up to a signal booster. So I ll be running a laptop, cellphone in draw lots of power mode , and a cellphone booster at minimum. Measurements suggest that this combination could easily put me in range of the typical 10A limit of dash electronics. (And doesn t even include the nice to haves of GPS, my handheld radio, and the amplifier for it.) My car has a direct high-current link from the battery to the trunk to support the 100W amateur radio mounted there. It is pretty easy to tap into it and add a cable that runs to the passenger seat to run all this stuff. Terah s car doesn t. So, I thought, what s the easy solution? Well, my solar power system, of course! If I charge up a couple of the large batteries beforehand (just pop them and a few panels in the sun for a day), then put the batteries in the trunk, it will probably power my electronics for a whole day (enough for the trip we have planned), and I have already cabled them up to support the kind of current draw I ll have. I ll have to recharge the batteries at the destination, of course, so I ll bring along a panel (no need to bring more than one since there will be plenty of time to charge). Once I m planning to take a panel along, I figured I might as well put it under the rear window so it can actually help charge the batteries as we go. The current drain will, of course, be larger than the 1A or so it can put out, but hey, who else drives down the interstate with an active solar panel in their car? I can actually split the load between the batteries and the vehicle system too, so I do not need a continuous 10A draw on the batteries. The sad part about this is I will probably never surpass a temporary battery system with a solar panel in terms of geeky travel.

3 May 2012

Gintautas Miliauskas: Online Haskell typechecker

Last weekend I put up haskellonline.org, which is basically a thin web frontend over ghc, the Haskell compiler. It is limited to one Haskell module, but it makes the experience of studying Haskell much more pleasurable by running the typechecker in the background as you type. Plus, it is web-based - no local installation necessary!

The thing about Haskell is that it has a fabulous type system which can weed out a broad class of bugs at compile time. However, the usual interfaces with the compiler leave a lot to be desired. Vim + ghci in a terminal of course works, but I have already been spoilt by IDEs, and the "Alt-Tab; Up; Enter; wait; Alt-Tab + jump to line manually" dance does not appeal to me at all. Emacs integration might be better, but I do not know emacs well, and it is never a good idea to start learning two things at the same time.

Note: I suspect that many of the problems I encountered were due to the fact that I use MacOSX.

I have looked a few of the existing IDEs, but that did not go well. EclipseFP did not offer any benefits over a text editor at all. I found a blog post saying that implied that it would work better with a newer version of GHC. Cue a few hours of reinstalling GHC and compiling Haskell packages, and I got the system set up, but that made the situation even worse. Eclipse would simply crash on startup every time until I wiped the configuration directory clean. So much for EclipseFP.

I also tried Leksah. At least it did not crash, nevertheless, I found it quite unwieldy, especially for learning purposes. The autocompletion is broken in annoying ways. For example, if you type "f a = a<enter>", Leksah will autocomplete the second "a" into "as". It took me a while to discover an error in my program that was due to this autocorrection mechanism.

Leksah does have a "compile in background" mode, which was admittedly a source of inspiration for haskellonline.org, but the implementation is subpar. It simply invokes the compiler and simply redirects the output to a display pane on the window. As you write code, the text in the pane constantly scrolls, which is very distracting. The implementation is also buggy. Occasionally errors are interleaved with status output:


There is also tryhaskell.org, but it does not work not nearly as well for Haskell as its cousins trypython.org and tryruby.org do for Python and Ruby, respectively. All are limited to evaluating expressions, but in Python/Ruby that actually gets you a long way, while nontrivial Haskell programs are complex static structures that are unwieldy without a persistent text editor.

Despite all the trouble, which, again, probably has to do with my choice of MacOSX, these issues do not detract from the beauty of Haskell as a language. If you are interested, I can heartily recommend learnyouahaskell.com as very good reading material, and Tony Morris's course exercises (in combination with haskellonline.org) for practice.

2 May 2012

Uwe Hermann: sigrok - cross-platform, open-source logic analyzer software with protocol decoder support

sigrok logo I'm happy to finally announce an open-source (GNU GPL), cross-platform (Linux, Mac OS X, FreeBSD, Windows, ...) logic analyzer software package myself and Bert Vermeulen have been working on for quite a long time now: sigrok (it groks your signals). History I originally started working on an open-source logic analyzer software named "flosslogic" in 2010, because I grew tired of almost all devices having a proprietary and Windows-only software, often with limited features, limited input/output file formats, limited usability, limited protocol decoder support, and so on. Thus, the goal was to write a portable, GPL'd, software that can talk to many different logic analyzers via modules/plugins, supports many input/output formats, and many different protocol decoders. The advantage being, that every time we add a new driver for another logic analyzer it automatically supports all the input/output formats we already have, you can use all the protocol decoders we already wrote, etc. It also works the other way around: If someone writes a new protocol decoder or file format driver, it can automatically be used with any of the supported logic analyzers out of the box. Turns out Bert Vermeulen had been working on a similar software for a while too (due to exactly the same reasons, crappy Windows software, etc.) so it was only logical that we joined forces and worked on this together. We kept Bert's name for the software package ("sigrok"), set up a SourceForge project, mailing lists, IRC channel, wiki, etc. and started working. Overview, Features You can get the lastest sigrok source code from our main git repository:
  $ git clone git://sigrok.git.sourceforge.net/gitroot/sigrok/sigrok
Here's a short overview of sigrok and its features as of today. The software consists of the following components: We're happy to hear about other (maybe special-purpose) frontends you may want to write using libsigrok/libsigrokdecode as helper libs! Firmware Saleae Logic Some logic analyzer devices require firmware to be uploaded before they can be used. As always, firmware is a bit of a pain, but here's what we currently do: For non-free firmware we provide instructions how to extract it from the vendor software or from USB dumps, if possible. For distributable firmware we have a git repo where you can get it (thanks ASIX for allowing us to distribute the ASIX Sigma/Sigma2 firmware files!).
  $ git clone git://sigrok.git.sourceforge.net/gitroot/sigrok/sigrok-firmwares
Finally, for all Cypress FX2 based logic analyzers we have an open-source (GNU GPL) firmware named fx2lafw, started by myself, but most work (and finishing the firmware) was then done by Joel Holdsworth, thanks! The support list includes Saleae Logic, CWAV USBee SX, CWAV USBee AX, Robomotic Minilogic/BugLogic3, Braintechnology USB-LPS, and many others. Get the code from the fw2lafw git repository:
  $ git clone git://sigrok.git.sourceforge.net/gitroot/sigrok/fx2lafw
Example dumps We collect various captured logic analyzer signals / protocol dumps in the sigrok-dumps git repository:
  $ git clone git://sigrok.git.sourceforge.net/gitroot/sigrok/sigrok-dumps
They can be useful for testing the sigrok command-line application, the sigrok GUIs, or the protocol decoders. We're happy to include further contributed example data in our repository, please send us .sr files of any interesting data/protocol you may come across (even if sigrok doesn't yet have a protocol decoder for that protocol). See the Example dumps wiki page for details. Packages, distros, installers sigrok Windows installer I'm currently working on updated Debian packages for sigrok (will be apt-get install sigrok to get everything), and we're happy about further packaging efforts for other distros. We have preliminary Windows installer files (using NSIS), but the Windows code needs some more fixes and portability improvements before it's really usable. On Mac OS X you can use fink/Macports to install as usual, fancier .app installer files are being worked on. Future Apart from support for more logic analyzers, input/output formats, and protocol decoders, we have a number of other plans for the next few releases. This includes support for analog data, i.e. support for (USB) oscilloscopes, multimeters, spectrum analyzers, and such stuff. This will also require additional GUI support (which could take a while). Also, we want to improve/fix the Windows support, and test/port sigrok to other architectures we come across. Performance improvements for the protocol decoding as well as more features there are also planned. Contact Feel free to contact us on the sigrok-devel mailing list, or in the IRC channel #sigrok on Freenode. There's also an identi.ca group for sigrok. We're always happy about feedback, bug reports, suggestions for improving sigrok, and patches of course!

Matthew Garrett: Anatomy of a Fedora 17 ISO image

My spare time has been limited lately, so progress on producing Mac-bootable Fedora install images has been a bit slow. Thankfully it looks like everything's gong to land in time for Fedora 17[1] so hurrah for that - all that's missing right now are the last couple of patches that make sure the boot picker displays useful labels on the drives.

So how is this accomplished? Here's a hex dump of the first few K of the image.
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER.............. 
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000020  33 ed fa 8e d5 bc 00 7c  fb fc 66 31 db 66 31 c9   3...... ..f1.f1. 
00000030  66 53 66 51 06 57 8e dd  8e c5 52 be 00 7c bf 00   fSfQ.W....R.. .. 
00000040  06 b9 00 01 f3 a5 ea 4b  06 00 00 52 b4 41 bb aa   .......K...R.A.. 
00000050  55 31 c9 30 f6 f9 cd 13  72 16 81 fb 55 aa 75 10   U1.0....r...U.u. 
00000060  83 e1 01 74 0b 66 c7 06  f1 06 b4 42 eb 15 eb 00   ...t.f.....B.... 
00000070  5a 51 b4 08 cd 13 83 e1  3f 5b 51 0f b6 c6 40 50   ZQ......?[Q...@P 
00000080  f7 e1 53 52 50 bb 00 7c  b9 04 00 66 a1 b0 07 e8   ..SRP.. ...f.... 
00000090  44 00 0f 82 80 00 66 40  80 c7 02 e2 f2 66 81 3e   D.....f@.....f.> 
000000a0  40 7c fb c0 78 70 75 09  fa bc ec 7b ea 44 7c 00   @ ..xpu.... .D . 
000000b0  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e   ....isolinux.bin 
000000c0  20 6d 69 73 73 69 6e 67  20 6f 72 20 63 6f 72 72    missing or corr 
000000d0  75 70 74 2e 0d 0a 66 60  66 31 d2 66 03 06 f8 7b   upt...f f1.f... 
000000e0  66 13 16 fc 7b 66 52 66  50 06 53 6a 01 6a 10 89   f... fRfP.Sj.j.. 
000000f0  e6 66 f7 36 e8 7b c0 e4  06 88 e1 88 c5 92 f6 36   .f.6. .........6 
00000100  ee 7b 88 c6 08 e1 41 b8  01 02 8a 16 f2 7b cd 13   . ....A...... .. 
00000110  8d 64 10 66 61 c3 e8 1e  00 4f 70 65 72 61 74 69   .d.fa....Operati 
00000120  6e 67 20 73 79 73 74 65  6d 20 6c 6f 61 64 20 65   ng system load e 
00000130  72 72 6f 72 2e 0d 0a 5e  ac b4 0e 8a 3e 62 04 b3   rror...^....>b.. 
00000140  07 cd 10 3c 0a 75 f1 cd  18 f4 eb fd 00 00 00 00   ...<.u.......... 
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
This is the boot sector. One of the fun things about ISO9660 is that it doesn't have a boot sector in the usual x86 sense - firmwares are expected to read enough of the filesystem that they can find the bootloader in it and run it directly. x86 isn't really smart enough to manage that, so it uses something called El Torito[2]. In any case, this is usually empty space on an x86 CD. But with an increasing number of systems shipping without optical drives, real CD installs are decreasing in popularity. We use a piece of software called isohybrid that allows the building of ISO9660 images that can be written directly onto a USB stick. Asking a machine to boot off them will execute the boot sector located here, which then loads a real bootloader that's capable of reading ISO and booting the OS. It'll just be ignored when it's on a real CD.

There's one difference here, though. The isohybrid bootsector has been modified so that the first 32 bytes are just noops, and this has been inserted:
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER.............. 
45 52 ("ER") indicates the presence of an Apple partition map. This will become important later. 0800 indicates that it's using 2048-byte sectors. The 9090 is a lie to convince the firmware that it's not a zero-sized disk without representing dangerous code. We need an Apple partition map because some older Macs won't boot off CD unless the CD has one. But it's sitting right at the start of the boot sector, and this code is going to be executed. Thankfully, the entire Apple header decodes to
00000000  45                inc bp
00000001  52                push dx
00000002  0800              or [bx+si],al
00000004  0000              add [bx+si],al
00000006  90                nop
00000007  90                nop
00000008  0000              add [bx+si],al
0000000A  0000              add [bx+si],al
0000000C  0000              add [bx+si],al
0000000E  0000              add [bx+si],al
00000010  0000              add [bx+si],al
00000012  0000              add [bx+si],al
00000014  0000              add [bx+si],al
00000016  0000              add [bx+si],al
00000018  0000              add [bx+si],al
0000001A  0000              add [bx+si],al
0000001C  0000              add [bx+si],al
0000001E  0000              add [bx+si],al
which is completely harmless - we can execute all these instructions without any interesting changes in state. They'll all be undone by the rest of the boot sector.
000001b0  14 05 00 00 00 00 00 00  4c 1e ed 76 00 00 80 00   ........L..v.... 
000001c0  01 00 00 3f a0 89 00 00  00 00 00 50 14 00 00 fe   ...?.......P.... 
000001d0  ff ff ef fe ff ff a4 00  00 00 70 04 00 00 00 fe   ..........p..... 
000001e0  ff ff 00 fe ff ff 44 05  00 00 c0 08 00 00 00 00   ......D......... 
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa   ..............U. 
This is the MBR partition table. The aim here was to produce media that would boot via BIOS just as well as it does via EFI, and it turns out that there are some systems that refuse to BIOS-boot off GPT media. So we need an MBR partition map. The first entry covers the entire disk and is of type 0. This is important, because some EFI implementations are strict about MBR parsing - if there's an MBR with overlapping partitions, the entire MBR will be ignored. Inconvenient. Fortunately, we've got the source code to this code, and it turns out that partitions of type 0 are ignored when performing this check. So, there's a partition of type 0.

Why does it cover the entire disk? Once we've booted the OS, we need to be able to read the ISO image contained on the USB stick. That means the kernel needs to be able to mount it, which means we need a partition entry that covers the entire disk. Linux is perfectly happy mounting a filesystem from a partition of type 0.

So, the next partition. This is an EFI boot partition that points at the embedded VFAT El-Torito image. Some firmware will decide that we've got an MBR and so will only boot off a partition that exists in it. So, here's an MBR partition to keep them happy.

The final partition covers the embedded HFS+ image. It's there purely for convenience - it means we can access it under Linux in order to update its contents for testing purposes.
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00   EFI PART....\... 
00000210  09 27 93 7e 00 00 00 00  01 00 00 00 00 00 00 00   .'.~............ 
00000220  fe 4f 14 00 00 00 00 00  30 00 00 00 00 00 00 00   .O......0....... 
00000230  de 4f 14 00 00 00 00 00  2a 26 0f 37 23 c9 7f 4f   .O......*&.7#..O 
00000240  90 df 5e fe 0a 60 1c b0  10 00 00 00 00 00 00 00   ..^.. .......... 
00000250  80 00 00 00 80 00 00 00  ec 70 63 c1 00 00 00 00   .........pc..... 
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
And here we have the GPT header. Nothing complicated here, other than it indicates that the first partition descriptor is at LBA 0x10, ie byte 0x2000 - that's further into the disk than normal. That leaves us enough space to fit the Apple partition entries.

Why have a GPT at all? The EFI spec says that machines should boot fine from an MBR partition. Sadly, not all seem to. It also lets us represent the HFS+ partition in a way that makes the Mac firmware happy.
00000800  50 4d 00 00 00 00 00 03  00 00 00 01 00 00 00 10   PM.............. 
00000810  41 70 70 6c 65 00 00 00  00 00 00 00 00 00 00 00   Apple........... 
00000820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000830  41 70 70 6c 65 5f 70 61  72 74 69 74 69 6f 6e 5f   Apple_partition_ 
00000840  6d 61 70 00 00 00 00 00  00 00 00 00 00 00 00 00   map............. 
00000850  00 00 00 00 00 00 00 0a  00 00 00 03 00 00 00 00   ................ 
00000860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
2K into the disk, here's an Apple partition map. It's 2K into the disk because we set the sector size to 2K in the partition map header. Why did we do that? Because the default is 512 bytes, and if we'd put the partition map at 512 bytes it would have been on top of the GPT header. Why 2K rather than 1K? A couple of reasons. First, the Apple partition map is only used when we boot off an actual CD, and the CD sector size is genuinely 2K. Secondly, because CDs have a sector size of 2K, 2K is a value that's actually been tested in the real world. It's nice to avoid tempting fate.
00001000  50 4d 00 00 00 00 00 03  00 00 00 29 00 00 04 70   PM.........)...p 
00001010  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001030  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001050  00 00 00 00 00 00 04 70  00 00 00 33 00 00 00 00   .......p...3.... 
00001060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00001800  50 4d 00 00 00 00 00 03  00 00 01 51 00 00 08 c0   PM.........Q.... 
00001810  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001830  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001840  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001850  00 00 00 00 00 00 08 c0  00 00 00 33 00 00 00 00   ...........3.... 
00001860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
The Apple partition map entries. One partition covers the entire disk, the other the embedded HFS+ filesystem. Why do this? Because not all Macs understand EFI El Torito, so won't EFI boot a CD unless there's an Apple partition map. Why have an HFS+ partition at all? Because the same Macs won't boot off FAT.
00002000  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002010  d0 84 a5 d4 aa 89 e0 40  81 f9 fc e4 04 e9 c2 99   .......@........ 
00002020  00 00 00 00 00 00 00 00  10 4b 14 00 00 00 00 00   .........K...... 
00002030  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002040  64 20 49 53 4f 00 49 53  4f 48 79 62 72 69 64 00   d ISO.ISOHybrid. 
00002050  41 70 70 6c 00 00 00 00  00 00 00 00 00 00 00 00   Appl............ 
00002060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002080  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002090  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
000020a0  a4 00 00 00 00 00 00 00  13 05 00 00 00 00 00 00   ................ 
000020b0  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
000020c0  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
000020d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002100  00 53 46 48 00 00 aa 11  aa 11 00 30 65 43 ec ac   .SFH.......0eC.. 
00002110  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
00002120  44 05 00 00 00 00 00 00  03 0e 00 00 00 00 00 00   D............... 
00002130  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002140  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
00002150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
Three GPT entries - one covers the entire CD, one covers the embedded FAT filesystem, one covers the embedded EFI filesystem. That ensures that they're all accessible to the firmware.
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20   .CD001..LINUX    
00008010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00008020  20 20 20 20 20 20 20 20  46 65 64 6f 72 61 2d 4c           Fedora-L 
00008030  69 76 65 43 44 20 20 20  20 20 20 20 20 20 20 20   iveCD            
00008040  20 20 20 20 20 20 20 20  00 00 00 00 00 00 00 00           ........ 
00008050  c4 12 05 00 00 05 12 c4  00 00 00 00 00 00 00 00   ................ 
00008060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00008070  00 00 00 00 00 00 00 00  01 00 00 01 01 00 00 01   ................ 
00008080  00 08 08 00 40 00 00 00  00 00 00 40 15 00 00 00   ....@......@.... 
00008090  00 00 00 00 00 00 00 17  00 00 00 00 22 00 1d 00   ............"... 
000080a0  00 00 00 00 00 1d 00 08  00 00 00 00 08 00 70 05   ..............p. 
000080b0  01 09 30 36 f0 02 00 00  01 00 00 01 01 00 20 20   ..06..........   
000080c0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
The ISO9660 superblock. This is completely standard. From here on there's nothing terribly surprising - the CD filesystem itself is entirely normal. The only slight oddity is that we have three embedded El Torito images. The first of these is a BIOS-bootable disk image. That'll be executed whenever the CD is put in a non-EFI machine. The second is a VFAT EFI boot image. That's used for the majority of EFI systems. The third is an HFS+ EFI boot image. There's no strict requirement for the HFS+ one to be an El Torito image - CD booting on Macs is already handled by the Apple partition map. Storing it as an El Torito is purely for convenience, since it guarantees correct alignment and avoids us having to parse the entire ISO9660 image to find its location when generating the partition maps.

In summary: Three partition maps, three bootable images, support for BIOS, UEFI and Mac platforms, works whether it's burned to a CD or written directly to a USB stick. I'm pretty pleased with that.

[1] Other than an awkward bug where something goes horribly wrong during Radeon graphics init, so Radeon-based Macs aren't working too well at the moment. We're working on it.

[2] The world decided that writing a real ISO9660 driver for BIOS was hard, so came up with El Torito. El Torito is a spec for embedding disk images inside ISO9660. There's a pointer to the disk image in the Cd header, so the BIOS simply reads a few blocks off CD, finds that pointer and then sets up a bunch of disk i/o interrupts to point at the embedded filesystem rather than a real disk. From that point on, the El Torito image simply behaves like a floppy or hard drive. This was a reasonable compromise given how resource limited older BIOS implementations were.

For reasons that aren't entirely clear, UEFI doesn't mandate that implementations support ISO9660. So even though our firmware is now sufficiently capable that the only thing standing between it and emacs is someone being sufficiently bored, we still use El Torito. Fortunately the spec allows multiple El Torito images to be embedded, so we can include one that's bootable by BIOS systems and another that's bootable by UEFI systems.

comment count unavailable comments

30 April 2012

Cyril Brulebois: Debian XSF News #11

Long time no see Summary Anyway, here are some quick news from the X packaging front, focussing on the past few weeks.
  1. While I was busy doing other things, Julien took care of uploading lots of libraries (making most of them multiarch-aware) along with our usual meta-packages (like x11-apps, x11-utils, etc.), preparing for the upcoming 7.7 katamari.
  2. Mesa 8.0.2 was finally uploaded to experimental, where it seems to be building fine, and working fine, at least for me.
  3. Accordingly, I m planning an upload to unstable very soon, and I won t be disabling wayland support this time, since wayland/weston reached their first milestone with a public release (0.85). There s nothing very interesting to see here, but since wayland support doesn t really hurt, I thought I d just keep it. That s why wayland hit unstable yesterday; weston should follow after mesa. (In case somebody is in a hurry, they have been in experimental for quite some time already.)
  4. Scrolling issues with the synaptics driver seem to be finally fixed thanks to rc4: #665004 was confirmed as fixed by many reporters (thanks everyone for the quick feedback after my ping). Accordingly, I ve pinged the release team to get an age-days 3 to make it migrate faster, and a kind RM added that hint to his file, so testing is getting the fixed package thanks to this midnight s britney run.
  5. libcairo2/server/EXA/ati/nouveau fun: will be added through an edition of this post. Executive summary: downgrade libcairo2 to testing s version if you re seeing text corruptions.
Upcoming X server transition X server 1.12 has been prepared in experimental for a while, from release candidates to the first stable release from server-1.12-branch. Since a bunch of drivers were uploaded to cope with that version when it shows up, we should be able to upload xorg-server 1.12 to unstable VERY SOON. What does that mean? Basically, that s a transition. In details:
  1. We upload xorg-server, and wait for it to be built everywhere, then we trigger binNMUs for all architectures at once when it s ready on all of them.
  2. Unlike shared libraries, there s no old and new library packages (old+new SONAME); instead, the server itself provides different packages (because of the different input and video ABI). That means that drivers won t be installable until they are rebuilt against the new server. Usually, we re talking about a few dinstall runs, meaning less than a day.
Frequently Yelled Phrases:
  1. Upgrades are broken! No, it just means you can t upgrade the server alone, you need the drivers too (see above).
  2. Installations are broken! Well, that s unstable, and you probably know installability-related issues are trivially solved by pulling packages from testing when needed. This is such a case.
Please don t report bugs on this topic, thanks already, and enjoy that new server.

John Goerzen: Review: Amateur Radio Handhelds (HTs), APRS, and Battery Tuning Tips

There are a lot of opinions about handhelds for amateur radio users (these handhelds are often known as HTs). I ve used three of them, two with APRS capability, and thought it s about time I write about them. First, I ll talk about the units in particular, then I ll get to some broader Yaesu vs. Kenwood generalizations and conclusions. Yaesu VX-7R a rugged first HT without APRS Cost: $350 as of 4/30/2012 This was the first HT I got as a ham, at the suggestion of Mike AE0MW. It truly does make a great first handheld for a ham. Radio-wise, one unique feature is a receive that s wideband enough to cover both broadcast FM and broadcast AM, the various ham bands, and plenty of others besides. The battery life quoted in the manual is 6 hours on 2m and 5.5 hours on 440 (using standard 6s TX / 6s RX / 48s squelched metric), and in my experience it seems it must be even better in some cases. The real piece of news about this unit, though, is just how rugged it is. It says submersible across the front and I absolutely believe it. I have personal experience with its ruggedness. I once was at an event and put the radio on my car roof while helping one of my children into the car. I then started driving home and forgot it there. When we got home, I realized what had happened, and also realized what that mysterious CLUNK along the busy county road was. So I drove 45 minutes back to where I heard the clunk, and found the VX-7R still lying in the roadway, where it had been for 1.5 hours on the 55 MPH road. The thing was really beat up. It had obviously been hit by numerous vehicles. The antenna was ripped off, taking the SMA mount with it. But wouldn t you know it, the darn thing still powered up. I took it to a local amateur radio shop, and about $80 later it was repaired. Incredible. The only real problem was the antenna connector. I took this radio with me on my bicycle many days, mounted on my handlebars. It got bounced around, but still worked great. Some days I listened to FM radio from it while riding my bike, monitoring a local repeater while I rode. Other days, I spent most of my ride (about an hour each way) chatting with people on the repeaters I have a mobile antenna mounted on the bike, fed to the VX-7R, and a headset. It all worked fine never a problem at all. The ruggedness has its downside. All of the connectors on the radio, except for DC in, are screw-down even the speaker/mic connector. This obviously helps keep water out, but means that it can take a lot longer than normal to do a simple thing like plug in a speaker/mic. It doesn t need any tools, but can get a little annoying. The radio physically is small (unless you install the belt clip, which has an odd peg-based mechanism that protrudes unnecessarily far and is surprisingly non-rugged). It can easily fit into a pocket. The other downsides of the VX-7R mostly surround its interface. I m not one of the Yaesu haters that seem to be so common. I find the interface usable if you read the manual; it seems a lot of people that complain about it don t. But it is quirky. The SET mode is one long menu in sort of a ring. You scroll with the knob, and when you get to the end, it repeats to the beginning. There are one or two shortcut keys to a specific setting, but overall either you use the thing so much you know exactly where you re going, or you re twisting the knob for awhile until the option you want rolls around. (I know of nobody that uses it so much they know exactly where they re going.) They do have options grouped into categories, but it doesn t help much because there s no quick way to skip between categories. The dual transceivers work less elegantly than on the TH-D72A or VX-8GR. Instead of being two equal transceivers A and B, they are main and sub . The sub transceiver is not broadband receive like the main receiver is. But there are also other limitations that have no apparent logic. For instance, you can program memory groups into the device putting, say, all the local repeaters into a certain group, or all the public safety frequencies (to use as a scanner). You can go to the special set menu, select group, then activate SCAN, and it will scan only the memories in that group. But you can t select the special group mode on the sub transceiver for no apparent reason. One other complaint is that the AC charger introduces so much noise to a transmission that you cannot really use the VX-7R to transmit while the charger is connected. Yaesu sells a programming cable that plugs into the speaker/mic port on one end, and a DB-9 serial port on the other, and includes programming software. For an incredibly high price. For far less, I found a similar cable on eBay that has a USB port on the other end (with a built-in USB/serial converter), and used VX-7R Commander though now that Chirp is available for Linux, I ll probably switch to it. I would probably continue to recommend this transceiver for a new ham. The ruggedness, plus the broadband RX, are features that should make it appealing. It is an excellent emergency preparedness/response radio due to those properties; having TX and RX on ham bands, plus RX on broadcast and public safety could be quite the asset. However, for someone that has any interest in APRS, one of the other two radios I mention is probably a better choice. At only $10 more, the VX-8GR sacrifices ruggedness and broadband receive for APRS functionality and could make a compelling alternative to the VX-7R at the same price point. The very low-power transmit at 50MHz and 220MHz on the VX-7R strikes me as a gimmick feature at best that has very little actual use. Shipped battery: FNB-80LI, 7.4V, 1400 mAh, lithium-ion Kenwood TH-D72A / D72E The Cadillac HT, with APRS and GPS Cost: $464 as of April 30, 2012 It s hard to use any word but incredible to describe this HT, though it does have its own rough edges. Starting with the basics, it has dual transceivers with full duplex (meaning you can be transmitting on one band while receiving on another). Its stored memory system and memory bank scanning system are both a lot easier and faster to use than Yaesu s, and work the same on both transceivers A and B, though perhaps offer slightly less flexibility than Yaesu s (on the VX-7R, a single memory can be a member of more than one group). The quoted operating time is 6 hours, which is probably accurate with APRS and GPS off, matching (or nearly matching) the VX-7R. The TH-D72A has the usual mic and speaker ports. Unlike Yaesu, Kenwood has used the same jack for years, so it is not necessary to purchase new accessories with each upgrade. The unit also has both serial (Kenwood connector) and standard mini USB ports, and even ships with a USB cable. The USB port is designed to interface to a PC, and the radio presents itself as a serial-over-USB device. Kenwood also includes MCP-4A, the programming software for the unit, free. This is one example of several things that Kenwood includes for free but are costly add-ons with Yaesu. Kenwood s configuration menu system is hierarchical. You can scroll through the four top-level menus (RADIO, GPS, APRS, and SKY), then navigate through sub menus. The result is that it s both faster and easier to find what you want than on the Yaesu menus. Plus, as you navigate, the radio shows you the shortcut key to get to a particular option (each level of menu is numbered starting) so, for instance, Menu-1-1-0 takes me directly to the configuration for RX battery saver interval far faster than I could get there on the VX-7R or VX-8GR. Only the APRS menu gets a bit crowded with this setup. The D72A supports Kenwood s Sky Command II an intriguing way of making it essentially a remote for a Kenwood HF rig such as the TS-2000. It works by engaging a crossband repeater for the audio, and setting up a AX.25 packet link for the control. It works, but as the response time to frequency adjustments is measured in seconds, is klunky for tuning around a HF band. A nifty, though perhaps not supremely useful, feature with some hackable potential that I haven t explored yet. The TH-D72A is one of only two shipping general-purpose amateur radios that have a built-in GPS the other is the Yaesu VX-8GR, which I ll review below. The GPS works well, and can get a fix indoors if desired. It has a system of saving waypoints, which you can then navigate back to later (using distance/bearing indicators). It also has a track log. While the track log can t be viewed on the unit, it can be downloaded with the included MCP-4A software and saved in a variety of formats including Google s KML for later importing into Google Maps or Google Earth. I enjoy this feature on trips to see where I went later on. There is also a GPS-only mode, which disables all of the other radio circuitry on the radio for very low power consumption. The GPS navigation screens all support heading-up or north-up indicators. But where this radio really shines is APRS. The D72 has all the basics you d expect sending and receiving beacons, sending and receiving messages and bulletins, etc. It has built-in support for being a digipeater in many different ways, including new-N UI-Trace. Coupled with an MFJ VHF amplifier, this could make a nice temporary (or even permanent) digi. The message interface permits sending, receiving, and shortcuts for replying to messages. Beacon transmission can be completely manual, at set intervals, or using SmartBeaconing. Outbound beacons also support proportional pathing (meaning adjusting via-path to not have wide coverage with every transmission). Also, if not using SmartBeaconing, a decay algorithm can be used to send beacons less frequently when the station s location isn t changing. An alternative tells the system to transmit a beacon every time PTT is released, subject to rate limiting by interval. So lots of choices there. For incoming beacons, there are various filters by callsign pattern, packet type, etc. that control whether they will be processed at all, and what kind of alert (visual or audible) to present. Beacons are also placed in a received beacon list, sorted with the most recent at the top (duplicates from the same station are always removed, so only the most recent is presented.) A long press of the LIST button shows time received and type of device next to the callsign. Selecting a specific beacon shows status text, comment, device type, bearing/distance (with graphical north-up or heading-up display), telemetry (weather information and the like), and a little bit of info about the via-path. Pretty nice to see weather reports from local weather stations on there. You can sort the list by received time, station call, or distance from your position but it doesn t stay sorted by anything other than received time, and takes several seconds to re-sort. Seems like a bit of an oversight there. The D72A supports QSY information in beacons. This can automatically insert the frequency of the other band on the transceiver into your transmitted beacons for you, or give you a quick way to tune to the frequency mentioned on others beacons. Either way, a nice touch. The D72 also supports query packets, such as packets asking your station to transmit its position right now. The TH-D72A can directly interface with several types of weather stations via serial link to read weather sensor information and transmit it into APRS packets directly, with no need for a PC to be in the picture at all. But if you do hook up a PC to it, there are quite a few more possibilities. You can hook it up to PC APRS software (or a hardware device such as the AVMAP) to see locations of other APRS stations on a map. It can emit APRS packets, plus even GPS sentences, down the USB port to the PC. Moreover, in PACKET mode, it acts as a full TNC, with a fairly robust TNC command set. In short, it can do more than APRS; it can also do regular AX.25 such as DX clusters, BBS, nodes, etc. As far as I know, this is the only HT that can interface with a PC in this manner, and only Kenwood s mobile D710 has the same kind of feature set at that. Kenwood includes a lot of documentation with the TH-D72A. The 51-page printed manual is a summary or introductory guide. The included CD-ROM supplies another 75 pages of detailed reference material, and the 92-page in-depth APRS manual has a level of detail that true geeks like me appreciate. I ve referred to Kenwood s documentation more than once while figuring out things about the VX-8GR (which woefully under-documents things like SmartBeaconing). That said, it appears that at least some of the documentation has been copied from the earlier TH-D7A. Page 4 of the printed book is a good example. It talks about the memory effect of the battery, and warns against unplugging and replugging the charger because the charge cycle will be reset and the battery will be over-charged (I find it hard to believe that a Li-Ion charge controller would be that stupid, and furthermore direct observation suggests that it isn t.) Parts of the in-depth APRS guide appear to have been written for the D710, but that s a really minor nit. Unlike Yaesu, Kenwood issues periodic firmware updates to the D72A, which you can apply over USB. They have fixed bugs and added features to the unit over time. Now onto the things that aren t so good about the D72A. I ll start with the battery system, since we were already discussing it. Despite shipping with a much larger battery than the VX-8GR (1800 mAh vs. 1100 mAh), it only manages roughly equal battery life with APRS and GPS engaged. (It will probably do better as a simple voice unit, however.) This has been confirmed by numerous reports on the Internet. I will discuss this topic more below. There is very little else to fault the D72A on, feature-wise. I have a minor nit in that it is impossible to cause the keylock feature to also lock PTT. It would be nice to be able to see raw APRS packets on beacons and messages. Other than that, I can t think of a feature it really lacks. Compared to the VX-7R and VX-8GR, the TH-D72A is significantly larger physically, in every dimension. It is not uncomfortably large, and still fits in my hand fine. But its size is enough different that it feels like the design is dated and could have been more compact if Kenwood would have bothered. It s a nit, sure, but a nontrivial one. With the single exception of the sturdy metal belt clip, the D72 doesn t feel nearly as rugged as the VX-7R, and not even as rugged as the VX-8GR. The keys have a squishy feel to them, the PTT button works fine but has a cheap plastic design (as opposed to the rubberized versions on Yaesu s HTs). The manual doesn t mention it, but one Kenwood brochure mentions and IP54 weatherproofing. That means it is protected against limited ingress of dust and against water sprayed from all directions limited ingress permitted. Contrast that to the VX-7R, which is rated for 30 minutes of submersion at 3ft and has a magnesium case. It seems to be specced similarly to the VX-8GR, though the VX-8GR certainly feels a lot more solid. I have no proof, as I m not about to sacrifice my HTs for science, but I doubt that the TH-D72A would have survived over an hour on a busy road as well as the VX-7R did, and probably not even as well as the VX-8GR would. The case is plastic a stout plastic, but still plastic. Like the VX-8GR, and unlike the VX-7R, the D72A does not have broadband receive. It can receive some bands adjacent to the ham bands it supports, which includes a lot of bands of interest to people with scanners, but still can t receive broadcast FM signals like the VX-7R. Shipped battery: PB-45L, 1800 mAh, lithium-ion Yaesu VX-8GR: Cheaper APRS & GPS Cost: $360 as of 4/30/2012 One could probably call this Yaesu s closest competitor to the D72A. At about $100 cheaper, it doesn t have a featureset that matches Kenwood s, but it does have a few innovations that the D72A lacks. Starting with the basics, it lacks the VX-7R s broadband receive, with the receive roughly matching the TH-D72A s. The 8GR is a dual-transceiver HT, and it works this effectively (moreso than the VX-7R). Yaesu has a bad habit of changing connectors on radios with regularity, and charging an arm and a leg for accessories to boot. Even within the VX-8 line, the 8GR uses different accessories than the 8DR, and of course has different plugs than the 7R (though it appears that the DC power plug is the same, and with some luck you might be able to use a VX-7R speaker/mic on the 8GR, though not the reverse). The menu system on the 8GR is still similar to the 7R, though Yaesu has done several things to improve it. For one, scrolling through the menu now shows several lines of context on the screen at once, making it faster to scan. Also, there are more ways to jump into the menu at a certain location than before and even a small bit of clumsily-implemented hierarchy (there are several settings on the SmartBeaconing page, for instance). The menu key now toggles between various displays: the default frequency/status display, a GPS display, an APRS staion list, and an APRS message list. Just about everything operationally takes more keypresses than the D72A. On the D72A, it s one or two keys (a physical button or F+button) to do almost everything: toggle GPS state, change TNC/APRS state (turn on or off), etc. All of that is in the scrolling menus on the 8GR. The quickest way there, which is documented in the manual, is to press MENU until a certain screen comes up, then press and hold MENU to bring up the settings (defaulting to a certain APRS-related category), then scroll and change settings. It s a pain to turn APRS on and off, because you have to do this process for both the GPS and the APRS modem. Inexplicably, there is a hotkey to change the APRS beacon method (manual, auto, or SmartBeaconing) but not to turn APRS itself on or off. (The D72A has a hotkey to toggle the beaconing, though not to change its method; though in truth, if you do it frequently, the menu shortcuts will get you there rapidly.) Physically, the VX-8GR is closer to the size of the 7R than the size of the D72A, and notably is significantly smaller than the D72A (at least with the standard battery). As noted in the D72A review, the VX-8GR feels more rugged than the D72A, though whether that is actually true or not is hard to pin down. It is definitely less rugged than the VX-7R. The keys require more pressure than the D72A, and in this they are similar to the 7R, but feel satisfyingly solid. Unlike the other radios here, the 8GR does not have a dedicated volume knob, having only one dial on the top. To adjust volume, you hold the volume button on the left while twisting the knob. In practice, this is probably not usually very annoying, but if the radio is on your belt or otherwise not in your hand, this would be a frustration. (I think it would be particularly annoying if using it with a speaker/mic). Setting 99 will let you change the key so you don t have to hold it in, which could probably go a long way to making the situation more tolerable. One unique feature of the VX-8GR, as far as I know completely unique to it, is a vibrating alert. This would most frequently be used with APRS to provide a nearly-silent alert of incoming messages and the like. The screen of the 8G, while similar in size to both the D72A and the 7R, appears to be higher-resolution (or if not, they make better use of it). More information is packed onto the screen at once, and although the D72A does have a small font, it uses it so rarely that lots of scrolling is needed. The 8GR uses a smaller font than the D72A to show content of messages and the like, which makes a much faster experience since there is a lot less scrolling. Neither radio has a setting to change this behavior, so if your eyesight isn t good, you may prefer the D72A anyhow. The VX-8GR supports a heading-up or a north-up view of the received GPS data, but strangely it does not support this when navigating to a received beacon (north-up is the only option). Unlike the TH-D72A, you can engage keylock on the 8GR while someone s beacon is on the display. Also unlike the TH-D72A, you can lock out the PTT key on the 8GR. However, the keylock feature is just a quick press of the power button, which makes me a bit nervous. And worse, if you tell it to lock the PTT key, then it also won t transmit any automatic APRS beacons! There are a lot of things the D72A supports in the APRS area that the 8GR doesn t. Here s a summary of features missing in the 8GR compared to the D72A: voice alert (can t even do it manually since tone/CTCSS are locked out in APRS mode), quick replying to messages, TNC mode (meaning you can t use it with PC-based APRS programs), full AX.25 packet mode, full PC linking, weather station inputs, built-in digipeater mode, heading-up display, GPS track logs, QSY in beacons, quick tuning to others QSY info, responding to position queries, proportional pathing, automatic decay, beaconing on PTT release, sorting of beacon list, hotkey access to many features, and I m sure the list goes on. On the flipside, the only thing feature-wise that you get on the 8GR s APRS implementation that the D72A lacks is ability to inspect raw APRS packets. The VX-8GR has the lowest-capacity battery of the three radios covered in this post, and this is an oft-cited problem for users of the unit. Yeasu does seem to do a much better job of power management than Kenwood, though, so in full APRS and GPS mode, the battery life is roughly the same, amazingly enough. It is unlikely that the talk time on the 8GR would match either the 7R or the D72A, however. Kenwood and Yaesu both have a battery saver feature which cuts power to the RX circuit except for brief checks for signals. Yaesu completely disables this while the APRS modem is on (and STILL matches Kenwood s battery life, despite the smaller battery!) Kenwood doesn t, though recommends changing the signal-check interval from 0.2s to 0.03s in order to avoid truncating APRS packets. Yaesu offers an 1800 mAh extended-life battery which should significantly exceed Kenwood s battery life. Aftermarket vendors have 2000 mAh batteries as well. Either way, expect APRS to be a drain on batteries. More on that below. Although Yaesu includes 168 pages of documentation with the 8GR, don t expect it to match the quality and detail of Kenwood s 218 pages. There are a lot of gimmicky/useless features on the 8GR (WIRES, for one) that get extensive treatment, and there is a fair bit of repetition. I particularly single out the SmartBeaconing documentation on the 8GR as being, at best, woefully incomplete and probably also inaccurate to boot. Read the D72A manuals to figure out how that feature works on your 8GR. I bought the VX-8GR intending it to be something akin to a more versatile TinyTrak. On that score, it mostly succeeds. It may be my go-to rig for use at bicycle rides, marathons, emergencies, etc., while the D72A will probably be the one that goes with me on trips. Shipped battery: FNB-101LI, 7.4V, 1100 mAh, lithium-ion Not covered: VX-8DR poor value for the dollar Cost: $434 ($550 with GPS and GPS connector) Yaesu s top-of-the-line HT is sometimes considered to be the VX-8DR. It maintains the broadband capability of the 7R, is probably almost as rugged as the 7R, and has APRS built in, like the 8GR. It even has an option to add a Bluetooth board. What is doesn t have built-in is a GPS. You can add one on for $77 for the FGPS-2 plus $39 for the CT-136 connector which connects the GPS to the unit. But then you have a unit a lot less rugged than even the D72A (that s a small connector for the GPS), a lot less small, with the same APRS featureset as the 8GR (meaning a lot less than the D72A), which costs about $100 more than the D72A. Frankly I think this makes for a poor APRS package, and would probably only be reasonable for someone that wants other 8DR features and has a desire for some very occasionaly APRS work. Battery Life and APRS One of my first surprises with APRS HTs was the poor battery life I saw. Doing my research seemed to suggest that, contrary to popular opinion, the transmitted beacons were not the culiprit, but the receive circuits were to blame. Because it is not acceptable to cut off the first 200ms of an AX.25 packet like it is with a repeater, the RX circuit is either engaged much more frequently (Kenwood) or continuously (Yaesu), putting a lot more drain on the battery. (This has also been confirmed by multiple others that have done testing) The GPS receiver also adds drain, and on the D72A, so does the TNC, which is always active when APRS is. If you don t care about received beacons, you can dramatically raise the battery life on the D72A by setting the power saver to 1 second or greater as high as 14 hours by one report. On either unit, raising the squelch should have some effect (though not as much; but in any case, weak packets won t decode anyhow, so there s no sense in letting them in.) On the 8GR, getting the extended life 1800 mAh (or aftermarket version) could produce battery life in nearly that same region, without compromising on the RX side of things. Conclusions If you want a HT capable of APRS, seriously think of the TH-D72A before anything else. It is $100 more expensive than the 8GR, but has more than $100 worth of extra features (and included things, like PC link cable and software, that are extra on the 8GR), so it has a better value. If you re particularly budget-conscious or want the most ruggedness you can find, the 7R and 8GR are solid contenders if you can sacrifice the extra APRS features. Update May 1: Clarified what the lack of TNC mode on the VX-8GR means, and also how the VX-8GR PTT keylock also locks out APRS transmissions.

Russell Coker: The Royal Caribbean Official Android app

I ve just played with the official Android app [1] for the Royal Carribean cruise line [2]. The cruise line is apparently great (I ve never been on one of their ships but the reviews are good) but the Android app isn t. Net Access The most obvious and significant problem with this app is that it s entirely useless without net access. All data of note comes from the Internet which means that the program is useless in any location where Internet access is unavailable (or unreasonably expensive). They wrote an app about cruising that can t be used on a cruise ship! Did they even think about what they were doing? The correct thing to do when writing such an application is to have all basic data about all ships included in the app. This means that when they change the deck plan of a ship they need to release a new version of the app and have people download it. Having done a lot of software development I understand that forcing software updates (even updates to included data files) involves some effort and expense. But when they spend $20,000,000 to update a ship (which is about the minimum that is spent for a major ship in dry-dock according to TV documentaries I ve watched) it seems quite reasonable to budget $10,000 to release new software. Also one benefit of updating the software is that it can promote the changes, after spending tens of millions of dollars improving a ship they probably want to promote that to customers and pushing a new app update with adverts for the improved ship seems like a good way of doing that. There is some data that can t reasonably be included in the app due to size constraints with photos of ships being the most notable example. The solution to this is to provide an option for the user to cache the data that interests them. For example if I was meeting some people to discuss the possibility of a group cruise on a RCL ship then I could download all the pictures of that ship on my home Wifi network and then have them all available with no delay or 3G costs. Also the app seems to hang if net access is temporarily interrupted. As phones are expected to have unreliable net access this is also a significant flaw. Maps The maps of the ships are comprised of a series of pictures which each show one deck. In addition to being downloaded (not cached or included in the app) they aren t scalable (they should be SVG or at least allow zooming the high resolution pictures) and they don t allow a 3D view. The paper maps used to promote cruises (including RCI cruises) and which are given to all passengers on Princess cruises (I m not sure about other lines) show a side cutaway view of the ship which is handy for working out which things are near where you are. It seems that an ideal cruise ship mapping program would have some sort of 3D component, maybe X3D. My experience is that a two night cruise isn t long enough to become familiar with one of the smaller cruise ships. Using a map is essential and a smart phone is a good way of managing such a map as typical 2D paper maps just aren t good enough for such a large and complex structure. Photos One of the significant things that is wrong with the app is a lack of care in displaying the photos of ships. They display three pictures of the Allure of the Seas (one of the two newest, biggest, and most luxurious ships in their fleet), but one of those three photos is actually of the Oasis of the Seas. The fact that the two ships are almost identical is no excuse, there is a principle at stake! Also only having three pictures is pretty poor, there is no way that less than 50 pictures could do justice to such a big ship! A Google search for the words cruise and photos turns up many sites with pictures of cruise ships and it s not particularly difficult to find pictures of any particular ship. Photographs by customers are often of high quality as some of the better DSLR cameras are in the same price range as some of the cheaper tickets for cruise ships. Probably the best thing that RCI could do is to run a contest and allow their customers to enter photos and vote towards the winning entries. That would get them photos that aren t as sterile as the official photos and which include the things that are of most interest to customers. Finally in terms of caching, pictures are the most easily cached source of data and as phones get higher resolution they keep getting bigger. The storage space for a modern phone is equivalent to the entire 3G download quota for about a year on an affordable Australian 3G plan. When dealing with photos downloaded from the net the default should be to cache everything. Navigation Navigating a smart-phone app is a lot more difficult than navigating the same data on a desktop system (which would be in a web browser). Users can compensate for some deficiencies with web site organisation by using a large monitor and having several web browser windows with multiple tabs. But with a phone it should be possible to switch between things quickly. The main menu has a View Our Ships option (which allows viewing deck plans and pictures) and a View Our Staterooms option which offers a list of ships and then describes the state rooms available for each one. This means that you can t see all the information about a ship in one place and even worse you can t easily compare ships. As it seems likely that people will want to use this app for selecting a cruise it should be possible to select a few ships that are of interest and then quickly flip between them. For example the Rhapsody of the Seas and the Voyager of the Seas are cruising in my part of the world so it would be nice if I could tell the app to compare those ships and then allow me to view a page about one ship and then flick to the equivalent page about the other ship. Another notable problem is that the ships are listed in alphabetical order. The sensible thing to do is to list them by class going from biggest to smallest. Lessons to be Learned These problems aren t specific to the RCI app, many other Android apps have the same flaws. For example the Google Play market app doesn t cache the icons of the installed apps so every time I want to see a list of installed apps it goes slow and wastes some of my bandwidth. Doing something wrong in the same way as Google isn t necessarily a great mistake, although using the Google Play market on a cruise ship is probably very uncommon. Probably the biggest problem is a lack of testing. They should have sent the developers on a cruise as a live test. Every cruise ship has a sales desk for booking future cruises so it wouldn t be difficult to have a dozen Android phones at the sales desk to see how real customers who really want to book a cruise find it. I presume that even if net access was available then such a test would fail dismally. If a 3D display of a ship combined with all the data management capabilities of a modern smart phone (which is a lot more powerful than the desktop systems I used prior to 2000) can t at least be a useful supplement to a stack of paper brochures then it s probably a failure. I think that the RCI app is an example of how to make an Android app which doesn t fall into the more common failings (such as being a quick and dirty port from iOS) but yet still isn t useful to customers. I recommend that people who develop apps which have an objective of imparting information to users try it out as an example of what not to do. Try a few basic tasks like comparing the three biggest classes of RCI ships in terms of features, after failing to do that with the app you can then use Wikipedia to get the result. But don t use the Wikipedia client apps, use a tabbed browser such as Opera Mini. Related posts:
  1. An Introduction to Android I gave a brief introductory talk about Android at this...
  2. Choosing an Android Phone My phone contract ends in a few months, so I m...
  3. Galaxy S vs Xperia X10 and Android Network Access Galaxy S Review I ve just been given an indefinite loan...

29 April 2012

Peter Eisentraut: Setting the time zone on remote SSH hosts

The tasks: I have one or more desktop/laptop machines with varying local time zones (because the persons using them are actually in different time zones, or because the one person using them travels). I also have a number of servers configured in some random time zones. (It could be the time zone where they are physically located, or the time zone of the person who installed it, or UTC for neutrality.) Now what I would like to have happen is that if I log in using SSH from a desktop to a server, I see time on that server in my local time zone. For things like ls -l, for example. Obviously, this illusion will never be perfect. Nothing (except something very complicated) will adjust the time stamps in the syslog output, for example. But the ls -l case in particular seems to come up a lot, to check how long ago was this file modified. This should be completely doable in principle, because you can set the TZ environment variable to any time zone you like, and it will be used for things like ls -l. But how do you get the TZ setting from here to there? First, you have to make the remote SSH server accept the TZenvironment variable. At least on Debian, this is not done by default. So make a setting like this in /etc/ssh/sshd_config:
# Allow client to pass locale environment variables
AcceptEnv LANG LC_* TZ
You also need to make the equivalent setting on the client side, either in /etc/ssh/ssh_config or in ~/.ssh/config:
SendEnv LANG LC_* TZ
Which leaves the question, how do you get your local time zone into the TZ variable to pass to the remote server? The actual time zone configuration is the file /etc/localtime, which belongs to glibc. In current Debian, this is (normally) a copy of some file under /usr/share/zoneinfo/. In the distant past, it was a symlink, which would have made things easier, but now it's a copy, so you don't know where it came from. But the name of the time zone is also written to /etc/timezone, so you can use that. The format of the TZ environment variable can be found in the glibc documentation. If you skip past most of the text, you will see the relevant parts:
The third format looks like this: :CHARACTERS Each operating system interprets this format differently; in the GNU C library, CHARACTERS is the name of a file which describes the time zone.
So what you could do is set
TZ=":$(cat /etc/timezone)"
Better yet, for hopping through multiple SSH hosts in particular, make sure to preserve an already set TZ:
TZ=$ TZ:-":$(cat /etc/timezone)" 
And finally, how does one hook this into ssh? The best I could think of is a shell alias:
alias ssh='TZ=$ TZ:-":$(cat /etc/timezone)"  ssh'
Now this set up has a number of flaws, including:
  • Possibly only works between Linux (Debian?) hosts.
  • Only works if available time zone names match.
  • Only works when calling ssh from the shell.
But it practice it has turned out to be quite useful. Comments? Improvements? Better ideas? Related thoughts:
  • With this system in hand, I have resorted to setting the time zone on most servers to UTC, since I will see my local time zone automatically.
  • Important for the complete server localization illusion: some ideas on dealing with locales on remote hosts.

28 April 2012

Raphael Geissert: You are doing it wrong...

... when you join a team and then every other team member stops working.

(There's a possibility that you are doing such a great job that there's no need of the others, but think about the case when your work or your way of working is killing them.)

27 April 2012

Ana Beatriz Guerrero Lopez: Debian in the Google Summer of Code 2012

This year our efforts have paid off and despite there being more mentoring organizations than there were in 2011 (175 in 2011 and 180 in 2012), this year in Debian we got 81 submissions versus 43 submissions in 2011.
You can see here the graphs of applications against time from this year: 2012 The result is this year we ll have 15 students in Debian versus 9 students last year! Without further ado, here is the list of projects and student who will be working with us this summer: If you want to know more about these projects, follow the links and ask the students (and mentors)!

Russell Coker: BTRFS and ZFS as Layering Violations

LWN has an interesting article comparing recent developments in the Linux world to the Unix Wars that essentially killed every proprietary Unix system [1]. The article is really interesting and I recommend reading it, it s probably only available to subscribers at the moment but should be generally available in a week or so (I used my Debian access sponsored by HP to read it). A comment on that article cites my previous post about the reliability of RAID [2] and then goes on to disagree with my conclusion that using the filesystem for everything is the right thing to do. The Benefits of Layers I don t believe as strongly in the BTRFS/ZFS design as the commentator probably thinks. The current way my servers (and a huge number of other Linux systems) work of having RAID to form a reliable array of disks from a set of cheap disks for the purpose of reliability and often capacity or performance is a good thing. I have storage on top of the RAID array and can fix the RAID without bothering about the filesystem(s) and have done so in the past. I can also test the RAID array without involving any filesystem specific code. Then I have LVM running on top of the RAID array in exactly the same way that it runs on top of a single hard drive or SSD in the case of a laptop or netbook. So Linux on a laptop is much the same as Linux on a server in terms of storage once we get past the issue of whether a single disk or a RAID array is used for the LVM PV, among other things this means that the same code paths are used and I m less likely to encounter a bug when I install a new system. LVM provides multiple LVs which can be used for filesystems, swap, or anything else that uses storage. So if a filesystem gets badly corrupted I can umount it, create an LVM snapshot, and then take appropriate measures to try and fix it without interfering with other filesystems. When using layered storage I can easily add or change layers when it s appropriate. For example I have encryption on only some LVs on my laptop and netbook systems (there is no point encrypting the filesystem used for .iso files of Linux distributions) and on some servers I use RAID-0 for cached data. When using a filesystem like BTRFS or ZFS which includes subvolumes (similar in result to LVM in some cases) and internal RAID you can t separate the layers. So if something gets corrupted then you have to deal with all the complexity of BTRFS or ZFS instead of just fixing the one layer that has a problem. Update: One thing I forgot to mention when I first published this is the benefits of layering for some uncommon cases such as network devices. I can run an Ext4 filesystem over a RAID-1 array which has one device on NBD on another system. That s a bit unusual but it is apparently working well for some people. The internal RAID on ZFS and BTRFS doesn t support such things and using software RAID underneath ZFS or BTRFS loses some features. When using DRBD you might have two servers with local RAID arrays, DRBD on top of that, and then an Ext4 filesystem. As any form of RAID other than internal RAID loses reliability features for ZFS and BTRFS that means that no matter how you might implement those filesystems with DRBD it seems that you will lose somehow. It seems that neither BTRFS nor ZFS supports a disconnected RAID mode (like a Linux software RAID with a bitmap so it can resync only the parts that didn t change) so it s not possible to use BTRFS or ZFS RAID-1 with an NBD device. The only viable way of combining ZFS data integrity features with DRBD replication seems to be using a zvol for DRBD and then running Ext4 on top of that. The Benefits of Integration When RAID and the filesystem are separate things (with some added abstraction from LVM) it s difficult to optimise the filesystem for RAID performance at the best of times and impossible in many cases. When the filesystem manages RAID it can optimise it s operation to match the details of the RAID layout. I believe that in some situations ZFS will use mirroring instead of RAID-Z for small writes to reduce the load and that ZFS will combine writes into a single RAID-Z stripe (or set of contiguous RAID-Z stripes) to improve write performance. It would be possible to have a RAID driver that includes checksums for all blocks, it could then read from another device when a checksum fails and give some of the reliability features that ZFS and BTRFS offer. Then to provide all the reliability benefits of ZFS you would at least need a filesystem that stores multiple copies of the data which would of course need checksums (because the filesystem could be used on a less reliable block device) and therefore you would end up with two checksums on the same data. Note that if you want to have a RAID array with checksums on all blocks then ZFS has a volume management feature (which is well described by Mark Round) [3]. Such a zvol could be used for a block device in a virtual machine and in an ideal world it would be possible to use one as swap space. But the zvol is apparently managed with all the regular ZFS mechanisms so it s not a direct list of blocks on disk and thus can t be extracted if there is a problem with ZFS. Snapshots are an essential feature by today s standards. The ability to create lots of snapshots with low overhead is a significant feature of filesystems like BTRFS and ZFS. Now it is possible to run BTRFS or ZFS on top of a volume manager like LVM which does snapshots to cover the case of the filesystem getting corrupted. But again that would end up with two sets of overhead. The way that ZFS supports snapshots which inherit encryption keys is also interesting. Conclusion It s technically possible to implement some of the ZFS features as separate layers, such as a software RAID implementation that put checksums on all blocks. But it appears that there isn t much interest in developing such things. So while people would use it (and people are using ZFS ZVols as block devices for other filesystems as described in a comment on Mark Round s blog) it s probably not going to be implemented. Therefore we have a choice of all the complexity and features of BTRFS or ZFS, or the current RAID+LVM+Ext4 option. While the complexity of BTRFS and ZFS is a concern for me (particularly as BTRFS is new and ZFS is really complex and not well supported on Linux) it seems that there is no other option for certain types of large storage at the moment. ZFS on Linux isn t a great option for me, but for some of my clients it seems to be the only option. ZFS on Solaris would be a better option in some ways, but that s not possible when you have important Linux software that needs fast access to the storage. Related posts:
  1. Starting with BTRFS Based on my investigation of RAID reliability [1] I have...
  2. ZFS vs BTRFS on Cheap Dell Servers I previously wrote about my first experiences with BTRFS [1]....
  3. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...

26 April 2012

Wouter Verhelst: Switching to duckduckgo

In the late 90s, google became popular for one reason: because they had a no-nonsense frontpage that loaded quickly and didn't try to play with your mind. Well, at least that was my motivation for switching. The fact that they were using a revolutionary new search algorithm which changed the way you search the web had nothing to do with it, but was a nice extra. Over the years, that small hand-written frontpage has morphed into something else. A behind-the-scenes look at the page shows that it's no longer the hand-written simple form of old, but something horrible that went through a minifier (read: obfuscator). Even so, a quick check against the Internet Wayback machine shows that the size of that page has increased twenty-fold, which is a lot. But I could live with that, since at least it looked superficially similar. Recently, however, they've changed their frontpage so that search-as-you-type is enabled by default. Switching that off requires you to log in. So, you have a choice between giving up your privacy by logging in before you enter a search term, or by having everything you type, including any typos and stuff you may not have confirmed yet, be sent over to a data center god knows where. Additionally, at the first character you type, the front page switches away to the results page, causing me to go "uh?!?" as I try to find where they moved my cursor to. This is annoying. Duckduckgo doesn't do these things; and since they also don't do things like combining my typing skills, phone contact list, calendar, and chat history to figure out that I might be interested in a date, I'm a lot more comfortable using them. So a few days ago, I decided to switch my default search engine in chromium to duckduckgo. It still feels a bit weird, to be using a browser written by one search engine to search something on another; but all in all, it's been a positive experience. And the fact that wikipedia results are shown first, followed by (maybe) one ad, followed by other search results, is refreshing. We'll see how far this gets us.

Sylvain Le Gall: On the importance of memory buffer...

I have been working since a long time on trying to solve random crashes on the OCaml Forge. I have first suspected failing hardware to be the cause of it, but found no evidence about that. In despair of a solution, I have installed cacti to investigate the issue. The good news is that it has allowed me to catch a pretty nice graph of a crash. Load average - Crash Just before the crash the average load was 200... My conclusion was that too many processes were running at the same time. So I start to hunt processes that load the server for nothing. One of them was darcsweb. This process doesn't really create a load, but it calls darcs for various operations and most of them are quite expensive. The most expensive one is darcs diff. I first turn on the caching of darcsweb, which already reduce the number of invocation of darcs. But it doesn't make a real difference (except that the website was faster to load). I continued to investigate. In order to reduce globally the load of the server, I installed various robots.txt to prevent crawler to call diff for all VCS on the forge. Crawler traffic is 10 times the normal traffic and indexing diff in Google is not very interesting. It doesn't make a real difference. Load average - Peaks I installed a script to analyze all the peak load above 2. The peaks happen every hour and are more or less important (from 4 to 10). I discovered that this peak were related to the hourly cronjob of FusionForge that fixed repository permissions. I was pretty surprise because updating permissions should no generate this load. This morning, I had a simple idea. There are a lot of process on the server and one of them (the bzr daemon, loggerhead) eats a big chunk of memory (at least never frees it). I just had a quick at the "buffers" memory... Only 4MB ! I just restarted loggerhead. The improvement made on robots.txt makes it a lot more stable with regard to memory consumption, since we don't run bzr diff anymore. The "buffers" memory is now at 100MB and guess what ! No more peaks... Here is the result: Load average - Restart Conclusion: sometimes the cause of a high load lies in a single number...

Alexander Pashaliyski: Google Summer of Code 2012 Clang support for build services in Debian

This project is about extension of the Debian infrastructure with ability to build packages with clang. The key idea is to have an alternative way to build packages and easily switch between clang and gcc/g++. Clang is faster than gcc, uses less memory, and produces faster code. It is getting better with time. It is available under the BSD license. It is well documented. It can help programmers find and analyze easily their errors, because clang has much more detailed error and warning messages and it allows better diagnostics. It also provides a good alternative for programmers using gcc in the coding and compiling processes. It enforces better code quality. In the compiling process, compiler choice is usually done by setting the environment variable CC. When Makefile is generated, it contains initialization of the CC variable based on the current environment. If CC is not set, the default compiler is gcc. CC is always used by make. I proved this with tests (I set CC=/usr/bin/clang).
There must be an easy way to switch the default compiler between gcc/g++ and clang. One way would be by issuing a command like
dpkg-reconfigure default-compiler . I think that debuild, sbuild can be changed to take optional compiler argument (something like clang gcc). If this argument is not specified, the default compiler is used (according to CC)
Also I think that when the default compiler is set to clang, some special cases should be checked.
Example: There is gcc hard coded in the Makefile. In this case there should be a dialog box that offers several options to choose from:
1. Leave gcc in this line.
2. Change gcc with the CC variable.
3. Edit with the default text editor.
I have seen the 3rd option in many applications, like svn or crontab. When some file modification is required, the default editor is launched to modify the file.
Also, I ll check what are the compiling problems of the current packages that can t compile by just changing gcc with clang. This way I ll see what checks for special cases to implement.

Paul Wise: What happens when a Debian derivative shuts down?

Over the past months we have seen the end of two Debian derivatives. In January the news came that Junta de Extremadura (Spain) were abandoning the development of LinEx and switching to Debian itself. Early in March the Debian derivatives census scripts noted that the Vanillux apt repository was down. Fabrice Quenneville then confirmed that he had to put a hold on the Vanillux project due to the cost of bandwidth and servers. In addition the future of StormOS is in doubt after Illumian was created. StormOS is a port of Debian to the OpenSolaris kernel and Illumian is similar but uses only apt/dpkg and repackages everything else. The LinEx page in the Debian derivatives census did not reveal much information about the project that would have been useful to Debian, in particular it does not list any apt repositories. As a result it is quite hard to say what has potentially been lost. Two mails from people close to or involved in the project indicate that much of the LinEx distribution was already merged into Debian. It is probably safe to say that everything of value has been merged into Debian, including at least one of the developers involved in LinEx. Vanillux was a small distribution with few developers according to the Google caches of their website. If we look at the patches created by the derivatives census scripts, we can see that the 5 source packages that were possibly derived from Debian source packages were simply imported from Christian Marillat's repository of non-free, patented, legally restricted and multimedia-related packages. The patches indicate that 3 source packages were forked from Debian and that 2 source packages were done from scratch. The forked packages seem to be mainly about enabling support for proprietary and patented codecs in several programs. This is a surprisingly small number of altered/differing packages, so what else could Vanillux folks have been working on? It appears that there were 12 new source packages that were not derived from Debian source packages. These appear to be mainly multimedia-related packages, one font imported from an Ubuntu PPA, some syslinux themes and a metapackage. The multimedia packages are all from Christian Marillat's repository. The Debian multimedia team is working hard on bringing multimedia related software to Debian and welcomes help with that. The font (Cantarell) is now in Debian under a different source package name. The metapackage appears to be very similar to from the ubuntu-meta source package from Ubuntu that uses germinate. So at first glance, the contribution of Vanillux to the world of Linux distributions appears to be in the area of artwork and package selection. The artwork produced is basically Vanillux branding and is thus not usable by Debian, although we would like more artists involved in Debian. The meta-package is not easily useful to Debian since we use a different mechanism for our task packages and our task packages have already been updated for the GNOME 3 transition. Still, the amount of difference between to source packages is relatively small. So, what else? Perusing the diff between the list of source packages exposed by the Packages and Sources files, I noted that a number of binary packages in the Packages files reference source packages not listed in the Sources files. When I saw picasa in that list, it occurred to me that Vanillux might have directly imported some binary packages without their corresponding source packages. Perusing their apt metadata confirms that they have imported some binary packages of non-free software directly from vendors. These include Google Desktop, Opera, Picasa and VirtualBox 3.2. The rest of the packages in the diff appear to be caused by some sort of issue with the import process from Debian and other apt repos. Most of the above could be achieved by adding some external commerical repositories to a normal Debian system or by merging some of those repositories (such as the Opera one) into Debian. The interesting thing about the Debian derivatives census is that it allows us to perform analyses like these and figure out what patches and packages we might like to integrate into Debian. In this way we can salvage some of the value of our derivatives if they abandon ship. If you have any ideas or code for improving the census or are running a Debian derivative, please join us at the Debian derivatives frontdesk.

25 April 2012

Vincent Bernat: XBMC Eden on Debian Wheezy

I bought some HTPC a few years ago to run XBMC, a neat media center solution. At the time, to avoid any problems, I installed it on top of a minimal Ubuntu Lucid installation with the official packages from the team XBMC. Recently, XBMC Eden has been released and XBMC has landed into Debian unstable. It was a good occasion to make the switch. Unofficial XBMC logo for Eden TL;DR: Installing XBMC on Debian Wheezy is quite easy: it almost works out of the box. The big difficulty is the configuration of the remote control: either it works as you expect or you will have to scratch your head over the pile of layers needed to work with a remote control. The configuration of my HTPC is as follows:
  • an Antec Micro Fusion 350 housing featuring a SoundGraph iMON IR receiver and LCD screen1,
  • a Zotac ION-ITX-D-E motherboard with an Intel Atom 330 CPU and an nVidia ION chipset,
  • some Onkyo AV receiver connected through HDMI,
  • a Logitech Harmony 555 as remote control.

Installation

Installing Debian Wheezy Installing Debian Wheezy2 is pretty easy. Nowadays, getting a bootable USB key from a netinst image of Debian Installer for Wheezy is simplified:
$ sudo dd if=debian-testing-i386-netinst.iso \
>         of=/dev/disk/by-id/usb...
The installation was smooth with the exception of GRUB which was unable to install itself on the disk. This is a known bug when dealing with LVM and it comes with a simple workaround. I hope it will be corrected in time for Wheezy release. While this has little to do with the installation of XBMC, I wanted to test systemd which may become the default init in Debian (at least in Debian GNU/Linux). From README.Debian:
systemd can be installed alongside sysvinit and will not change the behaviour of the system out of the box. This is intentional. To test systemd, add init=/bin/systemd to the kernel command line and then rebooting, or install the systemd-sysv package.
The final system boots in about 15 seconds.

Configuring X Because video decoding in nouveau driver is still a work in progress, the use of the proprietary NVIDIA drivers is mandatory to be able to watch high resolution videos. Therefore, /etc/apt/sources.list should be completed with contrib and non-free repository. Then, you can install the appropriate packages: xserver-xorg-video-nvidia, nvidia-vdpau-driver and xserver-xorg. Here is my /etc/X11/xorg.conf.d/nvidia.conf:
Section "Device"
    Identifier     "NVidia ION"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    Option         "HWCursor" "False"
    Option         "NoFlip" "False"
    Option         "FlatPanelProperties" "Scaling = Native"
    Option         "DynamicTwinView" "False"
    Option         "ConnectedMonitor" "DFP-1"
    Option         "CustomEDID" "DFP-1:/etc/X11/edid.bin"
    Option         "NoLogo" "True"
EndSection
Section "Extensions"
    Option "Composite" "false"
EndSection
The CustomEDID option allows the driver to get an appropriate EDID even when the AV receiver is off. You can get yours, free of charge, with get-edid from read-edid package.

Installing XBMC Thanks to the work of Andr s Mej a, XBMC is now available in Debian Wheezy. To install it, just type aptitude install xbmc. I have dropped the following xbmc.service in /etc/systemd/system directory:
[Unit]
Description = XBMC media center
After = syslog.target
[Service]
User = xbmc
Group = xbmc
Type = simple
ExecStart = /usr/bin/xinit /usr/bin/xbmc-standalone -- :0
Restart = on-failure
[Install]
WantedBy = multi-user.target
Enable this service on boot with systemctl enable xbmc.service. You need to allow xbmc user to run X. The simplest way is to run dpkg-reconfigure -plow x11-common and to allow anybody to run X. sudo may be an alternative.

Configuration

Sound While I wanted to use PulseAudio, I want the AV receiver to be able to upmix stereo streams itself. With PulseAudio, it would always receive a 6-channel signal. Therefore, I directly use ALSA. First, unmute the appropriate outputs:
$ amixer scontrols   grep IEC958
Simple mixer control 'IEC958',0
Simple mixer control 'IEC958 Default PCM',0
Simple mixer control 'IEC958',1
$ amixer sset 'IEC958',0 unmute
$ amixer sset 'IEC958 Default PCM',0 unmute
$ amixer sset 'IEC958',1 unmute
$ sudo systemctl stop alsa-utils.service
The order of channels is incorrect. With the following /etc/asound.conf, we declare a new output, hdmi2, with a different mapping:
pcm.hdmi2  
  type asym
  playback.pcm  
    type plug
    slave.pcm "remap-surround51"
   
 
pcm.!remap-surround51  
  type route
  slave.pcm "hdmi"
  ttable  
    0.0= 1
    1.1= 1
    2.4= 1
    3.5= 1
    4.2= 1
    5.3= 1
   
 
In XBMC, this output should be used instead of the default one. hdmi should still be used for passthrough. To check if each speaker is mapped correctly, one can use speaker-test -D hdmi2 -c 6.

LCD display The LCD display integrated into the SoundGraph iMON is supported by the imon kernel module and the lcdproc package. I have only modified a few lines of /etc/LCDd.conf to make it work:
[server]
Driver=imonlcd
ServerScreen=off
[imonlcd]
Protocol=1
OnExit=2
Contrast=400

Remote control This is the most difficult part. I have a Logitech Harmony remote which is a great universal remote. Its support in Linux is acceptable: you can configure through Logitech website and use congruity to push the new configuration.

Remote controls and Linux Before Linux 2.6.36, most remote controls would need LIRC to work:
  • The driver receives the signal from the IR receiver and make it available through /dev/lirc.
  • lircd, with the help of a configuration file describing the protocol used by the remote control, will read the signal and turn it into the appropriate LIRC code.
  • XBMC connects to lircd and receives incoming LIRC codes. It will translate them to an XBMC command. This translation is specified in Lircmap.xml.
  • XBMC maps each command to an action (like Play, Fullscreen, ) using a keymap. This keymap can handle commands received by a remote control, but also by a keyboard, a mouse or a joystick.
Since Linux 2.6.36, remote controls will be mapped as a generic input device (just like a keyboard):
  • The driver receives the signal from the IR receiver.
  • The signal will be handled by a decoder. The configuration of this decoder is done in userland by ir-keytable. The decoder will turn the signal into the appropriate event (usually, some keypress).
  • X will listen to those events and turn them into X key events.
  • XBMC will receive them and use the appropriate keymap to turn them into actions.
And to add more complexity to the mix, in this last case, you can still use LIRC: lircd will listen to events generated by the kernel and turn them into LIRC codes. This can be very confusing. Moreover, the SoundGraph iMON IR receiver accepts two IR protocols: the iMON protocol and the RC-6 one. The Linux driver accepts both of them but uses the first one by default. The RC-6 protocol is the protocol used by many MCE remote controls. I hope you are still with me here.

The easy way To get a reasonable configuration out of the box, here is how to configure each layer:
Logitech Harmony remote
Configure it as a Microsoft branded Media Center PC: Windows Media Center SE.
iMON IR receiver
It must use RC-6 protocol. See below for more details.
LIRC
In /etc/lirc/hardware.conf, put DEVICE=/dev/input/by-id/usb-15c2_0038-event-if00 and DRIVER=devinput. In /etc/lirc/lircd.conf, just put include "/usr/share/lirc/remotes/devinput/lircd.conf.devinput".
XBMC
With the previous bits done, it should just work out of the box.
To switch to RC-6 protocol, install the ir-keytable package and use the following commands:
$ sudo modprobe rc-imon-mce
$ sudo ir-keytable -s rc0 -p rc-6 -c -w /lib/udev/rc_keymaps/imon_mce
Read imon_mce table
Old keytable cleared
Wrote 77 keycode(s) to driver
Protocols changed to RC-6
To make the change permanent, add the rc-imon-mce module to /etc/modules and create /etc/udev/rules.d/90-imon.rules with the following content:
# Override the keytable for iMON
ACTION=="add change", SUBSYSTEM=="rc", DRV_NAME="imon", \
   RUN+="/usr/bin/ir-keytable -s $name -p rc-6 -c -w /lib/udev/rc_keymaps/imon_mce"

The hard way Now, you may want to bind custom actions to some (physical or virtual) buttons. Basically, you are left with two solutions:
  1. Start from the basic configuration with LIRC and add more buttons at each levels (there are five of them!).
  2. Remove LIRC and start with the Logitech Harmony acting as a Microsoft MCE keyboard.
The first option can be quite difficult. You need to find an unused code for the Logitech Harmony. You can try to make it learn a new code if you have some RC-6 remote control. Then, you need to ensure that this code will be present in the keytable used by ir-keytable. If not, you need to add it. That s not easy since you need a to enable some debug stuff in the kernel to find the appropriate scancode. After that, the code needs to be translated in lircd.conf. You will then have to translate it again in Lircmap.xml. At least, you need to add it to a keymap in XBMC. The other way is not ideal but seems less cumbersome. The first step is to configure the Logitech Harmony as a Microsoft MCE keyboard: it has a lot of available keys. Because of the lack of multimedia keys, let s match the keyboard configuration of XBMC:
Button Command Button Command
Channel Down PageDown Stop X
Channel Up PageUp Skip back Comma
Prev Backspace Skip forward .
Up DirectionUp Play P
Down DirectionDown Rewind R
Left DirectionLeft Fast forward F
Right DirectionRight Star Delete
OK Enter Pound W
Menu C Red F1
Exit Esc Green F2
Guide Tab Yellow F3
Info I Blue F4
Unfortunately, the keytables provided with ir-keytable are not complete enough. I have built a more complete table3. With this table and the bindings described above, most functions will work out of the box without LIRC. Additional keys can be configured in a dedicated keymap4. Here is an excerpt of mine:
<keymap>
 <global>
  <keyboard>
    <end>XBMC.ShutDown()</end>
    <f1>XBMC.ActivateWindow(MusicLibrary)</f1>
    <f2>XBMC.ActivateWindow(Videos,TvShowTitles)</f2>
    <f3>XBMC.ActivateWindow(Videos,MovieTitles)</f3>
    <f4>XBMC.ActivateWindow(Weather)</f4>
  </keyboard>
 </global>
 <FullscreenVideo>
  <keyboard>
   <opensquarebracket>SubtitleDelayMinus</opensquarebracket>
   <closesquarebracket>SubtitleDelayPlus</closesquarebracket>
   <f6>xbmc.runscript(script.xbmc.subtitles)</f6>
  </keyboard>
 </FullscreenVideo>
</keymap>

FTP Instead of using SSH, I prefer to drop new files with anonymous FTP. vsftpd fits this purpose. Here is my configuration file:
listen=YES
xferlog_enable=YES
use_localtime=YES
setproctitle_enable=YES
secure_chroot_dir=/var/run/vsftpd/empty
nopriv_user=ftp
ftpd_banner=XBMC
hide_ids=YES
ftp_username=xbmc
anon_umask=022
anon_root=/home/xbmc/media
anonymous_enable=YES
write_enable=YES
anon_upload_enable=YES
anon_world_readable_only=YES
It is currently not compatible with systemd (see bug #670308). I have removed the symlink in /etc/rc2.d and I have used the following unit file:
[Unit]
Description=Vsftpd ftp daemon
After=syslog.target network.target
[Service]
Type=simple
ExecStart=/usr/sbin/vsftpd /etc/vsftpd.conf
ExecReload=/bin/kill -HUP $MAINPID
ExecStartPre=-/bin/mkdir -p /var/run/vsftpd/empty
[Install]
WantedBy=multi-user.target

Miscellaneous
  1. In /etc/default/grub, reduce TIMEOUT to 0 to shorten the boot time.
  2. Enabling dirty regions can help speed up XBMC.
  3. aptitude install upower pm-utils to be able to shutdown/suspend from XBMC. Since XBMC was configured to be started outside any session, you need to explicitely give the appropriate rights by creating the following /var/lib/polkit-1/localauthority/50-local.d/xbmc.pkla:
    [Actions for xbmc user]
    Identity=unix-user:xbmc
    Action=org.freedesktop.upower.*;org.freedesktop.consolekit.system.*
    ResultAny=yes
    ResultInactive=yes
    ResultActive=yes
    

  1. The readibility of the LCD screen is very bad. You should look at the VFD version. The IR receiver reception is poor. The provided remote control is a joke.
  2. Debian Wheezy is not yet released. If you are unfamiliar with Debian, it may be cumbersome to maintain it until the freeze happens in a few months.
  3. Some keys are missing from the provided table. For example, there is no exclamation mark. While there is a scan code for such a key in RC-6 protocol, there is no appropriate key code to translate to: on a QWERTY keyboard, the exclamation mark is on the same key as the number 1. It is possible to map it to some other key code, but the mapping would have been difficult to use.
  4. For example, in ~/.xbmc/userdata/keymaps/harmony.xml.

24 April 2012

David Bremner: Free OpenCL Implementations

It's spring, and young(ish?) hackers' minds turn to OpenCL. What is the state of things? I haven't the faintest idea, but I thought I'd try to share what I find out. So far, just some links. Details to be filled in later, particularly if you, dear reader, tell them to me. Specification LLVM based front ends Mesa backend Rumours/hopes of something working in mesa 8.1?
  • r600g is merged into master as of this writing.
  • clover
Other projects
  • SNU This project seems be only for Cell/ARM/DSP at the moment. Although they make you register to download, it looks like it is LGPL.

23 April 2012

Christian Perrier: Boston marathon 2012

I just read reports about Boston marathon 2012. Incredibly though race this year as the heat raised up to 32-35 C (around 90 F?) for the last runners and 26 C at 10:00 at marathon start. The outcome is indeed impressive: the 2012 winner won with a "not so good" time of 2h12 while last year saw an unofficial world best time of 2h03'02" (there can't be any world record in Boston because of the race being one-way, thus prone to too favourable wind, plus a negative slope between start and finish). Even though this year is less likely to see great records (because this is an olympic year and the best runners won't run another marathon before the Olympics), we still had times around 2h05 in Paris, Rotterdam....and this year's best time in London with 2h04'44" (Dubai time is 2h04'23" but I do not count this one as a mass marathon). So, 2h12 in Boston just says how hard this race was. But, still, that's my dream marathon, even more than NYC marathon. And I'll do it, I promise....though I now have to run below 3h30 to qualify....unless I'm still running marathons when I reach 55, where the qualificaiton mark will be only 3h40 (which is exactly my PB as of now). Will I still be able to run in 3h40 in 4 years? Or will I succeed in running in less than 3h30 in the meantime? We'll see..:-) Until then....my first marathon for this year will be on June 9th, in Caen, as planned. Target: less than 3h40, despite the quite hard profile (and possible windy conditions as the first half of the race is along the famous Normandy beaches code named Sword, Gold and Juno, between Courseulles sur Mer and Ouistreham). So, until then, I'll forget about my nice races in forests and plains, and concentrate on interval trainings, boring sequences of 400m or 1000m runs, etc, etc. That will be the first time I'm doing this but everybody keeps telling me I need to do that if I want to lower my PB (and every marathoner wants to!).

Julien Danjou: OpenStack Swift eventual consistency analysis & bottlenecks

Swift is the software behind the OpenStack Object Storage service. This service provides a simple storage service for applications using RESTful interfaces, providing maximum data availability and storage capacity. I explain here how some parts of the storage and replication in Swift works, and show some of its current limitations. If you don't know Swift and want to read a more "shallow" overview first, you can read John Dickinson's Swift Tech Overview. How Swift storage works If we refer to the CAP theorem, Swift chose availability and partition tolerance and dropped consistency. That means that you'll always get your data, they will be dispersed on many places, but you could get an old version of them (or no data at all) in some odd cases (like some server overload or failure). This compromise is made to allow maximum availability and scalability of the storage platform. But there are mechanisms built into Swift to minimize the potential data inconsistency window: they are responsible for data replication and consistency. The official Swift documentation explains the internal storage in a certain way, but I'm going to write my own explanation here about this. Consistent hashing Swift uses the principle of consistent hashing. It builds what it calls a ring. A ring represents the space of all possible computed hash values divided in equivalent parts. Each part of this space is called a partition. The following schema (stolen from the Riak project) shows the principle nicely: Consistent hashing ring In a simple world, if you wanted to store some objects and distribute them on 4 nodes, you would split your hash space in 4. You would have 4 partitions, and computing hash(object) modulo 4 would tell you where to store your object: on node 0, 1, 2 or 3. But since you want to be able to extend your storage cluster to more nodes without breaking the whole hash mapping and moving everything around, you need to build a lot more partitions. Let's say we're going to build 210 partitions. Since we have 4 nodes, each node will have 210 4 = 256 partitions. If we ever want to add a 5th node, it's easy: we just have to re-balance the partitions and move 1 4 of the partitions from each node to this 5th node. That means all our nodes will end up with 210 5 204 partitions. We can also define a weight for each node, in order for some nodes to get more partitions than others. With 210 partitions, we can have up to 210 nodes in our cluster. Yeepee. For reference, Gregory Holt, one of the Swift authors, also wrote an explanation post about the ring. Concretely, when building one Swift ring, you'll have to say how much partitions you want, and this is what this value is really about. Data duplication Now, to assure availability and partitioning (as seen in the CAP theorem) we also want to store replicas of our objects. By default, Swift stores 3 copies of every objects, but that's configurable. In that case, we need to store each partition defined above not only on 1 node, but on 2 others. So Swift adds another concept: zones. A zone is an isolated space that does not depends on other zone, so in case of an outage on a zone, the other zones are still available. Concretely, a zone is likely to be a disk, a server, or a whole cabinet, depending on the size of your cluster. It's up to you to chose anyway. Consequently, each partitions has not to be mapped to 1 host only anymore, but to N hosts. Each node will therefore store this number of partitions:
number of partition stored on one node = number of replicas   total number of partitions   number of node
Examples:
We split the ring in 210 = 1024 partitions. We have 3 nodes. We want 3 replicas of data.
Each node will store a copy of the full partition space: 3 210 3 = 210 = 1024 partitions.
We split the ring in 211 = 2048 partitions. We have 5 nodes. We want 3 replicas of data.
Each node will store 211 3 5 1129 partitions.
We split the ring in 211 = 2048 partitions. We have 6 nodes. We want 3 replicas of data.
Each node will store 211 3 6 = 1024 partitions.
Three rings to rule them all In Swift, there is 3 categories of thing to store: account, container and objects. An account is what you'd expect it to be, a user account. An account contains containers (the equivalent of Amazon S3's buckets). Each container can contains user-defined key and values (just like a hash table or a dictionary): values are what Swift call objects. Swift wants you to build 3 different and independent rings to store its 3 kind of things (accounts, containers and objects). Internally, the two first categories are stored as SQLite databases, whereas the last one is stored using regular files. Note that this 3 rings can be stored and managed on 3 completely different set of servers. Swift storage schema Data replication Now that we have our storage theory in place (accounts, containers and objects distributed into partitions, themselves stored into multiple zones), let's go the replication practice. When you put something in one of the 3 rings (being an account, a container or an object) it is uploaded into all the zones responsible for the ring partition the object belongs to. This upload into the different zones is the responsibility of the swift-proxy daemon. Swift proxy schema But if one of the zone is failing, you can't upload all your copies in all zones at the upload time. So you need a mechanism to be sure the failing zone will catch up to a correct state at some point. That's the role of the swift- container,account,object -replicator processes. This processes are running on each node part of a zone and replicates their contents to nodes of the other zones. When they run, they walk through all the contents from all the partitions on the whole file system and for each partition, issue a special REPLICATE HTTP request to all the other zones responsible for that same partition. The other zone responds with information about the local state of the partition. That allows the replicator process to decide if the remote zone has an up-to-date version of the partition. In case of account and containers, it doesn't check at the partition level, but check each account/container contained inside each partition. If something is not up-to-date, it will be pushed using rsync by the replicator process. This is why you'll read that the replication updates are "push based" in Swift documentation.
# Pseudo code describing replication process for accounts
# The principle is exactly the same for containers
for account in accounts:
    # Determine the partition used to store this account
    partition = hash(account) % number_of_partitions
    # The number of zone is the number of replicas configured
    for zone in partition.get_zones_storing_this_partition():
        # Send a HTTP REPLICATE command to the remote swift-account-server process
        version_of_account = zone.send_HTTP_REPLICATE_for(account):
        if version_of_account < account.version()
            account.sync_to(zone)
This replication process is O(number of account number of replicas). The more your number of account will increase and the more you will want replicas for your data, the more the replication time for your accounts will grow. The same rule applies for containers.
# Pseudo code describing replication process for objects
for partition in partitions_storing_objects:
    # The number of zone is the number of replicas configured
    for zone in partition.get_zones_storing_this_partition():
        # Send a HTTP REPLICATE command to the remote swift-object-server process
        verion_of_partition = zone.send_HTTP_REPLICATE_for(partition):
        if version_of_partition < partition.version()
            # Use rsync to synchronize the whole partition
            # and all its objects
            partition.rsync_to(zone)
This replication process is O(number of objects partitions number of replicas). The more your number of objects partitions will increase, and the more you will want replicas for your data, the more the replication time for your objects will grow. I think this is something important to know when deciding how to build your Swift architecture. Choose the right number the number of replicas, partitions and nodes. Replication process bottlenecks Copycat File accesses The problem, as you might have guessed, is that to replicate, it walks through every damn things, things being accounts, containers, or object's partition hash files. This means it need to open and read (part of) a every file your node stores to check that data need or not to be replicated! For accounts & containers replication, this is done every 30 seconds by default, but it will likely take more than 30 seconds as soon as you hit around 12 000 containers on a node (see measurements below). Therefore you'll end up checking consistency of accounts & containers on each all node all the time, using obviously a lot of CPU time. For reference, Alex Yang also did an analysis of that same problem. TCP connections Worst, the HTTP connections used to send the REPLICATE commands are not pooled: a new TCP connection is established each time something has to be checked against the same thing stored on a remote zone. This is why you'll see in the Swift's Deployment Guide this lines listed under "general system tuning":
# disable TIME_WAIT.. wait..
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_tw_reuse=1
# double amount of allowed conntrack
net.ipv4.netfilter.ip_conntrack_max = 262144
In my humble opinion, this is more an ugly hack than a tuning. If you don't activate this and if you have a lot of containers on your node, you'll end up soon with thousands of connections in TIME_WAIT state, and you indeed risk to overload the IP conntrack module. Container deletion We also should talk about container deletion. When a user deletes a container from its account, the container is marked as deleted. And that's it. It's not deleted. Therefore the SQLite database file representing the container will continue to be checked for synchronization, over and over. The only way to have a container permanently deleted is to mark an account as deleted. This way the swift-account-reaper will delete all its containers and, finally, the account. Measurement On a pretty big server, I measured the replications to be done at a speed of around 350 account,container,object-partitions /second, which can be a real problem if you chose to build a lots of partition and you have a low number_of_node number_of_replicas ratio. For example, the default parameters runs the container replication every 30 seconds. To check replication status of 12 000 containers stored on one node at the speed of 350 containers/seconds, you'll need around 34 seconds to do so. In the end, you'll never stop checking replication of your containers, and the more you'll have containers, the more your inconsistency window will increase. Conclusion Until some of the code is fixed (the HTTP connection pooling probably being the "easiest" one), I warmly recommend to chose correctly the different Swift parameters for your setup. The replication process optimization consists in having the minimum amount of partitions per node, which can be done by:
  • decreasing the number of partitions
  • decreasing the number of replicas
  • increasing the number of node
For very large setups, some code to speed up accounts and containers synchronization, and remove deleted containers will be required, but this does not exist yet, as far as I know.

Next.

Previous.