Search Results: "nate"

1 June 2015

Russ Allbery: Review: Hild

Review: Hild, by Nicola Griffith
Series: Hild #1
Publisher: Farrar, Straus and Giroux
Copyright: 2013
ISBN: 0-374-28087-8
Format: Hardcover
Pages: 539
Hild was born in seventh-century Britain, daughter of Hereric (would-be king of Daria) and Breguswith. Born, her mother said, to be a light to the land. This much is documented by the Venerable Bede, in The Ecclesiastical History of the English Nation, along with Hild's later rise to become one of the most powerful abbesses in British history. But nearly all of Hild's early life is a cipher. Hild fills in some of that gap with fiction. Specifically, it takes Hild from a child of three, learning her father has been killed, to a young woman, an advisor of Edwin king. It's a coming of age story in part, following her maturation both physically and mentally, her training in when to speak and how, and the dangers of being close to royalty in a fractious, political, and war-torn land. It's also the story of endless maneuvering and care, initially by her mother Breguswith and then by Hild herself sometimes in opposition to her mother, sometimes in alliance as her mother attempts to make a safe place for her daughter and herself in a treacherous court of shifting dangers. But Hild is also a story about Britain. It's a novel about how it felt and how it sounded. How it was organized, primarily among the high-born but with snippets of perspective from the lower classes. And it's a story about women: about weaving, about medicine, about friendships and partnerships and alliances among women, about the politics of marriage and childbirth, but also about the places women held and made in a time when surviving official history is all about the men. Hild is a painstakingly-researched, sprawling, lush, and sensual immersion in a part of history that gets little formal attention: after the Romans, before what we think of as medieval, before England as a country. I've been waiting for a new Nicola Griffith novel for quite a while, since Always in 2007. Hild doesn't disappoint, but it's also different than Griffith's previous writing. It has less of the strong narrative drive and clarity of either her SF or Aud Torvingen stories. Instead, the goal of Hild is to immerse and transport you, to help you feel the shape of Hild's world, to understand her days and tasks, her dreams and dangers. Like all of Griffith's work, it's beautifully written. Griffith puts description at the fore, in the sharp eyes of an observant girl who loves the outdoors, and who has been taught to watch for signs of weather and tools of healing. From early on, Hild's mother sets her up as a seer and prophetess as a way of establishing her value to war leaders and kings, and while some of it is drama and cryptic words, so much of it is careful observation, networks of information-gathering, and sharp deduction about the motives and politics of surrounding kings. Hild is very good at what she does because she has a sharp, quick mind that has been carefully trained, and because she has the aid of her mother's networks and then aid in building her own. Griffith does a wonderful job showing the reader what Hild sees, how she appreciates the world both for its own beauty and for the information she can gather from it, and how to build influence by navigating tense and dangerous moments: waiting for just the right moment and the right word, and taking sudden, impulsive risks and accepting their consequences. Unfortunately for me, this setting is also rich in complex politics and numerous actors, with older and unfamiliar names, and I got lost. Constantly. That's the drawback to immersion: Griffith doesn't hold the reader's hand. We get Hild's thoughts and analysis, and the reader has to keep up. Sometimes I did; sometimes I didn't. There's a dizzying flurry of names here, both personal and place, and while there is a map and a single family tree, neither helped me as much as I wanted them to. At several points, I found myself skimming through the latest shift in the balance between various petty kings because, while I knew I'd seen all the names before, they had come adrift from their context in the story. That was my major frustration with this book. It was all interesting enough that I would have kept thumbing back to a detailed dramatis personae, and indeed I kept checking the family tree, but there just wasn't enough detail there. Even better would have been a brief factual history of the political and military conflicts Hild was living through, keyed by chapter. Hild is startlingly intelligent, leaping from insight to insight, which is wonderful for building character, but which occasionally leaves the reader scrambling to catch up with the connections between her thoughts. I felt like, had I the broader context, I could have understood her insight more readily. All this information is likely available, since Griffith is playing off of documented history, but I'm not the sort of reader who likes doing Internet research while engrossed in a book. So, that's the downside, at least for me. But this book has many strengths, even if you're lost much of the time. Hild as written by Griffith is a fascinating character, full of sharp edges and difficult moods and a powerful belief in what she feels is right. Griffith is at the height of her writing ability when describing Hild making hard choices and taking on burdens that seem too large for her to bear. There are two sections of the book, where Hild is forced by circumstances to lead men in violence, that I think are two of the best bits of writing Griffith has ever published not just because of those scenes themselves, but because of the aftermath, the lingering echos, the way that they shape and inform everything Hild does afterwards. The mingling of reward and loss, maturation and trauma, the sense that the world has shifted both inside and out and it's nearly impossible to say whether the change is for the better or worse. Griffith also knows when not to say too much, and while I found that frustrating for the politics, it does wonders for the characters. Hild's complex and fraught relationship with Gwaldus is the best example. We never know exactly what Gwaldus is thinking; Hild can only guess, and at times one is fairly sure that she guesses wrong. But that doesn't lead to sudden revelations, where the characters finally understand each other. Instead, they both adjust, they maneuver around each other, they find space and understanding where they can, and sometimes they just close off. This book is full of relationships like this: loves that are too complicated for words, bonds that are too dangerous to acknowledge, and characters who can't relax even though they wish the best for each other. At times, it's exhausting reading, but it gives Hild a tension that one wouldn't expect from a sprawling novel full of description and scene-building. Hild is clearly the first book of a series, and leaves quite a lot unresolved. If you want closure in relationships and in politics, there's a lot here that you may find frustrating. And if, like me, you struggle to keep names and politics straight, you're probably going to get lost. But it's well worth the effort for the description, for Hild's thought processes, and for a few haunting scenes that I will be replaying in my head for a very long time. Expect to take your time with this, and wait until you're in the mood for immersion and puzzling out context as you go, but recommended. I suspect it would be even better on a second reading. Rating: 8 out of 10

31 May 2015

Dirk Eddelbuettel: rfoaas 0.1.6

After a few quiet months, a new version of rfoaas is now on CRAN. As before, it shadows upstream release of FOAAS from which carried over the version number 0.1.6. The rfoaas package provides an interface for R to the most excellent FOAAS service--which provides a modern, scalable and RESTful web service for the frequent need to tell someone to f$#@ off. Release 0.1.6 of FOAAS builds on the initial support for filters and now adds working internationalization. This is best shown by example:
R> library(rfoaas)
R> off("Tom", "Everyone")                                      # standard upstream example
[1] "Fuck off, Tom. - Everyone"
R> off("Tom", "Everyone", language="fr")                       # now in French
[1] "Va te faire foutre, Tom. - Everyone"
R> off("Tom", "Everyone", language="de", filter="shoutcloud")  # or in German and LOUD
[1] "VERPISS DICH, TOM. - EVERYONE"
R> 
We start with a standard call to off(), add the (now fully functional upstream) language support and finally illustrate the shoutcloud.io filter added in the preceding 0.1.3 release. As usual, CRANberries provides a diff to the previous CRAN release. Questions, comments etc should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dominique Dumont: Improving update of existing debian/copyright file

Hello One of my last blog introduced the new cme update dpkg-copyright command and explained how to use this command to create a debian/copyright file from scratch. We will see in this blog how cme update dpkg-copyright command can also be used if a debian/copyright file already exists. During the lifetime of a package, creating a new debian/copyright copyright file should only happen once. When updating a package to a new upstream version, the content of the copyright file should be modified to reflect upstream changes by: cme update dpkg-copyright can be run with an existing debian/copyright file: the information extracted by licensecheck are merged in debian/copyright. Unfortunately, this merge may bring back the errors coming from licensecheck that you ve cleaned up manually last time. This last part is not optimal as the manual clean up must be repeated after cme update dpkg-copyright is run. Here s a way to avoid the last problem: cme update dpkg-copyright supports a way to modify the content of debian/copyright after the merge: the optional file debian/fix.scanned.copyright contains instructions to tweak the content of debian/copyright. (See Config::Model::Dpkg::Copyright man page for more details) You may remember that Config::Model handles all configuration information as a tree. The information contained in the copyright file is also stored in a tree structure. debian/fix.scanned.copyright contains instructions to navigate this tree and modify the content of the tree. Imagine that there s a robot (or a monkey) in that configuration tree. Each instruction tells the robot either to move in the tree or to perform an operation. The most useful instructions for debian/fix.scanned.copyright are: Otherwise: Let s see a commented example:
# go to the root of the tree (with '!')
# and remove a bogus Files entry (with ":~")
! Files:~"foo/bogus.c"
// go to a Files entry and append (operator '.=') some info to   owner
! Files:"foo/bar.h"
  Copyright.=" Appended info."
# go back to tree root and assign   info
! Files:"foo/baz.c" Copyright="2014, FooBar"
# add a global license and a file entry missed by licensecheck
# global license must be set before using it in Files entry
# note: MIT text license is provided by cme
! License=MIT
! Files:"3rdparty/missed.c" Copyright="2005 FooBar" License short_name=MIT
Other possible instructions for this file are described in Config::Model::Loader doc. The number of instructions depends on the accuracy of licencecheck command. Hopefully, this number can be reduced by improving licensecheck. Last but not least, the following packages are required: You may want to install also libconfig-model-tkui-perl to update the copyright file with cme GUI. In summary, the command cme updated dpkg-copyright associated with instructions stored in debian/fix.scanned.copyright enable Debian packagers to update with less effort debian/copyright file when updating a package. All the best

27 May 2015

Norbert Preining: USB stick update: TAILS 1.4, GParted 0.22, SysResCD 4.5.2, Debian Jessie

I have posted a view times (here and here) about how to get a multi-boot/multi-purpose USB stick working. Now that TAILS has seens a major upgrade, and Debian 8.0 Jessie has been released, I think it is time to update the procedure to reflect the latest releases. That turned out to be a painful experience, in particular since Debian removed support for any reasonable boot method.

(Update: See this post for fixes to get Debian working again)
multi-usb-stick So going through these explanations one will end up with a usable USB stick that can boot you into TAILS, System Rescue CD, GNU Parted Live CD, but unfortunately not anymore to boot into an installation of Debian 8.0 Jessie installation. But the USB stick will still be usable as normal media. Let us repeat some things from the original post concerning the wishlist and the main players: I have a long wishlist of items a boot stick should fulfill This time I have added the GNOME/GNU Partition Editor gparted as it came in useful at times. Requirements A USB stick, the iso images of TAILS 1.4, SystemRescueCD 4.5.2, GParted Lice CD 0.22.0, and some tool to access iso images, for example ISOmaster (often available from your friendly Linux distribution). I assume that you have already an USB stick prepared as described previously. If this is not the case, please go there and follow the section on preparing your usb stick. Two types of boot options We will employ two different approaches to boot special systems: the one is directly from an iso image, the other via extraction of the necessary kernels and images. At the moment we have the following status with respect to boot methods: What is a pity is that during the testing phase, booting and installation from testing images worked for Debian as documented previously. But with the final images (and my guess it has to do with systemd, wouldn t surprise me), there is no Debian CD detected as it cannot find the iso image on the USB stick. Bummer. That means that for having a Debian/USB stick you need a dedicated one. Booting from ISO image Grub has gained quite some time ago the ability to boot directly from an ISO image. In this case the iso image is mounted via loopback, and the kernel and initrd are specified relatively to the iso image root. For both SystemRescueCD and GNOME Partition Live CD, just drop the iso files into /boot/iso/, in my case /boot/iso/systemrescuecd-x86-4.5.2.iso and /boot/iso/gparted-live-0.22.0-1-i586.iso. After that, entries like the following have to be added to grub.cfg. For the full list see grub.cfg:
submenu "System Rescue CD 4.5.2 (via ISO) ---> "  
  set isofile="/boot/iso/systemrescuecd-x86-4.5.2.iso"
  menuentry "SystemRescueCd (64bit, default boot options)"  
        set gfxpayload=keep
        loopback loop (hd0,1)$isofile
        linux   (loop)/isolinux/rescue64 isoloop=$isofile
        initrd  (loop)/isolinux/initram.igz
   
  ...
 
 
submenu "GNU/Gnome Parted Live CD 0.22.0 (via ISO) ---> "  
  set isofile="/boot/iso/gparted-live-0.22.0-1-i586.iso"
  menuentry "GParted Live (Default settings)" 
    loopback loop (hd0,1)$isofile
    linux (loop)/live/vmlinuz boot=live username=user config components quiet noswap noeject  ip=  nosplash findiso=$isofile
    initrd (loop)/live/initrd.img
   
  ...
 
Note the added isoloop=$isofile and findiso=$isofile that helps the installer find the iso images. Booting via extraction of kernels and images This is a bit more tedious, but still not too bad. Installation of TAILS files Assuming you have access to the files on the TAILS CD via the directory ~/tails, execute the following commands:
mkdir -p /usbstick/boot/tails
cp -a ~/tails/live/* /usbstick/boot/tails/
The grub.cfg entries look now similar to the following:
submenu "TAILS Environment 1.4 ---> "  
  menuentry "Tails64 Live System"  
        linux   /boot/tails/vmlinuz2 boot=live live-media-path=/boot/tails config live-media=removable nopersistent noprompt timezone=Etc/UTC block.events_dfl_poll_msecs=1000 splash noautologin module=Tails  libata.force=noncq
        initrd  /boot/tails/initrd2.img
   
  ...
 
The important part here is the live-media-path=/boot/tails, otherwise TAILS will not find the correct files for booting. The rest of the information was extracted from the boot setup of TAILS itself. Current status of USB stick Just to make sure, the usb stick should contain at the current stage the following files:
/boot/
    iso/
        gparted-live-0.22.0-1-i586.iso
        systemrescuecd-x86-4.5.2.iso
    tails/
        vmlinuz Tails.module initrd.img ....
    grub/
        fonts/
            lots of files
        locale/
            lots of files
        x86_64-efi/
            lots of files
        font.pf2
        grubenv
        grub.cfg            *this file we create in the next step!!*
/EFI
    BOOT/
        BOOTX64.EFI
The Grub config file grub.cfg The final step is to provide a grub config file in /usbstick/boot/grub/grub.cfg. I created one by looking at the isoboot.cfg files both in the SystemRescueCD, TAILS iso images, GParted iso image, and the Debian/Jessie image, and converting them to grub syntax. Excerpts have been shown above in the various sections. I spare you all the details, grab a copy here: grub.cfg Conclusion That s it. Now you can anonymously provide data about your evil government, rescue your friends computer, fix a forgotten Windows password, and above all, install a proper free operating system. If you have any comments, improvements or suggestions, please drop me a comment. I hope this helps a few people getting a decent USB boot stick running. Enjoy. Postscriptum concerning Debian/Jessie Working method is now described in this post. So I have first tried to boot directly into the Debian Jessie firmware ISO image by dropping the ISO into /boot/iso/firmware-8.0.0-amd64-i386-netinst.iso and adding a grub entry like:
submenu "Debian 8.0 Jessie NetInstall ---> "  
    set isofile="/boot/iso/firmware-8.0.0-amd64-i386-netinst.iso"
 
    menuentry '64 bit Install'  
        set background_color=black
        loopback loop (hd0,1)$isofile
        linux    (loop)/install.amd/vmlinuz iso-scan/ask_second_pass=true iso-scan/filename=$isofile vga=788 -- quiet 
        initrd   (loop)/install.amd/initrd.gz
     
    ...
 
That was a no go. I have added the iso-scan/ask_second_pass=true iso-scan/filename=$isofile thingy after some research in forum and web, without any change. Of course I have also tried the official ISO image debian-8.0.0-amd64-netinst.iso, with the same effect. Although I was sure it doesn t make any difference, I have also tried to extract the kernel and initrd, and boot directly from it, i.e., copying the files to /usbstick/boot/debian/ as follows:
mkdir -p /usbstick/boot/debian
cp -a ~/tails/install.amd /usbstick/boot/debian/
cp -a ~/tails/install.386 /usbstick/boot/debian/
In addition, copy the iso image itself into /usbstick/boot/iso (or directly into the root of the usb stick, didn t change anything), and added a grub.cfg entry as follows:
submenu "Debian 8.0 Jessie NetInstall ---> "  
  menuentry '64 bit Install'  
    set background_color=black
    linux    /boot/debian/install.amd/vmlinuz vga=788 -- quiet 
    initrd   /boot/debian/install.amd/initrd.gz
   
  ...
 
All these were without success, always ending up in error messages like: Mounting sdb, this is not a Debian Installation CD, etc etc. I have also submitted a bug report to the installation reports, unfortunately no answer (not that I expected one). In case someone has a better idea, please let me know! It is very sad that these kind of support is removed. The Installation Manual lists some options to install from USB, and the only difference is that there syslinux on a FAT16 system is used. This restricts the USB stick size a lot, and does not allow for easy multi boot via grub. As reported here, I had this actually running. Unfortunately, I stupidly removed the old iso image and replaced it with the current Jessie ISO image, assuming that there will be no regression. Wrong assumption. Now I cannot even investigate the changes by looking into the initrd. Looking at the date of the original post I see that it is more or less one year ago, so before the Systemd introduction. I don t know whether this has any effect, I doubt, as the installer is separate. But something happened in the mean time. Bad for us.

Vincent Bernat: Live patching QEMU for VENOM mitigation

CVE-2015-3456, also known as VENOM, is a security vulnerability in QEMU virtual floppy controller:
The Floppy Disk Controller (FDC) in QEMU, as used in Xen [ ] and KVM, allows local guest users to cause a denial of service (out-of-bounds write and guest crash) or possibly execute arbitrary code via the FD_CMD_READ_ID, FD_CMD_DRIVE_SPECIFICATION_COMMAND, or other unspecified commands.
Even when QEMU has been configured with no floppy drive, the floppy controller code is still active. The vulnerability is easy to test1:
#define FDC_IOPORT 0x3f5
#define FD_CMD_READ_ID 0x0a
int main()  
    ioperm(FDC_IOPORT, 1, 1);
    outb(FD_CMD_READ_ID, FDC_IOPORT);
    for (size_t i = 0;; i++)
        outb(0x42, FDC_IOPORT);
    return 0;
 
Once the fix installed, all processes still have to be restarted for the upgrade to be effective. It is possible to minimize the downtime by leveraging virsh save. Another possibility would be to patch the running processes. The Linux kernel attracted a lot of interest in this area, with solutions like Ksplice (mostly killed by Oracle), kGraft (by Red Hat) and kpatch (by Suse) and the inclusion of a common framework in the kernel. The userspace has far less out-of-the-box solutions2. I present here a simple and self-contained way to patch a running QEMU to remove the vulnerability without requiring any sensible downtime. Here is a short demonstration:

Proof of concept First, let s find a workaround that would be simple to implement through live patching: while modifying running code text is possible, it is easier to modify a single variable.

Concept Looking at the code of the floppy controller and the patch, we can avoid the vulnerability by not accepting any command on the FIFO port. Each request would be answered by Invalid command (0x80) and a user won t be able to push more bytes to the FIFO until the answer is read and the FIFO queue reset. Of course, the floppy controller would be rendered useless in this state. But who cares? The list of commands accepted by the controller on the FIFO port is contained in the handlers[] array:
static const struct  
    uint8_t value;
    uint8_t mask;
    const char* name;
    int parameters;
    void (*handler)(FDCtrl *fdctrl, int direction);
    int direction;
  handlers[] =  
      FD_CMD_READ, 0x1f, "READ", 8, fdctrl_start_transfer, FD_DIR_READ  ,
      FD_CMD_WRITE, 0x3f, "WRITE", 8, fdctrl_start_transfer, FD_DIR_WRITE  ,
    /* [...] */
      0, 0, "unknown", 0, fdctrl_unimplemented  , /* default handler */
 ;
To avoid browsing the array each time a command is received, another array is used to map each command to the appropriate handler:
/* Associate command to an index in the 'handlers' array */
static uint8_t command_to_handler[256];
static void fdctrl_realize_common(FDCtrl *fdctrl, Error **errp)
 
    int i, j;
    static int command_tables_inited = 0;
    /* Fill 'command_to_handler' lookup table */
    if (!command_tables_inited)  
        command_tables_inited = 1;
        for (i = ARRAY_SIZE(handlers) - 1; i >= 0; i--)  
            for (j = 0; j < sizeof(command_to_handler); j++)  
                if ((j & handlers[i].mask) == handlers[i].value)  
                    command_to_handler[j] = i;
                 
             
         
     
    /* [...] */
 
Our workaround is to modify the command_to_handler[] array to map all commands to the fdctrl_unimplemented() handler (the last one in the handlers[] array).

Testing with gdb To check if the workaround works as expected, we test it with gdb. Unless you have compiled QEMU yourself, you need to install a package with debug symbols. Unfortunately, on Debian, they are not available, yet3. On Ubuntu, you can install the qemu-system-x86-dbgsym package after enabling the appropriate repositories. The following function for gdb maps every command to the unimplemented handler:
define patch
  set $handler = sizeof(handlers)/sizeof(*handlers)-1
  set $i = 0
  while ($i < 256)
   set variable command_to_handler[$i++] = $handler
  end
  printf "Done!\n"
end
Attach to the vulnerable process (with attach), call the function (with patch) and detach of the process (with detach). You can check that the exploit is not working anymore. This could be easily automated.

Limitations Using gdb has two main limitations:
  1. It needs to be installed on each host to be patched.
  2. The debug packages need to be installed as well. Moreover, it can be difficult to fetch previous versions of those packages.

Writing a custom patcher To overcome those limitations, we can write a customer patcher using the ptrace() system call without relying on debug symbols being present.

Finding the right memory spot Before being able to modify the command_to_handler[] array, we need to know its location. The first clue is given by the symbol table. To query it, use readelf -s:
$ readelf -s /usr/lib/debug/.build-id/09/95121eb46e2a4c13747ac2bad982829365c694.debug   \
>   sed -n -e 1,3p -e /command_to_handler/p
Symbol table '.symtab' contains 27066 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
  8485: 00000000009f9d00   256 OBJECT  LOCAL  DEFAULT   26 command_to_handler
This table is usually stripped out of the executable to save space, like shown below:
$ file -b /usr/bin/qemu-system-x86_64   tr , \\n
ELF 64-bit LSB shared object
 x86-64
 version 1 (SYSV)
 dynamically linked
 interpreter /lib64/ld-linux-x86-64.so.2
 for GNU/Linux 2.6.32
 BuildID[sha1]=0995121eb46e2a4c13747ac2bad982829365c694
 stripped
If your distribution provides a debug package, the debug symbols are installed in /usr/lib/debug. Most modern distributions are now relying on the build ID4 to map an executable to its debugging symbols, like the example above. Without a debug package, you need to recompile the existing package without stripping debug symbols in a clean environment5. On Debian, this can be done by setting the DEB_BUILD_OPTIONS environment variable to nostrip. We have now two possible cases:
  • the easy one, and
  • the hard one.

The easy case On x86, here is the standard layout of a regular Linux process in memory6: Memory layout of a regular process on x86 The random gaps (ASLR) are here to prevent an attacker from reliably jumping to a particular exploited function in memory. On x86-64, the layout is quite similar. The important point is that the base address of the executable is fixed. The memory mapping of a process is also available through /proc/PID/maps. Here is a shortened and annotated example on x86-64:
$ cat /proc/3609/maps
00400000-00401000         r-xp 00000000 fd:04 483  not-qemu [text segment]
00601000-00602000         r--p 00001000 fd:04 483  not-qemu [data segment]
00602000-00603000         rw-p 00002000 fd:04 483  not-qemu [BSS segment]
[random gap]
02419000-0293d000         rw-p 00000000 00:00 0    [heap]
[random gap]
7f0835543000-7f08356e2000 r-xp 00000000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08356e2000-7f08358e2000 ---p 0019f000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e2000-7f08358e6000 r--p 0019f000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e6000-7f08358e8000 rw-p 001a3000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e8000-7f08358ec000 rw-p 00000000 00:00 0
7f08358ec000-7f083590c000 r-xp 00000000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835aca000-7f0835acd000 rw-p 00000000 00:00 0
7f0835b08000-7f0835b0c000 rw-p 00000000 00:00 0
7f0835b0c000-7f0835b0d000 r--p 00020000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835b0d000-7f0835b0e000 rw-p 00021000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835b0e000-7f0835b0f000 rw-p 00000000 00:00 0
[random gap]
7ffdb0f85000-7ffdb0fa6000 rw-p 00000000 00:00 0    [stack]
With a regular executable, the value given in the symbol table is an absolute memory address:
$ readelf -s not-qemu   \
>   sed -n -e 1,3p -e /command_to_handler/p
Symbol table '.dynsym' contains 9 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    47: 0000000000602080   256 OBJECT  LOCAL  DEFAULT   25 command_to_handler
So, the address of command_to_handler[], in the above example, is just 0x602080.

The hard case To enhance security, it is possible to load some executables at a random base address, just like a library. Such an executable is called a Position Independent Executable (PIE). An attacker won t be able to rely on a fixed address to find some helpful function. Here is the new memory layout: Memory layout of a PIE process on x86 With a PIE process, the value in the symbol table is now an offset from the base address.
$ readelf -s not-qemu-pie   sed -n -e 1,3p -e /command_to_handler/p
Symbol table '.dynsym' contains 17 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    47: 0000000000202080   256 OBJECT  LOCAL  DEFAULT   25 command_to_handler
If we look at /proc/PID/maps, we can figure out where the array is located in memory:
$ cat /proc/12593/maps
7f6c13565000-7f6c13704000 r-xp 00000000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13704000-7f6c13904000 ---p 0019f000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13904000-7f6c13908000 r--p 0019f000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13908000-7f6c1390a000 rw-p 001a3000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c1390a000-7f6c1390e000 rw-p 00000000 00:00 0
7f6c1390e000-7f6c1392e000 r-xp 00000000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b2e000-7f6c13b2f000 r--p 00020000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b2f000-7f6c13b30000 rw-p 00021000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b30000-7f6c13b31000 rw-p 00000000 00:00 0
7f6c13b31000-7f6c13b33000 r-xp 00000000 fd:04 4594  not-qemu-pie [text segment]
7f6c13cf0000-7f6c13cf3000 rw-p 00000000 00:00 0
7f6c13d2e000-7f6c13d32000 rw-p 00000000 00:00 0
7f6c13d32000-7f6c13d33000 r--p 00001000 fd:04 4594  not-qemu-pie [data segment]
7f6c13d33000-7f6c13d34000 rw-p 00002000 fd:04 4594  not-qemu-pie [BSS segment]
[random gap]
7f6c15c46000-7f6c15c67000 rw-p 00000000 00:00 0     [heap]
[random gap]
7ffe823b0000-7ffe823d1000 rw-p 00000000 00:00 0     [stack]
The base address is 0x7f6c13b31000, the offset is 0x202080 and therefore, the location of the array is 0x7f6c13d33080. We can check with gdb:
$ print &command_to_handler
$1 = (uint8_t (*)[256]) 0x7f6c13d33080 <command_to_handler>

Patching a memory spot Once we know the location of the command_to_handler[] array in memory, patching it is quite straightforward. First, we start tracing the target process:
/* Attach to the running process */
static int
patch_attach(pid_t pid)
 
    int status;
    printf("[.] Attaching to PID %d...\n", pid);
    if (ptrace(PTRACE_ATTACH, pid, NULL, NULL) == -1)  
        fprintf(stderr, "[!] Unable to attach to PID %d: %m\n", pid);
        return -1;
     
    if (waitpid(pid, &status, 0) == -1)  
        fprintf(stderr, "[!] Error while attaching to PID %d: %m\n", pid);
        return -1;
     
    assert(WIFSTOPPED(status)); /* Tracee may have died */
    if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &si) == -1)  
        fprintf(stderr, "[!] Unable to read siginfo for PID %d: %m\n", pid);
        return -1;
     
    assert(si.si_signo == SIGSTOP); /* Other signals may have been received */
    printf("[*] Successfully attached to PID %d\n", pid);
    return 0;
 
Then, we retrieve the command_to_handler[] array, modify it and put it back in memory7.
static int
patch_doit(pid_t pid, unsigned char *target)
 
    int ret = -1;
    unsigned char *command_to_handler = NULL;
    size_t i;
    /* Get the table */
    printf("[.] Retrieving command_to_handler table...\n");
    command_to_handler = ptrace_read(pid,
                                     target,
                                     QEMU_COMMAND_TO_HANDLER_SIZE);
    if (command_to_handler == NULL)  
        fprintf(stderr, "[!] Unable to read command_to_handler table: %m\n");
        goto out;
     
    /* Check if the table has already been patched. */
    /* [...] */
    /* Patch it */
    printf("[.] Patching QEMU...\n");
    for (i = 0; i < QEMU_COMMAND_TO_HANDLER_SIZE; i++)  
        command_to_handler[i] = QEMU_NOT_IMPLEMENTED_HANDLER;
     
    if (ptrace_write(pid, target, command_to_handler,
           QEMU_COMMAND_TO_HANDLER_SIZE) == -1)  
        fprintf(stderr, "[!] Unable to patch command_to_handler table: %m\n");
        goto out;
     
    printf("[*] QEMU successfully patched!\n");
    ret = 0;
out:
    free(command_to_handler);
    return ret;
 
Since ptrace() only allows to read or write a word at a time, ptrace_read() and ptrace_write() are wrappers to read or write arbitrary large chunks of memory8. Here is the code for ptrace_read():
/* Read memory of the given process */
static void *
ptrace_read(pid_t pid, void *address, size_t size)
 
    /* Allocate the buffer */
    uword_t *buffer = malloc((size/sizeof(uword_t) + 1)*sizeof(uword_t));
    if (!buffer) return NULL;
    /* Read word by word */
    size_t readsz = 0;
    do  
        errno = 0;
        if ((buffer[readsz/sizeof(uword_t)] =
                ptrace(PTRACE_PEEKTEXT, pid,
                       (unsigned char*)address + readsz,
                       0)) && errno)  
            fprintf(stderr, "[!] Unable to peek one word at address %p: %m\n",
                    (unsigned char *)address + readsz);
            free(buffer);
            return NULL;
         
        readsz += sizeof(uword_t);
      while (readsz < size);
    return (unsigned char *)buffer;
 

Putting the pieces together The patcher is provided with the following information:
  • the PID of the process to be patched,
  • the command_to_handler[] offset from the symbol table, and
  • the build ID of the executable file used to get this offset (as a safety measure).
The main steps are:
  1. Attach to the process with ptrace().
  2. Get the executable name from /proc/PID/exe.
  3. Parse /proc/PID/maps to find the address of the text segment (it s the first one).
  4. Do some sanity checks:
    • check there is a ELF header at this location (4-byte magic number),
    • check the executable type (ET_EXEC for regular executables, ET_DYN for PIE), and
    • get the build ID and compare with the expected one.
  5. From the base address and the provided offset, compute the location of the command_to_handler[] array.
  6. Patch it.
You can find the complete patcher on GitHub.
$ ./patch --build-id 0995121eb46e2a4c13747ac2bad982829365c694 \
>         --offset 9f9d00 \
>         --pid 16833
[.] Attaching to PID 16833...
[*] Successfully attached to PID 16833
[*] Executable name is /usr/bin/qemu-system-x86_64
[*] Base address is 0x7f7eea912000
[*] Both build IDs match
[.] Retrieving command_to_handler table...
[.] Patching QEMU...
[*] QEMU successfully patched!

  1. The complete code for this test is on GitHub.
  2. An interesting project seems to be Katana. But there are also some insightful hacking papers on the subject.
  3. Some packages come with a -dbg package with debug symbols, some others don t. Fortunately, a proposal to automatically produce debugging symbols for everything is near completion.
  4. The Fedora Wiki contains the rationale behind the build ID.
  5. If the build is incorrectly reproduced, the build ID won t match. The information provided by the debug symbols may or may not be correct. Debian currently has a reproducible builds effort to ensure that each package can be reproduced.
  6. Anatomy of a program in memory is a great blog post explaining in more details how a program lives in memory.
  7. Being an uninitialized static variable, the variable is in the BSS section. This section is mapped to a writable memory segment. If it wasn t the case, with Linux, the ptrace() system call is still allowed to write. Linux will copy the page and mark it as private.
  8. With Linux 3.2 or later, process_vm_readv() and process_vm_writev() can be used to transfer data from/to a remote process without using ptrace() at all. However, ptrace() would still be needed to reliably stop the main thread.

Dirk Eddelbuettel: drat 0.0.4: Yet more features and documentation

A new version, now at 0.0.4, of the drat package arrived on CRAN yesterday. Its name stands for drat R Archive Template, and it helps with easy-to-create and easy-to-use repositories for R packages, and is finding increasing by other projects. Version 0.0.4 brings both new code and more documentation: Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

26 May 2015

Julien Danjou: OpenStack Summit Liberty from a Ceilometer & Gnocchi point of view

Last week I was in Vancouver, BC for the OpenStack Summit, discussing the new Liberty version that will be released in 6 months. I've attended the summit mainly to discuss and follow-up new developments on Ceilometer, Gnocchi and Oslo. It has been a pretty good week and we were able to discuss and plan a few interesting things. Ops feedback
We had half a dozen Ceilometer sessions, and the first one was dedicated to getting feedbacks from operators using Ceilometer. We had a few operators present, and a few of the Ceilometer team. We had constructive discussion, and my feeling is that operators struggles with 2 things so far: scaling Ceilometer storage and having Ceilometer not killing the rest of OpenStack. We discussed the first point as being addressed by Gnocchi, and I presented a bit Gnocchi itself, as well as how and why it will fix the storage scalability issue operators encountered so far. Ceilometer putting down the OpenStack installation is more interesting problem. Ceilometer pollsters request information from Nova, Glance to gather statistics. Until Kilo, Ceilometer used to do that regularly and at fixed interval, causing high pike load in OpenStack. With the introduction of jitter in Kilo, this should be less of a problem. However, Ceilometer hits various endpoints in OpenStack that are poorly designed, and hitting those endpoints of Nova or other components triggers a lot of load on the platform. Unfortunately, this makes operators blame Ceilometer rather than blaming the components being guilty of poor designs. We'd like to push forward improving these components, but it's probably going to take a long time. Componentisation
When I started the Gnocchi project last year, I pretty soon realized that we would be able to split Ceilometer itself in different smaller components that could work independently, while being able to leverage each others. For example, Gnocchi can run standalone and store your metrics even if you don't use Ceilometer nor even OpenStack itself. My fellow developer Chris Dent had the same idea about splitting Ceilometer a few months ago and drafted a proposal. The idea is to have Ceilometer split in different parts that people could assemble together or run on their owns. Interestingly enough, we had three 40 minutes sessions planned to talk and debate about this division of Ceilometer, though we all agreed in 5 minutes that this was the good thing to do. Five more minutes later, we agreed on which part to split. The rest of the time was allocated to discuss various details of that split, and I engaged to start doing the work with Ceilometer alarming subsystem. I wrote a specification on the plane bringing me to Vancouver, that should be approved pretty soon now. I already started doing the implementation work. So fingers crossed, Ceilometer should have a new components in Liberty handling alarming on its own. This would allow users for example to only deploys Gnocchi and Ceilometer alarm. They would be able to feed data to Gnocchi using their own system, and build alarms using Ceilometer alarm subsystem relying on Gnocchi's data. Gnocchi
We didn't have a Gnocchi dedicated slot mainly because I indicated I didn't feel we needed one. We anyway discussed a few points around coffee, and I've been able to draw a few new ideas and changes I'd like to see in Gnocchi. Mainly changing the API contract to be more asynchronously so we can support InfluxDB more correctly, and improve Carbonara (the library we created to manipulate timeseries) based drivers to be faster. All of those should plus a few Oslo tasks I'd like to tackle should keep me busy for the next cycle!

Norbert Preining: Debian/TeX Live 2015 preparations

I have uploaded a preliminary version of the texlive-bin based on the 2015 sources (plus the first fixes) to the Debian archive, targeting experimental. As there are four new packages built from the sources (libtexlua52, -dev, libtexluajit2, -dev) the packages have to go through the NEW queue, which at the moment is an impressive 500+ entries long (nearly top in total history). But ftp-masters are currently very active and I hope they continue for some time. Debian - TeX Live 2015 Anyway, there are still many things to work out, especially a rework of tex-common for the new fmtutil, in the same way we did some time ago for updmap That also means that all packages shipping formats will need to be rebuild with the new tex-common. Fortunately there are not many additional formats shipped, I think all of them are under my control, so that should be easy. For those who want to peak at the current packages, here they are compiled for amd64. Also available as apt-sources:
deb http://people.debian.org/~preining/TeX/ exp/
deb-src http://people.debian.org/~preining/TeX/ exp/
WARNING Do not try to actually run these binaries unless you know what you are doing  In case you have some spare time and bit of programming experience, the Debian TeX Team is looking for a bit of support in these days.

24 May 2015

Wouter Verhelst: Fixing CVE-2015-0847 in Debian

Because of CVE-2015-0847 and CVE-2013-7441, two security issues in nbd-server, I've had to updates for nbd, for which there are various supported versions: upstream, unstable, stable, oldstable, oldoldstable, and oldoldstable-backports. I've just finished uploading security fixes for the various supported versions of nbd-server in Debian. There're various relevant archives, and unfortunately it looks like they all have their own way of doing things regarding security: While I understand how the differences between the various approaches have come to exist, I'm not sure I understand why they are necessary. Clearly, there's some room for improvement here. As anyone who reads the above may see, doing an upload for squeeze-lts is in fact the easiest of the three "stable" approaches, since no intermediate steps are required. While I'm not about to advocate dropping all procedures everywhere, a streamlining of them might be appropriate.

23 May 2015

Eddy Petri&#537;or: HOWTO: No SSH logins SFTP only chrooted server configuration with OpenSSH

If you are in a situation where you want to set up a SFTP server in a more secure way, don't want to expose anything from the server via SFTP and do not want to enable SSH login on the account allowed to sftp, you might find the information below useful.

What do we want to achive:
To obtain all of the above we will create a dedicated account which will be chroot-ed, its home will be stored on a removable/no always mounted drive (acessing SFTP will not work when the drive is not mounted).

Mount the removable drive which will hold the SFTP area (you might need to add some entry in fstab).

Create the account to be used for SFTP access (on a Debian system this will do the trick):
# adduser --system --home /media/Store/sftp --shell /usr/sbin/nologin sftp

This will create the account sftp which has login disabled, shell is /usr/sbin/nologin and create the home directory for this user.

Unfortunately the default ownership of the home directory of this user are incompatible with chroot-ing in SFTP (which prevents access to other files on the server). A message like the one below will be generated in this kind of case:
$ sftp -v sftp@localhost
[..]
sftp@localhost's password:
debug1: Authentication succeeded (password).
Authenticated to localhost ([::1]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
Write failed: Broken pipe
Couldn't read packet: Connection reset by peer
Also /var/log/auth.log will contain something like this:
fatal: bad ownership or modes for chroot directory "/media/Store/sftp"

The default permissions are visible using the 'namei -l' command on the sftp home directory:
# namei -l /media/Store/sftp
f: /media/Store/sftp
drwxr-xr-x root root /
drwxr-xr-x root root media
drwxr-xr-x root root Store
drwxr-xr-x sftp nogroup sftp
We change the ownership of the sftp directory and make sure there is a place for files to be uploaded in the SFTP area:
# chown root:root /media/Store/sftp
# mkdir /media/Store/sftp/upload
# chown sftp /media/Store/sftp/upload

We isolate the sftp users from other users on the system and configure a chroot-ed environment for all users accessing the SFTP server:
# addgroup sftpusers
# adduser sftp sftusers
Set a password for the sftp user so password authentication works:
# passwd sftp
Putting all pieces together, we restrict access only to the sftp user, allow it access via password authentication only to SFTP, but not SSH (and disallow tunneling and forwarding or empty passwords).

Here are the changes done in /etc/ssh/sshd_config:
PermitEmptyPasswords no
PasswordAuthentication yes
AllowUsers sftp
Subsystem sftp internal-sftp
Match Group sftpusers
ChrootDirectory %h
ForceCommand internal-sftp
X11Forwarding no
AllowTcpForwarding no
PermitTunnel no
Reload the sshd configuration (I'm using systemd):
# systemctl reload ssh.service
Check sftp user can't login via SSH:
$ ssh sftp@localhost
sftp@localhost's password:
This service allows sftp connections only.
Connection to localhost closed.
But SFTP is working and is restricted to the SFTP area:
$ sftp sftp@localhost
sftp@localhost's password:
Connected to localhost.
sftp> ls
upload
sftp> pwd
Remote working directory: /
sftp> put netbsd-nfs.bin
Uploading netbsd-nfs.bin to /netbsd-nfs.bin
remote open("/netbsd-nfs.bin"): Permission denied
sftp> cd upload
sftp> put netbsd-nfs.bin
Uploading netbsd-nfs.bin to /upload/netbsd-nfs.bin
netbsd-nfs.bin 100% 3111KB 3.0MB/s 00:00
Now your system is ready to accept sftp connections, things can be uploaded in the upload directory and whenever the external drive is unmounted, SFTP will NOT work.

Note: Since we added 'AllowUsers sftp', you can test no local user can login via SSH. If you don't want to restrict access only to the sftp user, you can whitelist other users by adding them in the AllowUsers directive, or dropping it entirely so all local users can SSH into the system.

22 May 2015

Michal &#268;iha&#345;: Weblate 2.3

Weblate 2.3 has been released today. It comes with better features for project owners, better file formats support and more configuration options for users. Full list of changes for 2.3: You can find more information about Weblate on http://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, Gammu, Weblate itself and other projects. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! PS: The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: English phpMyAdmin SUSE Weblate 0 comments

21 May 2015

Dirk Eddelbuettel: BH release 1.58.0-1

A new released of BH is now on CRAN. BH provides a large part of the Boost C++ libraries as a set of template headers for use by R and Rcpp. This release both upgrades the version of Boost to the current release, and adds a new library: Boost MultiPrecision . A brief summary of changes from the NEWS file is below.
Changes in version 1.58.0-1 (2015-05-21)
  • Upgraded to Boost 1.58 installed directly from upstream source
  • Added Boost MultiPrecision as requested in GH ticket #12 based on rcpp-devel request by Jordi Molins Coronado
Courtesy of CRANberries, there is also a diffstat report for the most recent release. Comments and suggestions are welcome via the mailing list or the issue tracker at the GitHubGitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: RInside 0.2.13

A new release 0.2.13 of RInside is now on CRAN. RInside provides a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by Rcpp. This release works around a bug in R 3.2.0, and addressed in R 3.2.0-patched. The NEWS extract below has more details.
Changes in RInside version 0.2.13 (2015-05-20)
  • Added workaround for a bug in R 3.2.0: by including the file RInterface.h only once we do not getting linker errors due to multiple definitions of R_running_as_main_program (which is now addressed in R-patched as well).
  • Small improvements to the Travis CI script.
CRANberries also provides a short report with changes from the previous release. More information is on the RInside page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

20 May 2015

Rhonda D'Vine: Berge

I wrote well over one year ago about Earthlings. It really did have some impact on my life. Nowadays I try to avoid animal products where possible, especially for my food. And in the context of vegan information that I tracked I stumbled upon a great band from Germany: Berge. They recently started a deal with their record label which says that if they receive one million clicks within the next two weeks on their song 10.000 Tr nen their record label is going to donate 10.000,- euros to a German animal rights organization. Reason enough for me to share this band with you! :)
(For those who are puzzled by the original upload date of the video: Don't let yourself get confused, the call for it is from this monday) Like always, enjoy!

/music permanent link Comments: 3 Flattr this

Norbert Preining: Shishiodoshi or Us and They On the perceived exclusivity of Japanese

The other day I received from my Japanese teacher an interesting article by Yamazaki Masakazu comparing garden styles, and in particular the attitude towards and presentation of water in Japanese and European gardens (page 1, page 2). The author s list of achievements is long, various professorships, dramatist, literature critique, recognized as Person of Cultural Merit, just to name a view. I was looking forward to an interesting and high quality article! The article itself introduces the reader to Shishiodoshi, one of the standard ingredients of a Japanese garden: It is a device where water drips into a bamboo tube that is also a seesaw. At some point the water in the bamboo tube makes the seesaw switch over and the water pours out, after which the seesaw returns to the original position and a new cycle begins. Shishiodoshi The author describes his feelings and thoughts about the shishiodoshi, in particular connects human life (stress and relieve, cycles), the flow of time, and some other concepts with the shishiodoshi. Up to here it is a wonderful article providing interesting insights into the way the author thinks. Unfortunately, then the author tries to underline his ideas by comparing the Japanese shishiodoshi with European style water fountains, describing the former with all favorable properties and full of deep meaning, while the latter is qualified as beautiful and nice, but bare of any deeper meaning. I don t go into details that the whole comparison is anyway a bad one, as he is comparing Baroque style fountains, a very limited period, and furthermore ignores the fact that water fountains are not genuinely European (isn t there one in the Kenrokuen, one of the three most famous gardens in Japan!?), nor does he consider any other water-installation that might be available. What really destroys the in principle very nice article is the tone: The general tone of the article then can be summarized into: The shishiodoshi is rich on meaning, connects to the life of humans, instigates philosophical reflections, represents nature, the flow of time etc. The water fountain is beautiful and gorgeous, but that is all. I don t think that this separation, or this negative undertone, was created on purpose by the author. A person of his stature is supposedly above this level of primitive comparison. I believe that it is nothing else but a consequence of upbringing and the general attitude that permeates the whole society with this feeling of separateness. Us and They Repeatedly providing sentences like Japanese people and Western people have different tastes.. ( ). About 10 times in this short article expressions like Japanese and Westerner appear, leaving the reader with a bitter taste of an author that considers first the Japanese a people (what about Ainu, Ryukyu, etc?), and second that the Japanese are exclusive in the sense that they are set apart from the rest of the world in their way of thinking, living, being. What puzzles me is that this is not only a singular opinion, but a very general straight in the Japanese media, TV, radio, newspaper, books. Everyone considers Japan and Japanese as something that is fundamentally and profoundly different from everyone else in the world. There is We the Japanese (and that doesn t mean nationality of passport, but blood line!), and there are They the Rest or, as the way of writing and and description on many occasion suggestions, They the Barbarians . A short anecdote will underly this: One of the favorite TV talk show / pseudo-documentary style is about Japanese living abroad. That day a lady married in Paris was interviewed. What followed was a guided tour through Paris showing: Dirt in the gutter, street cleaning cars, waste disposal places. Yes, that was all. Just about the dirt . Of course, at length the (unfortunately only apparent) cleanliness of Japanese cities and neighborhoods are mentioned and shown to remind everyone how wonderful Japan is and how dirty the Barbarians. I don t want to say that I consider Japan more dirty than most other countries just the visible part is clean, the moment you step a bit aside and around the corner, there are the worst trash just thrown away without consideration. Anyway. To return to the topic of Us and They I consider us all humans, first and foremost, and nationality, birthplace, and all that are just by chance. I do NOT reject cultural differences, they are here, of course. But cultural differences are one thing, separating one self and one s perceived people from the rest of the world is another. Conclusion I repeat, I don t think that the author had any ill intentions, but it would have been nicer if the article wouldn t make such a stark distinction. He could have written about Shishiodoshi and water fountains without using the Us They categorization. He could have compared other water installations, could have discussed the long tradition of small ponds in European gardens, just to name a few things. But the author choose to highlight differences instead of commonalities. It is the Us against Them feeling that often makes life in Japan for a foreigner difficult. Japanese are not special, Austrians, too, are not special, nor are Americans, Russians, Tibetans, or any other nationality. No nationality is special, we are all humans. Maybe at some point this will arrive also in the Japanese society and thinking.
Email this to someonePrint this pageShare on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInFlattr the author

19 May 2015

Thorsten Alteholz: alpine and UTF-8 and Debian lists

This is a note for my future self: When writing an email with only charset=US-ASCII , alpine creates an email with:

Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII

and everything is fine. In case of UTF-8 characters inside the text, alpine creates something like:

Content-Type: MULTIPART/MIXED; BOUNDARY="705298698-1667814148-1432049085=:28313"

and the only available part contains:

Content-Type: TEXT/PLAIN; format=flowed; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Google tells me that the reason for this is:
Alpine uses a single part MULTIPART/MIXED to apply a protection wrapper around QUOTED-PRINTABLE and BASE64 content to prevent it from being corrupted by various mail delivery systems that append little (typically advertising) things at the end of the message.
Ok, this behavior might come from bad experiences and it seems to work most of the time. Unfortunately if one sends a signed email to a Debian list that checks whether the signature is valid (like for example debian-lts-announce), such an email will be rejected with:
Failed to understand the email or find a signature: UDFormatError:
Cannot handle multipart messages not of type multipart/signed
*sigh*

Ritesh Raj Sarraf: Lenovo Yoga 2 13 with Debian

I recently acquired a Lenovo Yoga 2 13. While, at the time, the Yoga 3 was available, I decided to go for Yoga 2 13. The Yoga 3 comes with the newer Core M Broadwell family, which, in my opinion, doesn't really bring any astounding benefits. The Yoga 2 13 comes in mulitple variants worldwide. Infact these hardware variations have different effets when run under Linux. My varaint of Yoga 2 13 is:
CPU: Intel(R) Core(TM) i7-4510U CPU @ 2.00GHz
RAM: 8 GiB - Occupying 2 slots
Memory Controller Information
        Supported Interleave: One-way Interleave
        Current Interleave: One-way Interleave
        Maximum Memory Module Size: 8192 MB
        Maximum Total Memory Size: 16384 MB
       
Handle 0x0006, DMI type 6, 12 bytes
Handle 0x0007, DMI type 6, 12 bytes
The usual PCI devices:
rrs@learner:~$ lspci
00:00.0 Host bridge: Intel Corporation Haswell-ULT DRAM Controller (rev 0b)
00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated Graphics Controller (rev 0b)
00:03.0 Audio device: Intel Corporation Haswell-ULT HD Audio Controller (rev 0b)
00:14.0 USB controller: Intel Corporation 8 Series USB xHCI HC (rev 04)
00:16.0 Communication controller: Intel Corporation 8 Series HECI #0 (rev 04)
00:1b.0 Audio device: Intel Corporation 8 Series HD Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 8 Series PCI Express Root Port 4 (rev e4)
00:1d.0 USB controller: Intel Corporation 8 Series USB EHCI #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation 8 Series LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 8 Series SATA Controller 1 [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 8 Series SMBus Controller (rev 04)
01:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8723BE PCIe Wireless Network Adapter
17:37          
And the storage devices
Device Model:     WDC WD5000M22K-24Z1LT0-SSHD-16GB
Device Model:     KINGSTON SM2280S3120G
Storage

The drive runs into serious performance problems when its SSHD's NCQ (mis)feature is under use in Linux <= 4.0.

[28974.232550] ata2.00: configured for UDMA/133
[28974.232565] ahci 0000:00:1f.2: port does not support device sleep
[28983.680955] ata1.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x400100 action 0x6 frozen
[28983.681000] ata1.00: irq_stat 0x08000000, interface fatal error
[28983.681027] ata1: SError:   UnrecovData Handshk  
[28983.681052] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.681082] ata1.00: cmd 61/40:00:b8:84:88/05:00:0a:00:00/40 tag 0 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.681152] ata1.00: status:   DRDY  
[28983.681171] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.681202] ata1.00: cmd 61/40:08:f8:89:88/05:00:0a:00:00/40 tag 1 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.681271] ata1.00: status:   DRDY  
[28983.681289] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.681316] ata1.00: cmd 61/40:10:38:8f:88/05:00:0a:00:00/40 tag 2 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.681387] ata1.00: status:   DRDY  
[28983.681407] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.681435] ata1.00: cmd 61/40:18:78:94:88/05:00:0a:00:00/40 tag 3 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697642] ata1.00: status:   DRDY  
[28983.697643] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697646] ata1.00: cmd 61/40:c8:38:65:88/05:00:0a:00:00/40 tag 25 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697647] ata1.00: status:   DRDY  
[28983.697648] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697651] ata1.00: cmd 61/40:d0:78:6a:88/05:00:0a:00:00/40 tag 26 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697651] ata1.00: status:   DRDY  
[28983.697652] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697656] ata1.00: cmd 61/40:d8:b8:6f:88/05:00:0a:00:00/40 tag 27 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697657] ata1.00: status:   DRDY  
[28983.697658] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697661] ata1.00: cmd 61/40:e0:f8:74:88/05:00:0a:00:00/40 tag 28 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697662] ata1.00: status:   DRDY  
[28983.697663] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697666] ata1.00: cmd 61/40:e8:38:7a:88/05:00:0a:00:00/40 tag 29 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697667] ata1.00: status:   DRDY  
[28983.697668] ata1.00: failed command: WRITE FPDMA QUEUED
[28983.697672] ata1.00: cmd 61/40:f0:78:7f:88/05:00:0a:00:00/40 tag 30 ncq 688128 out
                        res 40/00:3c:78:a9:88/00:00:0a:00:00/40 Emask 0x10 (ATA bus error)
[28983.697672] ata1.00: status:   DRDY  
[28983.697676] ata1: hard resetting link
[28984.017356] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[28984.022612] ata1.00: configured for UDMA/133
[28984.022740] ata1: EH complete
[28991.611732] Suspending console(s) (use no_console_suspend to debug)
[28992.183822] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
[28992.186569] sd 1:0:0:0: [sdb] Stopping disk
[28992.186604] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[28992.189594] sd 0:0:0:0: [sda] Stopping disk
[28992.967426] PM: suspend of devices complete after 1351.349 msecs
[28992.999461] PM: late suspend of devices complete after 31.990 msecs
[28993.000058] ehci-pci 0000:00:1d.0: System wakeup enabled by ACPI
[28993.000306] xhci_hcd 0000:00:14.0: System wakeup enabled by ACPI
[28993.016463] PM: noirq suspend of devices complete after 16.978 msecs
[28993.017024] ACPI: Preparing to enter system sleep state S3
[28993.017349] PM: Saving platform NVS memory
[28993.017357] Disabling non-boot CPUs ...
[28993.017389] intel_pstate CPU 1 exiting
[28993.018727] kvm: disabling virtualization on CPU1
[28993.019320] smpboot: CPU 1 is now offline
[28993.019646] intel_pstate CPU 2 exiting

In the interim, to overcome this problem, we can force the device to run in degraded mode. I'm not sure if it is really the degraded mode, or the device was falsely advertised as a 6 GiB capable device. Time will tell, but for now, force it to run in 3 GiB mode, and so far, I haven't run into the above mentioned probems. To force 3 GiB speed, apply the following.

rrs@learner:~$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.0.4+ root=/dev/mapper/sdb_crypt ro cgroup_enable=memory swapaccount=1 rootflags=data=writeback libata.force=1:3 quiet
16:42          

And then verify it... As you can see below, I've forced it for ata1 because I want my SSD drive to run at full-speed. I've done enough I/O, which earlier resulted in the kernel spitting the SATA errors. With this workaround, the kernel does not spit any error messages.

[    1.273365] libata version 3.00 loaded.
[    1.287290] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0x3 impl SATA mode
[    1.288238] ata1: FORCE: PHY spd limit set to 3.0Gbps
[    1.288240] ata1: SATA max UDMA/133 abar m2048@0xb051b000 port 0xb051b100 irq 41
[    1.288242] ata2: SATA max UDMA/133 abar m2048@0xb051b000 port 0xb051b180 irq 41
[    1.288244] ata3: DUMMY
[    1.288245] ata4: DUMMY
[    1.606971] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[    1.607906] ata1.00: ATA-9: WDC WD5000M22K-24Z1LT0-SSHD-16GB, 02.01A03, max UDMA/133
[    1.607910] ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    1.608856] ata1.00: configured for UDMA/133
[    1.609106] scsi 0:0:0:0: Direct-Access     ATA      WDC WD5000M22K-2 1A03 PQ: 0 ANSI: 5
[    1.927167] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    1.928980] ata2.00: ATA-8: KINGSTON SM2280S3120G, S8FM06.A, max UDMA/133
[    1.928983] ata2.00: 234441648 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    1.929616] ata2.00: configured for UDMA/133

And the throughput you get out of your WD SATA SSHD drive, with capability set to 3.0 GiB is:

rrs@learner:/media/SSHD/tmp$ while true; do dd if=/dev/zero of=foo.img bs=1M count=20000; sync; rm -rf foo.img; sync; done
20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 202.014 s, 104 MB/s
20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 206.111 s, 102 MB/s

Hannes Reinecke has submitted patches for NCQ enhancements, for Linux 4.1, which I hope will resolve these problems. Another option is to disable NCQ for the drive, or else blacklist the make/model in driver/ata/libata-core.c

By the time I finished this blog entry draft, I had tests to conclude that this did not look like an NCQ problem. Because in degraded mode too, it runs with NCQ enabled (check above).

rrs@learner:~$ sudo fstrim -vv /media/SSHD
/media/SSHD: 268.2 GiB (287930949632 bytes) trimmed
16:58          
rrs@learner:~$ sudo fstrim -vv /
[sudo] password for rrs:
/: 64 GiB (68650749952 bytes) trimmed
16:56          

Another interesting feature of this drive is support for TRIM / DISCARD. This drive's FTL accepts the TRIM command. Ofcourse, you need to ensure that you have discard enabled in all the layers. In my case, SATA + Device Mapper (Crypt and LVM) + File System (ext4) Display

The overall display of this device is amazing. It is large enough to give you vibrant look. At 1920x1080 resolution, things look good. The display support was available out-of-the-box.

There were some suspend / resume hangs that occured with kernels < 4.x, during suspend / resume. The issue was root caused and fixed for Linux 4.0.

You may still notice the following kernel messages, though not problematic to me so far.

[28977.518114] PM: thaw of devices complete after 3607.979 msecs
[28977.590389] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.590582] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.591095] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.591185] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.591368] acpi device:30: Cannot transition to power state D3cold for parent in (unknown)
[28977.591911] pci_bus 0000:01: Allocating resources
[28977.591933] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.592093] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[28977.592401] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment

You may need to disable the Intel Management Engine Interface (mei.ko), incase you run into suspend/resume problems.

rrs@learner:/media/SSHD/tmp$ cat /etc/modprobe.d/intel-mei-blacklist.conf
blacklist mei
blacklist mei-me
17:01          

You may also run into the following Kernel Oops during suspend/resume. Below, you see 2 interation of sleep because it first hibernates and then sleeps (s2both).

[  180.470206] Syncing filesystems ... done.
[  180.473337] Freezing user space processes ... (elapsed 0.001 seconds) done.
[  180.475210] PM: Marking nosave pages: [mem 0x00000000-0x00000fff]
[  180.475213] PM: Marking nosave pages: [mem 0x0006f000-0x0006ffff]
[  180.475215] PM: Marking nosave pages: [mem 0x00088000-0x000fffff]
[  180.475220] PM: Marking nosave pages: [mem 0x97360000-0x97b5ffff]
[  180.475274] PM: Marking nosave pages: [mem 0x9c36f000-0x9cffefff]
[  180.475356] PM: Marking nosave pages: [mem 0x9d000000-0xffffffff]
[  180.476877] PM: Basic memory bitmaps created
[  180.477003] PM: Preallocating image memory... done (allocated 380227 pages)
[  180.851800] PM: Allocated 1520908 kbytes in 0.37 seconds (4110.56 MB/s)
[  180.851802] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[  180.853355] Suspending console(s) (use no_console_suspend to debug)
[  180.853520] wlan0: deauthenticating from c4:6e:1f:d0:67:26 by local choice (Reason: 3=DEAUTH_LEAVING)
[  180.864159] cfg80211: Calling CRDA to update world regulatory domain
[  181.172222] PM: freeze of devices complete after 319.294 msecs
[  181.196080] ------------[ cut here ]------------
[  181.196124] WARNING: CPU: 3 PID: 3707 at drivers/gpu/drm/i915/intel_display.c:7904 hsw_enable_pc8+0x659/0x7c0 [i915]()
[  181.196125] SPLL enabled
[  181.196159] Modules linked in: rfcomm ctr ccm bnep pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) bridge stp llc xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM xt_tcpudp iptable_mangle ip_tables x_tables nls_utf8 nls_cp437 vfat fat rtsx_usb_ms memstick snd_hda_codec_hdmi joydev mousedev hid_sensor_rotation hid_sensor_incl_3d hid_sensor_als hid_sensor_accel_3d hid_sensor_magn_3d hid_sensor_gyro_3d hid_sensor_trigger industrialio_triggered_buffer kfifo_buf industrialio hid_sensor_iio_common iTCO_wdt iTCO_vendor_support hid_multitouch x86_pkg_temp_thermal intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm btusb hid_sensor_hub bluetooth uvcvideo videobuf2_vmalloc videobuf2_memops
[  181.196203]  videobuf2_core v4l2_common videodev media pcspkr evdev mac_hid arc4 psmouse serio_raw efivars i2c_i801 rtl8723be btcoexist rtl8723_common rtl_pci rtlwifi mac80211 snd_soc_rt5640 cfg80211 snd_soc_rl6231 snd_hda_codec_realtek i915 snd_soc_core snd_hda_codec_generic ideapad_laptop ac snd_compress dw_dmac sparse_keymap drm_kms_helper rfkill battery dw_dmac_core snd_hda_intel snd_pcm_dmaengine snd_soc_sst_acpi snd_hda_controller video 8250_dw regmap_i2c snd_hda_codec drm snd_hwdep snd_pcm spi_pxa2xx_platform i2c_designware_platform soc_button_array snd_timer i2c_designware_core snd i2c_algo_bit soundcore shpchp lpc_ich button processor fuse ipv6 autofs4 ext4 crc16 jbd2 mbcache btrfs xor raid6_pq algif_skcipher af_alg dm_crypt dm_mod sg usbhid sd_mod rtsx_usb_sdmmc rtsx_usb crct10dif_pclmul
[  181.196220]  crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ahci libahci libata xhci_pci ehci_pci xhci_hcd ehci_hcd scsi_mod usbcore usb_common thermal fan thermal_sys hwmon i2c_hid hid i2c_core sdhci_acpi sdhci mmc_core gpio_lynxpoint
[  181.196224] CPU: 3 PID: 3707 Comm: kworker/u16:7 Tainted: G           O    4.0.4+ #14
[  181.196225] Hardware name: LENOVO 20344/INVALID, BIOS 96CN29WW(V1.15) 10/16/2014
[  181.196230] Workqueue: events_unbound async_run_entry_fn
[  181.196233]  0000000000000000 ffffffffa0706f68 ffffffff81522198 ffff880064debc88
[  181.196235]  ffffffff8106c5b1 ffff880251460000 ffff880250f83b68 ffff880250f83b78
[  181.196237]  ffff880250f83800 0000000000000001 ffffffff8106c62a ffffffffa071407c
[  181.196238] Call Trace:
[  181.196248]  [<ffffffff81522198>] ? dump_stack+0x40/0x50
[  181.196251]  [<ffffffff8106c5b1>] ? warn_slowpath_common+0x81/0xb0
[  181.196254]  [<ffffffff8106c62a>] ? warn_slowpath_fmt+0x4a/0x50
[  181.196278]  [<ffffffffa06ae349>] ? hsw_enable_pc8+0x659/0x7c0 [i915]
[  181.196289]  [<ffffffffa0643ee0>] ? intel_suspend_complete+0xe0/0x6e0 [i915]
[  181.196300]  [<ffffffffa0644501>] ? i915_drm_suspend_late+0x21/0x90 [i915]
[  181.196311]  [<ffffffffa0644690>] ? i915_pm_poweroff_late+0x40/0x40 [i915]
[  181.196318]  [<ffffffff813fa7ba>] ? dpm_run_callback+0x4a/0x100
[  181.196321]  [<ffffffff813fb010>] ? __device_suspend_late+0xa0/0x180
[  181.196324]  [<ffffffff813fb10e>] ? async_suspend_late+0x1e/0xa0
[  181.196326]  [<ffffffff8108b973>] ? async_run_entry_fn+0x43/0x160
[  181.196330]  [<ffffffff81083a5d>] ? process_one_work+0x14d/0x3f0
[  181.196332]  [<ffffffff81084463>] ? worker_thread+0x53/0x480
[  181.196334]  [<ffffffff81084410>] ? rescuer_thread+0x300/0x300
[  181.196338]  [<ffffffff81089191>] ? kthread+0xc1/0xe0
[  181.196341]  [<ffffffff810890d0>] ? kthread_create_on_node+0x180/0x180
[  181.196346]  [<ffffffff81527898>] ? ret_from_fork+0x58/0x90
[  181.196349]  [<ffffffff810890d0>] ? kthread_create_on_node+0x180/0x180
[  181.196350] ---[ end trace 8e339004db298838 ]---
[  181.220094] PM: late freeze of devices complete after 47.936 msecs
[  181.220972] PM: noirq freeze of devices complete after 0.875 msecs
[  181.221577] ACPI: Preparing to enter system sleep state S4
[  181.221886] PM: Saving platform NVS memory
[  181.222702] Disabling non-boot CPUs ...
[  181.222731] intel_pstate CPU 1 exiting
[  181.224041] kvm: disabling virtualization on CPU1
[  181.224680] smpboot: CPU 1 is now offline
[  181.225121] intel_pstate CPU 2 exiting
[  181.226407] kvm: disabling virtualization on CPU2
[  181.227025] smpboot: CPU 2 is now offline
[  181.227441] intel_pstate CPU 3 exiting
[  181.227728] Broke affinity for irq 19
[  181.227747] Broke affinity for irq 41
[  181.228771] kvm: disabling virtualization on CPU3
[  181.228793] smpboot: CPU 3 is now offline
[  181.229624] PM: Creating hibernation image:
[  181.563651] PM: Need to copy 379053 pages
[  181.563655] PM: Normal pages needed: 379053 + 1024, available pages: 1697704
[  182.472910] PM: Hibernation image created (379053 pages copied)
[  181.232347] PM: Restoring platform NVS memory
[  181.233171] Enabling non-boot CPUs ...
[  181.233246] x86: Booting SMP configuration:
[  181.233248] smpboot: Booting Node 0 Processor 1 APIC 0x1
[  181.246771] kvm: enabling virtualization on CPU1
[  181.249339] CPU1 is up
[  181.249389] smpboot: Booting Node 0 Processor 2 APIC 0x2
[  181.262313] kvm: enabling virtualization on CPU2
[  181.264853] CPU2 is up
[  181.264903] smpboot: Booting Node 0 Processor 3 APIC 0x3
[  181.277831] kvm: enabling virtualization on CPU3
[  181.280317] CPU3 is up
[  181.288471] ACPI: Waking up from system sleep state S4
[  182.340655] PM: noirq thaw of devices complete after 0.637 msecs
[  182.378087] PM: early thaw of devices complete after 37.428 msecs
[  182.378436] rtlwifi: rtlwifi: wireless switch is on
[  182.451021] rtc_cmos 00:01: System wakeup disabled by ACPI
[  182.697575] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[  182.697617] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  182.699248] ata1.00: configured for UDMA/133
[  182.699911] ata2.00: configured for UDMA/133
[  182.699917] ahci 0000:00:1f.2: port does not support device sleep
[  186.059539] PM: thaw of devices complete after 3685.338 msecs
[  186.134292] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.134479] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.134992] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.135080] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.135266] acpi device:30: Cannot transition to power state D3cold for parent in (unknown)
[  186.135950] pci_bus 0000:01: Allocating resources
[  186.135974] pcieport 0000:00:1c.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01] add_size 200000
[  186.135980] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.136049] pcieport 0000:00:1c.0: res[15]=[mem 0x00100000-0x000fffff 64bit pref] get_res_add_size add_size 200000
[  186.136072] pcieport 0000:00:1c.0: BAR 15: assigned [mem 0x9fb00000-0x9fcfffff 64bit pref]
[  186.136174] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  186.136490] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  199.454497] Suspending console(s) (use no_console_suspend to debug)
[  200.024190] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
[  200.024356] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[  200.025359] sd 1:0:0:0: [sdb] Stopping disk
[  200.028701] sd 0:0:0:0: [sda] Stopping disk
[  201.106085] PM: suspend of devices complete after 1651.336 msecs
[  201.106591] ------------[ cut here ]------------
[  201.106628] WARNING: CPU: 0 PID: 3725 at drivers/gpu/drm/i915/intel_display.c:7904 hsw_enable_pc8+0x659/0x7c0 [i915]()
[  201.106628] SPLL enabled
[  201.106656] Modules linked in: rfcomm ctr ccm bnep pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) bridge stp llc xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM xt_tcpudp iptable_mangle ip_tables x_tables nls_utf8 nls_cp437 vfat fat rtsx_usb_ms memstick snd_hda_codec_hdmi joydev mousedev hid_sensor_rotation hid_sensor_incl_3d hid_sensor_als hid_sensor_accel_3d hid_sensor_magn_3d hid_sensor_gyro_3d hid_sensor_trigger industrialio_triggered_buffer kfifo_buf industrialio hid_sensor_iio_common iTCO_wdt iTCO_vendor_support hid_multitouch x86_pkg_temp_thermal intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm btusb hid_sensor_hub bluetooth uvcvideo videobuf2_vmalloc videobuf2_memops
[  201.106694]  videobuf2_core v4l2_common videodev media pcspkr evdev mac_hid arc4 psmouse serio_raw efivars i2c_i801 rtl8723be btcoexist rtl8723_common rtl_pci rtlwifi mac80211 snd_soc_rt5640 cfg80211 snd_soc_rl6231 snd_hda_codec_realtek i915 snd_soc_core snd_hda_codec_generic ideapad_laptop ac snd_compress dw_dmac sparse_keymap drm_kms_helper rfkill battery dw_dmac_core snd_hda_intel snd_pcm_dmaengine snd_soc_sst_acpi snd_hda_controller video 8250_dw regmap_i2c snd_hda_codec drm snd_hwdep snd_pcm spi_pxa2xx_platform i2c_designware_platform soc_button_array snd_timer i2c_designware_core snd i2c_algo_bit soundcore shpchp lpc_ich button processor fuse ipv6 autofs4 ext4 crc16 jbd2 mbcache btrfs xor raid6_pq algif_skcipher af_alg dm_crypt dm_mod sg usbhid sd_mod rtsx_usb_sdmmc rtsx_usb crct10dif_pclmul
[  201.106711]  crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ahci libahci libata xhci_pci ehci_pci xhci_hcd ehci_hcd scsi_mod usbcore usb_common thermal fan thermal_sys hwmon i2c_hid hid i2c_core sdhci_acpi sdhci mmc_core gpio_lynxpoint
[  201.106714] CPU: 0 PID: 3725 Comm: kworker/u16:25 Tainted: G        W  O    4.0.4+ #14
[  201.106715] Hardware name: LENOVO 20344/INVALID, BIOS 96CN29WW(V1.15) 10/16/2014
[  201.106720] Workqueue: events_unbound async_run_entry_fn
[  201.106723]  0000000000000000 ffffffffa0706f68 ffffffff81522198 ffff880064dd7c88
[  201.106725]  ffffffff8106c5b1 ffff880251460000 ffff880250f83b68 ffff880250f83b78
[  201.106727]  ffff880250f83800 0000000000000002 ffffffff8106c62a ffffffffa071407c
[  201.106728] Call Trace:
[  201.106737]  [<ffffffff81522198>] ? dump_stack+0x40/0x50
[  201.106740]  [<ffffffff8106c5b1>] ? warn_slowpath_common+0x81/0xb0
[  201.106742]  [<ffffffff8106c62a>] ? warn_slowpath_fmt+0x4a/0x50
[  201.106765]  [<ffffffffa06ae349>] ? hsw_enable_pc8+0x659/0x7c0 [i915]
[  201.106776]  [<ffffffffa0643ee0>] ? intel_suspend_complete+0xe0/0x6e0 [i915]
[  201.106786]  [<ffffffffa0644501>] ? i915_drm_suspend_late+0x21/0x90 [i915]
[  201.106797]  [<ffffffffa0644690>] ? i915_pm_poweroff_late+0x40/0x40 [i915]
[  201.106802]  [<ffffffff813fa7ba>] ? dpm_run_callback+0x4a/0x100
[  201.106805]  [<ffffffff813fb010>] ? __device_suspend_late+0xa0/0x180
[  201.106809]  [<ffffffff813fb10e>] ? async_suspend_late+0x1e/0xa0
[  201.106811]  [<ffffffff8108b973>] ? async_run_entry_fn+0x43/0x160
[  201.106813]  [<ffffffff81083a5d>] ? process_one_work+0x14d/0x3f0
[  201.106815]  [<ffffffff81084463>] ? worker_thread+0x53/0x480
[  201.106818]  [<ffffffff81084410>] ? rescuer_thread+0x300/0x300
[  201.106821]  [<ffffffff81089191>] ? kthread+0xc1/0xe0
[  201.106824]  [<ffffffff810890d0>] ? kthread_create_on_node+0x180/0x180
[  201.106827]  [<ffffffff81527898>] ? ret_from_fork+0x58/0x90
[  201.106830]  [<ffffffff810890d0>] ? kthread_create_on_node+0x180/0x180
[  201.106832] ---[ end trace 8e339004db298839 ]---
[  201.130052] PM: late suspend of devices complete after 23.960 msecs
[  201.130725] ehci-pci 0000:00:1d.0: System wakeup enabled by ACPI
[  201.130885] xhci_hcd 0000:00:14.0: System wakeup enabled by ACPI
[  201.146986] PM: noirq suspend of devices complete after 16.930 msecs
[  201.147591] ACPI: Preparing to enter system sleep state S3
[  201.147942] PM: Saving platform NVS memory
[  201.147948] Disabling non-boot CPUs ...
[  201.147999] intel_pstate CPU 1 exiting
[  201.149324] kvm: disabling virtualization on CPU1
[  201.149337] smpboot: CPU 1 is now offline
[  201.149640] intel_pstate CPU 2 exiting
[  201.151096] kvm: disabling virtualization on CPU2
[  201.151108] smpboot: CPU 2 is now offline
[  201.152017] intel_pstate CPU 3 exiting
[  201.153250] kvm: disabling virtualization on CPU3
[  201.153256] smpboot: CPU 3 is now offline
[  201.156229] ACPI: Low-level resume complete
[  201.156307] PM: Restoring platform NVS memory
[  201.160033] CPU0 microcode updated early to revision 0x1c, date = 2014-07-03
[  201.160190] Enabling non-boot CPUs ...
[  201.160241] x86: Booting SMP configuration:
[  201.160243] smpboot: Booting Node 0 Processor 1 APIC 0x1
[  201.172665] kvm: enabling virtualization on CPU1
[  201.174982] CPU1 is up
[  201.175013] smpboot: Booting Node 0 Processor 2 APIC 0x2
[  201.187569] CPU2 microcode updated early to revision 0x1c, date = 2014-07-03
[  201.188796] kvm: enabling virtualization on CPU2
[  201.191130] CPU2 is up
[  201.191158] smpboot: Booting Node 0 Processor 3 APIC 0x3
[  201.203297] kvm: enabling virtualization on CPU3
[  201.205679] CPU3 is up
[  201.210414] ACPI: Waking up from system sleep state S3
[  201.224617] ehci-pci 0000:00:1d.0: System wakeup disabled by ACPI
[  201.332523] xhci_hcd 0000:00:14.0: System wakeup disabled by ACPI
[  201.332634] PM: noirq resume of devices complete after 121.623 msecs
[  201.372718] PM: early resume of devices complete after 40.058 msecs
[  201.372892] rtlwifi: rtlwifi: wireless switch is on
[  201.373270] sd 0:0:0:0: [sda] Starting disk
[  201.373271] sd 1:0:0:0: [sdb] Starting disk
[  201.445954] rtc_cmos 00:01: System wakeup disabled by ACPI
[  201.692510] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  201.694719] ata2.00: configured for UDMA/133
[  201.694724] ahci 0000:00:1f.2: port does not support device sleep
[  201.836724] usb 2-4: reset high-speed USB device number 2 using xhci_hcd
[  201.890158] psmouse serio1: synaptics: queried max coordinates: x [..5702], y [..4730]
[  201.930768] psmouse serio1: synaptics: queried min coordinates: x [1242..], y [1124..]
[  202.076784] usb 2-5: reset full-speed USB device number 3 using xhci_hcd
[  202.205100] usb 2-5: ep 0x2 - rounding interval to 64 microframes, ep desc says 80 microframes
[  202.316799] usb 2-7: reset full-speed USB device number 5 using xhci_hcd
[  202.444945] usb 2-7: No LPM exit latency info found, disabling LPM.
[  202.556817] usb 2-8: reset full-speed USB device number 6 using xhci_hcd
[  202.908691] usb 2-6: reset high-speed USB device number 4 using xhci_hcd
[  203.932602] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[  204.044890] ata1.00: configured for UDMA/133
[  206.228698] PM: resume of devices complete after 4855.892 msecs
[  206.380738] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.383152] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.385775] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.388066] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.390415] acpi device:30: Cannot transition to power state D3cold for parent in (unknown)
[  206.393078] pci_bus 0000:01: Allocating resources
[  206.393098] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.395470] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.397927] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2] has bogus alignment
[  206.518516] Restarting kernel threads ... done.
[  206.518812] PM: Basic memory bitmaps freed
[  206.518816] Restarting tasks ... done.

There is one more occasional Kernel Oops (below), which I believe again has to do with Intel.

[ 8770.745396] ------------[ cut here ]------------
[ 8770.745441] WARNING: CPU: 0 PID: 7206 at drivers/gpu/drm/i915/intel_display.c:9756 intel_check_page_flip+0xd2/0xe0 [i915]()
[ 8770.745444] Kicking stuck page flip: queued at 466186, now 466191
[ 8770.745445] Modules linked in: cpuid rfcomm ctr ccm bnep pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) bridge stp llc xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM xt_tcpudp iptable_mangle ip_tables x_tables nls_utf8 nls_cp437 vfat fat rtsx_usb_ms memstick snd_hda_codec_hdmi joydev mousedev hid_sensor_rotation hid_sensor_incl_3d hid_sensor_als hid_sensor_accel_3d hid_sensor_magn_3d hid_sensor_gyro_3d hid_sensor_trigger industrialio_triggered_buffer kfifo_buf industrialio hid_sensor_iio_common iTCO_wdt iTCO_vendor_support hid_multitouch x86_pkg_temp_thermal intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm btusb hid_sensor_hub bluetooth uvcvideo videobuf2_vmalloc videobuf2_memops
[ 8770.745484]  videobuf2_core v4l2_common videodev media pcspkr evdev mac_hid arc4 psmouse serio_raw efivars i2c_i801 rtl8723be btcoexist rtl8723_common rtl_pci rtlwifi mac80211 snd_soc_rt5640 cfg80211 snd_soc_rl6231 snd_hda_codec_realtek i915 snd_soc_core snd_hda_codec_generic ideapad_laptop ac snd_compress dw_dmac sparse_keymap drm_kms_helper rfkill battery dw_dmac_core snd_hda_intel snd_pcm_dmaengine snd_soc_sst_acpi snd_hda_controller video 8250_dw regmap_i2c snd_hda_codec drm snd_hwdep snd_pcm spi_pxa2xx_platform i2c_designware_platform soc_button_array snd_timer i2c_designware_core snd i2c_algo_bit soundcore shpchp lpc_ich button processor fuse ipv6 autofs4 ext4 crc16 jbd2 mbcache btrfs xor raid6_pq algif_skcipher af_alg dm_crypt dm_mod sg usbhid sd_mod rtsx_usb_sdmmc rtsx_usb crct10dif_pclmul
[ 8770.745536]  crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ahci libahci libata xhci_pci ehci_pci xhci_hcd ehci_hcd scsi_mod usbcore usb_common thermal fan thermal_sys hwmon i2c_hid hid i2c_core sdhci_acpi sdhci mmc_core gpio_lynxpoint
[ 8770.745561] CPU: 0 PID: 7206 Comm: icedove Tainted: G        W  O    4.0.4+ #14
[ 8770.745563] Hardware name: LENOVO 20344/INVALID, BIOS 96CN29WW(V1.15) 10/16/2014
[ 8770.745565]  0000000000000000 ffffffffa0706f68 ffffffff81522198 ffff88025f203dc8
[ 8770.745569]  ffffffff8106c5b1 ffff880250f83800 ffff880254dcc000 0000000000000000
[ 8770.745572]  0000000000000000 0000000000000000 ffffffff8106c62a ffffffffa0709d50
[ 8770.745575] Call Trace:
[ 8770.745577]  <IRQ>  [<ffffffff81522198>] ? dump_stack+0x40/0x50
[ 8770.745592]  [<ffffffff8106c5b1>] ? warn_slowpath_common+0x81/0xb0
[ 8770.745595]  [<ffffffff8106c62a>] ? warn_slowpath_fmt+0x4a/0x50
[ 8770.745616]  [<ffffffffa06a0bb3>] ? __intel_pageflip_stall_check+0x113/0x120 [i915]
[ 8770.745634]  [<ffffffffa06af042>] ? intel_check_page_flip+0xd2/0xe0 [i915]
[ 8770.745652]  [<ffffffffa067cde1>] ? ironlake_irq_handler+0x2e1/0x1010 [i915]
[ 8770.745657]  [<ffffffff81092d1a>] ? check_preempt_curr+0x5a/0xa0
[ 8770.745663]  [<ffffffff812d66c2>] ? timerqueue_del+0x22/0x70
[ 8770.745668]  [<ffffffff810bb7d5>] ? handle_irq_event_percpu+0x75/0x190
[ 8770.745672]  [<ffffffff8101b945>] ? read_tsc+0x5/0x10
[ 8770.745676]  [<ffffffff810bb928>] ? handle_irq_event+0x38/0x50
[ 8770.745680]  [<ffffffff810be841>] ? handle_edge_irq+0x71/0x120
[ 8770.745685]  [<ffffffff810153bd>] ? handle_irq+0x1d/0x30
[ 8770.745689]  [<ffffffff8152a866>] ? do_IRQ+0x46/0xe0
[ 8770.745694]  [<ffffffff8152866d>] ? common_interrupt+0x6d/0x6d
[ 8770.745695]  <EOI>  [<ffffffff8152794d>] ? system_call_fastpath+0x16/0x1b
[ 8770.745701] ---[ end trace 8e339004db29883a ]---
Network

In my case, the laptop came with the Realtek Wireless device (details above in lspci output). Note: The machine has no wired interface.

While the Intel Wifi devices shipped with this laptop have their own share of problems, this device (rtl8723be) works out of the box. But only for a while. There is no certain pattern on what triggers the bug, but once triggered, the network just freezes. Nothing is logged.

If your Yoga 2 13 came with the RTL chip, the following workaround may help avoid the network issues.

rrs@learner:/media/SSHD/tmp$ cat /etc/modprobe.d/rtl8723be.conf
options rtl8723be fwlps=0
17:06          
MCE

Almost every boot, eventually, the kernel reports MCE errors. Not something I understand well, but so far, it hasn't caused any visible issues. And from what I have googled so far, nobody seems to have fixed it anywhere

So, with fingers crossed, lets just hope this never translates into a real problem.

What the kernel reports of the CPU's capabilities.

[    0.041496] mce: CPU supports 7 MCE banks
[  299.540930] mce: [Hardware Error]: Machine check events logged

The MCE logs extracted from the buffer.

mcelog: failed to prefill DIMM database from DMI data
Hardware event. This is not a software error.
MCE 0
CPU 0 BANK 5
MISC 38a0000086 ADDR fef81880
TIME 1432455005 Sun May 24 13:40:05 2015
MCG status:
MCi status:
Error overflow
Uncorrected error
MCi_MISC register valid
MCi_ADDR register valid
Processor context corrupt
MCA: corrected filtering (some unreported errors in same region)
Generic CACHE Level-2 Generic Error
STATUS ee0000000040110a MCGSTATUS 0
MCGCAP c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 69
Hardware event. This is not a software error.
MCE 1
CPU 0 BANK 6
MISC 78a0000086 ADDR fef81780
TIME 1432455005 Sun May 24 13:40:05 2015
MCG status:
MCi status:
Uncorrected error
MCi_MISC register valid
MCi_ADDR register valid
Processor context corrupt
MCA: corrected filtering (some unreported errors in same region)
Generic CACHE Level-2 Generic Error
STATUS ae0000000040110a MCGSTATUS 0
MCGCAP c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 69
Hardware event. This is not a software error.
MCE 2
CPU 0 BANK 5
MISC 38a0000086 ADDR fef81880
TIME 1432455114 Sun May 24 13:41:54 2015
MCG status:
MCi status:
Error overflow
Uncorrected error
MCi_MISC register valid
MCi_ADDR register valid
Processor context corrupt
MCA: corrected filtering (some unreported errors in same region)
Generic CACHE Level-2 Generic Error
STATUS ee0000000040110a MCGSTATUS 0
MCGCAP c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 69
Hardware event. This is not a software error.
MCE 3
CPU 0 BANK 6
MISC 78a0000086 ADDR fef81780
TIME 1432455114 Sun May 24 13:41:54 2015
MCG status:
MCi status:
Uncorrected error
MCi_MISC register valid
MCi_ADDR register valid
Processor context corrupt
MCA: corrected filtering (some unreported errors in same region)
Generic CACHE Level-2 Generic Error
STATUS ae0000000040110a MCGSTATUS 0
MCGCAP c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 69

Categories:

Keywords:

18 May 2015

Dirk Eddelbuettel: random 0.2.4

A new release of our random package for truly (hardware-based) random numbers as provided by random.org is now on CRAN. The R 3.2.0 release brought the change to use an internal method="libcurl" which we are using if available; else the curl::curl() method added in release 0.2.3 is used. We are also a little more explicit about closing connections, and added really basic regression tests -- as it is hard to test hardware-based RNGs draws. Courtesy of CRANberries comes a diffstat report for this release. Current and previous releases are available here as well as on CRAN.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

17 May 2015

Lunar: Reproducible builds: week 2 in Stretch cycle

What happened about the reproducible builds effort for this week: Media coverage Debian's effort on reproducible builds has been covered in the June 2015 issue of Linux Magazin in Germany. Cover of Linux Magazin June 2015 Article about reproducible builds in Linux Magazin June 2015 Toolchain fixes josch rebased the experimental version of debhelper on 9.20150507. Packages fixed The following 515 packages became reproducible due to changes of their build dependencies: airport-utils, airspy-host, all-in-one-sidebar, ampache, aptfs, arpack, asciio, aspell-kk, asused, balance, batmand, binutils-avr, bioperl, bpm-tools, c2050, cakephp-instaweb, carton, cbp2make, checkbot, checksecurity, chemeq, chronicle, cube2-data, cucumber, darkstat, debci, desktop-file-utils, dh-linktree, django-pagination, dosbox, eekboek, emboss-explorer, encfs, exabgp, fbasics, fife, fonts-lexi-saebom, gdnsd, glances, gnome-clocks, gunicorn, haproxy, haskell-aws, haskell-base-unicode-symbols, haskell-base64-bytestring, haskell-basic-prelude, haskell-binary-shared, haskell-binary, haskell-bitarray, haskell-bool-extras, haskell-boolean, haskell-boomerang, haskell-bytestring-lexing, haskell-bytestring-mmap, haskell-config-value, haskell-mueval, haskell-tasty-kat, itk3, jnr-constants, jshon, kalternatives, kdepim-runtime, kdevplatform, kwalletcli, lemonldap-ng, libalgorithm-combinatorics-perl, libalgorithm-diff-xs-perl, libany-uri-escape-perl, libanyevent-http-scopedclient-perl, libanyevent-perl, libanyevent-processor-perl, libapache-session-wrapper-perl, libapache-sessionx-perl, libapp-options-perl, libarch-perl, libarchive-peek-perl, libaudio-flac-header-perl, libaudio-wav-perl, libaudio-wma-perl, libauth-yubikey-decrypter-perl, libauthen-krb5-simple-perl, libauthen-simple-perl, libautobox-dump-perl, libb-keywords-perl, libbarcode-code128-perl, libbio-das-lite-perl, libbio-mage-perl, libbrowser-open-perl, libbusiness-creditcard-perl, libbusiness-edifact-interchange-perl, libbusiness-isbn-data-perl, libbusiness-tax-vat-validation-perl, libcache-historical-perl, libcache-memcached-perl, libcairo-gobject-perl, libcarp-always-perl, libcarp-fix-1-25-perl, libcatalyst-action-serialize-data-serializer-perl, libcatalyst-controller-formbuilder-perl, libcatalyst-dispatchtype-regex-perl, libcatalyst-plugin-authentication-perl, libcatalyst-plugin-authorization-acl-perl, libcatalyst-plugin-session-store-cache-perl, libcatalyst-plugin-session-store-fastmmap-perl, libcatalyst-plugin-static-simple-perl, libcatalyst-view-gd-perl, libcgi-application-dispatch-perl, libcgi-application-plugin-authentication-perl, libcgi-application-plugin-logdispatch-perl, libcgi-application-plugin-session-perl, libcgi-application-server-perl, libcgi-compile-perl, libcgi-xmlform-perl, libclass-accessor-classy-perl, libclass-accessor-lvalue-perl, libclass-accessor-perl, libclass-c3-adopt-next-perl, libclass-dbi-plugin-type-perl, libclass-field-perl, libclass-handle-perl, libclass-load-perl, libclass-ooorno-perl, libclass-prototyped-perl, libclass-returnvalue-perl, libclass-singleton-perl, libclass-std-fast-perl, libclone-perl, libconfig-auto-perl, libconfig-jfdi-perl, libconfig-simple-perl, libconvert-basen-perl, libconvert-ber-perl, libcpan-checksums-perl, libcpanplus-dist-build-perl, libcriticism-perl, libcrypt-cracklib-perl, libcrypt-dh-gmp-perl, libcrypt-mysql-perl, libcrypt-passwdmd5-perl, libcrypt-simple-perl, libcss-packer-perl, libcss-tiny-perl, libcurses-widgets-perl, libdaemon-control-perl, libdancer-plugin-database-perl, libdancer-session-cookie-perl, libdancer2-plugin-database-perl, libdata-format-html-perl, libdata-uuid-libuuid-perl, libdata-validate-domain-perl, libdate-jd-perl, libdate-simple-perl, libdatetime-astro-sunrise-perl, libdatetime-event-cron-perl, libdatetime-format-dbi-perl, libdatetime-format-epoch-perl, libdatetime-format-mail-perl, libdatetime-tiny-perl, libdatrie, libdb-file-lock-perl, libdbd-firebird-perl, libdbix-abstract-perl, libdbix-class-datetime-epoch-perl, libdbix-class-dynamicdefault-perl, libdbix-class-introspectablem2m-perl, libdbix-class-timestamp-perl, libdbix-connector-perl, libdbix-oo-perl, libdbix-searchbuilder-perl, libdbix-xml-rdb-perl, libdevel-stacktrace-ashtml-perl, libdigest-hmac-perl, libdist-zilla-plugin-emailnotify-perl, libemail-date-format-perl, libemail-mime-perl, libemail-received-perl, libemail-sender-perl, libemail-simple-perl, libencode-detect-perl, libexporter-tidy-perl, libextutils-cchecker-perl, libextutils-installpaths-perl, libextutils-libbuilder-perl, libextutils-makemaker-cpanfile-perl, libextutils-typemap-perl, libfile-counterfile-perl, libfile-pushd-perl, libfile-read-perl, libfile-touch-perl, libfile-type-perl, libfinance-bank-ie-permanenttsb-perl, libfont-freetype-perl, libfrontier-rpc-perl, libgd-securityimage-perl, libgeo-coordinates-utm-perl, libgit-pureperl-perl, libgnome2-canvas-perl, libgnome2-wnck-perl, libgraph-readwrite-perl, libgraphics-colornames-www-perl, libgssapi-perl, libgtk2-appindicator-perl, libgtk2-gladexml-simple-perl, libgtk2-notify-perl, libhash-asobject-perl, libhash-moreutils-perl, libhtml-calendarmonthsimple-perl, libhtml-display-perl, libhtml-fillinform-perl, libhtml-form-perl, libhtml-formhandler-model-dbic-perl, libhtml-html5-entities-perl, libhtml-linkextractor-perl, libhtml-tableextract-perl, libhtml-widget-perl, libhtml-widgets-selectlayers-perl, libhtml-wikiconverter-mediawiki-perl, libhttp-async-perl, libhttp-body-perl, libhttp-date-perl, libimage-imlib2-perl, libimdb-film-perl, libimport-into-perl, libindirect-perl, libio-bufferedselect-perl, libio-compress-lzma-perl, libio-compress-perl, libio-handle-util-perl, libio-interface-perl, libio-multiplex-perl, libio-socket-inet6-perl, libipc-system-simple-perl, libiptables-chainmgr-perl, libjoda-time-java, libjsr305-java, libkiokudb-perl, liblemonldap-ng-cli-perl, liblexical-var-perl, liblingua-en-fathom-perl, liblinux-dvb-perl, liblocales-perl, liblog-dispatch-configurator-any-perl, liblog-log4perl-perl, liblog-report-lexicon-perl, liblwp-mediatypes-perl, liblwp-protocol-https-perl, liblwpx-paranoidagent-perl, libmail-sendeasy-perl, libmarc-xml-perl, libmason-plugin-routersimple-perl, libmasonx-processdir-perl, libmath-base85-perl, libmath-basecalc-perl, libmath-basecnv-perl, libmath-bigint-perl, libmath-convexhull-perl, libmath-gmp-perl, libmath-gradient-perl, libmath-random-isaac-perl, libmath-random-oo-perl, libmath-random-tt800-perl, libmath-tamuanova-perl, libmemoize-expirelru-perl, libmemoize-memcached-perl, libmime-base32-perl, libmime-lite-tt-perl, libmixin-extrafields-param-perl, libmock-quick-perl, libmodule-cpanfile-perl, libmodule-load-conditional-perl, libmodule-starter-pbp-perl, libmodule-util-perl, libmodule-versions-report-perl, libmongodbx-class-perl, libmoo-perl, libmoosex-app-cmd-perl, libmoosex-attributehelpers-perl, libmoosex-blessed-reconstruct-perl, libmoosex-insideout-perl, libmoosex-relatedclassroles-perl, libmoosex-role-timer-perl, libmoosex-role-withoverloading-perl, libmoosex-storage-perl, libmoosex-types-common-perl, libmoosex-types-uri-perl, libmoox-singleton-perl, libmoox-types-mooselike-numeric-perl, libmousex-foreign-perl, libmp3-tag-perl, libmysql-diff-perl, libnamespace-clean-perl, libnet-bonjour-perl, libnet-cli-interact-perl, libnet-daap-dmap-perl, libnet-dbus-glib-perl, libnet-dns-perl, libnet-frame-perl, libnet-google-authsub-perl, libnet-https-any-perl, libnet-https-nb-perl, libnet-idn-encode-perl, libnet-idn-nameprep-perl, libnet-imap-client-perl, libnet-irc-perl, libnet-mac-vendor-perl, libnet-openid-server-perl, libnet-smtp-ssl-perl, libnet-smtp-tls-perl, libnet-smtpauth-perl, libnet-snpp-perl, libnet-sslglue-perl, libnet-telnet-perl, libnhgri-blastall-perl, libnumber-range-perl, libobject-signature-perl, libogg-vorbis-header-pureperl-perl, libopenoffice-oodoc-perl, libparse-cpan-packages-perl, libparse-debian-packages-perl, libparse-fixedlength-perl, libparse-syslog-perl, libparse-win32registry-perl, libpdf-create-perl, libpdf-report-perl, libperl-destruct-level-perl, libperl-metrics-simple-perl, libperl-minimumversion-perl, libperl6-slurp-perl, libpgobject-simple-perl, libplack-middleware-fixmissingbodyinredirect-perl, libplack-test-externalserver-perl, libplucene-perl, libpod-tests-perl, libpoe-component-client-ping-perl, libpoe-component-jabber-perl, libpoe-component-resolver-perl, libpoe-component-server-soap-perl, libpoe-component-syndicator-perl, libposix-strftime-compiler-perl, libposix-strptime-perl, libpostscript-simple-perl, libproc-processtable-perl, libprotocol-osc-perl, librcs-perl, libreadonly-xs-perl, libreturn-multilevel-perl, librivescript-perl, librouter-simple-perl, librrd-simple-perl, libsafe-isa-perl, libscope-guard-perl, libsemver-perl, libset-tiny-perl, libsharyanto-file-util-perl, libshell-command-perl, libsnmp-info-perl, libsoap-lite-perl, libstat-lsmode-perl, libstatistics-online-perl, libstring-compare-constanttime-perl, libstring-format-perl, libstring-toidentifier-en-perl, libstring-tt-perl, libsub-recursive-perl, libsvg-tt-graph-perl, libsvn-notify-perl, libswish-api-common-perl, libtap-formatter-junit-perl, libtap-harness-archive-perl, libtemplate-plugin-number-format-perl, libtemplate-plugin-yaml-perl, libtemplate-tiny-perl, libtenjin-perl, libterm-visual-perl, libtest-block-perl, libtest-carp-perl, libtest-classapi-perl, libtest-cmd-perl, libtest-consistentversion-perl, libtest-data-perl, libtest-databaserow-perl, libtest-differences-perl, libtest-file-sharedir-perl, libtest-hasversion-perl, libtest-kwalitee-perl, libtest-lectrotest-perl, libtest-module-used-perl, libtest-object-perl, libtest-perl-critic-perl, libtest-pod-coverage-perl, libtest-script-perl, libtest-script-run-perl, libtest-spelling-perl, libtest-strict-perl, libtest-synopsis-perl, libtest-trap-perl, libtest-unit-perl, libtest-utf8-perl, libtest-without-module-perl, libtest-www-selenium-perl, libtest-xml-simple-perl, libtest-yaml-perl, libtex-encode-perl, libtext-bibtex-perl, libtext-csv-encoded-perl, libtext-csv-perl, libtext-dhcpleases-perl, libtext-diff-perl, libtext-quoted-perl, libtext-trac-perl, libtext-vfile-asdata-perl, libthai, libthread-conveyor-perl, libthread-sigmask-perl, libtie-cphash-perl, libtie-ical-perl, libtime-stopwatch-perl, libtk-dirselect-perl, libtk-pod-perl, libtorrent, libturpial, libunicode-japanese-perl, libunicode-maputf8-perl, libunicode-stringprep-perl, libuniversal-isa-perl, libuniversal-moniker-perl, liburi-encode-perl, libvi-quickfix-perl, libvideo-capture-v4l-perl, libvideo-fourcc-info-perl, libwiki-toolkit-plugin-rss-reader-perl, libwww-mechanize-formfiller-perl, libwww-mechanize-gzip-perl, libwww-mechanize-perl, libwww-opensearch-perl, libx11-freedesktop-desktopentry-perl, libxc, libxml-dtdparser-perl, libxml-easy-perl, libxml-handler-trees-perl, libxml-libxml-iterator-perl, libxml-libxslt-perl, libxml-rss-perl, libxml-validator-schema-perl, libxml-xpathengine-perl, libxml-xql-perl, llvm-py, madbomber, makefs, mdpress, media-player-info, meta-kde-telepathy, metamonger, mmm-mode, mupen64plus-audio-sdl, mupen64plus-rsp-hle, mupen64plus-ui-console, mupen64plus-video-z64, mussort, newpid, node-formidable, node-github-url-from-git, node-transformers, nsnake, odin, otcl, parsley, pax, pcsc-perl, pd-purepd, pen, prank, proj, proot, puppet-module-puppetlabs-postgresql, python-async, python-pysnmp4, qrencode, r-bioc-graph, r-bioc-hypergraph, r-bioc-iranges, r-bioc-xvector, r-cran-pscl, rbenv, rlinetd, rs, ruby-ascii85, ruby-cutest, ruby-ejs, ruby-factory-girl, ruby-hdfeos5, ruby-kpeg, ruby-libxml, ruby-password, ruby-zip-zip, sdl-sound1.2, stterm, systemd, taktuk, tcc, tryton-modules-account-invoice, ttf-summersby, tupi, tuxpuck, unknown-horizons, unsafe-mock, vcheck, versiontools, vim-addon-manager, vlfeat, vsearch, xacobeo, xen-tools, yubikey-personalization-gui, yubikey-personalization. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: reproducible.debian.net Alioth now hosts a script that can be used to redo builds and test for a package. This was preliminary done manually through requests over the IRC channel. This should reduce the number of interruptions for jenkins' maintainers The graph of the oldest build per day has been fixed. Maintainance scripts will not error out when they are no files to remove. Holger Levsen started work on being able to test variations of CPU features and build date (as in build in another month of 1984) by using virtual machines. debbindiff development Version 18 has been released. It will uses proper comparators for pk3 and info files. Tar member names are now assumed to be UTF-8 encoded. The limit for the maximum number of different lines has been removed. Let's see on reproducible.debian.net how it goes for pathological cases. It's now possible to specify both --html and --text output. When neither of them is specified, the default will be to print a text report on the standard output (thanks to Paul Wise for the suggestion). Documentation update Nicolas Boulenguez investigated Ada libraries. Package reviews 451 obsolete reviews have been removed and 156 added this week. New identified issues: running kernel version getting captured, random filenames in GHC debug symbols, and timestamps in headers generated by qdbusxml2cpp. Misc. Holger Levsen went to re:publica and talked about reproducible builds to developers and users there. Holger also had a chance to meet FreeBSD developers and discuss the status of FreeBSD. Investigations have started on how it could be made part of our current test system. Laurent Guerby gave Lunar access to systems in the GCC Compile Farm. Hopefully access to these powerful machines will help to fix packages for GCC, Iceweasel, and similar packages requiring long build times.

15 May 2015

Sune Vuorela: Getting a Q_INVOKABLE C++ function reevaluated by QML engine

Unfortunately, with all the normal magic of QML property bindings, getting a property updated in a setup that involves return values from functions isn t really doable, like this:

Text
text: qtobject.calculatedValue()


I m told there is a low priority feature request for a way of signalling that a function now returns a different value and all properties using it should be reevaluated. I have so far discovered two different workarounds for that that I will be presenting here. Using an extra property Appending an extra property to trigger the reevaluation of the function is one way of doing it.

Text
text: qtobject.calculatedValue() + qtobject.emptyNotifierThing


with the following on the C++ side:

Q_PROPERTY(QString emptyNotifierThing READ emptyString NOTIFY valueChanged)
QString emptyString() const
return QString();

This is a bit more code to write and to remember to use, but it does get the job done. Intermediate layer
Another way is to inject an intermediate layer, an extra object, that has the function. It can even be simplified by having a pointer to itself.

Text
text: qtobject.dataAccess.calculatedValue()


with the following on the C++ side:

Q_PROPERTY(QObject* dataAccess READ dataAccess NOTIFY valueChanged)
QObject* dataAccess()
return this;

It seems a bit simpler for the reader on the QML side, but also gets the job done. I am not sure which way is the best one, but the intermediate layer has a nicer feeling to it when more complicated types are involved.

Next.

Previous.