Search Results: "James Hunt"

4 October 2015

Lunar: Reproducible builds: week 23 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Andreas Metzler uploaded autogen/1:5.18.6-1 in experimental with several patches for reproducibility issues written by Valentin Lorentz. Groovy upstream has merged a change proposed by Emmanuel Bourg to remove timestamps generated by groovydoc. Ben Hutchings submitted a patch to add support for SOURCE_DATE_EPOCH in linux-kbuild as an alternate way to specify the build timestamp. Reiner Herrman has sent a patch adding support for SOURCE_DATE_EPOCH in docbook-utils. Packages fixed The following packages became reproducible due to changes in their build dependencies: commons-csv. fest-reflect, sunxi-tools, xfce4-terminal, The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: Tomasz Rybak uploaded pycuda/2015.1.3-1 which should fix reproducibility issues. The package has not been tested as it is in contrib. akira found an embedded code copy of texi2html in fftw. reproducible.debian.net Email notifications are now only sent once a day per package, instead of on each status change. (h01ger) disorderfs has been temporarily disabled to see if it had any impact on the disk space issues. (h01ger) When running out of disk space, build nodes will now automatically detect the problem. This means test results will not be recorded as FTBFS and the problem will be reported to Jenkins maintainers. (h01ger) The navigation menu of package pages has been improved. (h01ger) The two amd64 builders now use two different kernel versions: 3.16 from stable and 4.1 from backports on the other. (h01ger) We now graph the number of packages which needs to be fixed. (h01ger) Munin now creates graphs on how many builds were performed by build nodes (example). (h01ger) A migration plan has been agreed with DSA on how to turn Jenkins into an official Debian service. A backport of jenkins-job-builder for Jessie is currently missing. (h01ger) Package reviews 119 reviews have been removed, 103 added and 45 updated this week. 16 fail to build from source issues were reported by Chris Lamb and Mattia Rizzolo. New issue this week: timestamps_in_manpages_generated_by_docbook_utils. Misc. Allan McRae has submitted a patch to make ArchLinux pacman record a .BUILDINFO file.

18 September 2015

Dimitri John Ledkov: Clear Containers for Docker* Engine

Today at work, I announced something James Hunt, Ikey Doherty and myself have been working on. We integrated Clear Containers technology with Docker* Engine to create Clear Containers for Docker* Engine.

After following installation instructions, one can pull and run existing Docker* containers in the secure Clear Containers environment. This means that instead of namespaces, a fast virtual machine is started using the kvmtool hypervisor. This VM is running an optimised minimal Linux* kernel and the optimised Clear Linux* for Intel Architecture Project user-space, with the only goal to execute the Docker* workload and then shut down.

The net effect is almost indistinguishable from typical Docker* container usage:
$ docker run -ti ubuntu:vivid
root@d88a60502ed7:/# systemd-detect-virt
kvm
Apart from, as you see, it's running inside a kvm VM, and thus protected by Intel Virtualization Technology.

This is available on Clear Linux* as well as multiple other operating systems.

I hope this is exciting enough for people to try out, and if you have any feedback, feel free to leave comments or join our mailing list.

*Other names and brands may be claimed as the property of others

The postings on this site are my own and don't necessarily represent Intel s positions, strategies, or opinions.

28 August 2015

Dimitri John Ledkov: Go enjoy Python3

Given a string, get a truncated string of length up to 12.

The task is ambiguous, as it doesn't say anything about whether or not 12 should include terminating null character or not. None the less, let's see how one would achieve this in various languages.
Let's start with python3

import sys
print(sys.argv[1][:12])

Simple enough, in essence given first argument, print it up to length 12. As an added this also deals with unicode correctly that is if passed arg is , it will correctly print . (note these are just random Unicode strings to me, no idea what they stand for).

In C things are slightly more verbose, but in essence, I am going to use strncpy function:

#include <stdio.h>
#include <string.h>
void main(int argc, char *argv[])
char res[12];
strncpy(res,argv[1],12);
printf("%s\n",res);

This treats things as byte-array instead of unicode, thus for unicode test it will end up printing just . But it is still simple enough.
Finally we have Go
package main

import "os"
import "fmt"
import "math"

func main()
fmt.Printf("%s\n", os.Args[1][:int(math.Min(12, float64(len(os.Args[1]))))])

This similarly treats argument as a byte array, and one needs to cast the argument to a rune to get unicode string handling. But there are quite a few caveats. One cannot take out of bounds slices. Thus a na ve os.Args[1][:12] can result in a runtime panic that slice bounds are out of range. Or if a string is known at compile time, a compile time error. Hence one needs to calculate length, and do a min comparison. And there lies the next caveat, math.Min() is only defined for float64 type, and slice indexes can only be integers and thus we end up writing ]))))])...

12 points for python3, 8 points for C, and Go receives nul points Eurovision style.

EDIT: Andreas R ssland and James Hunt are full of win. Both suggesting fmt.Printf("%.12s\n", os.Args[1]) for go. I like that a lot, as it gives simplicity & readability without compromising the default safety against out of bounds access. Hence the scores are now: 14 points for Go, 12 points for python3 and 8 points for C.

EDIT2: I was pointed out much better C implementation by Keith Thompson - http://pastebin.com/5i7rFmMQ in essence it uses strncat() which has much better null termination semantics. And Ben posted a C implementation which handles wide characters http://www.decadent.org.uk/ben/blog/truncating-a-string-in-c.html. I regret to inform you that this blog post got syndicated onto hacker news and has now become the top viewed post on my blog of all time, overnight. In retrospect, I regret awarding points at the end of the blog post, as that's just was merely an expression of opinion and is highly subjective measure. But this problem statement did originate from me reviewing go code that did "if/then/else" comparison and got it wrong to truncate a string and I thought surely one can just do [:12] which has lead me down the rabbit hole of discovering a lot about Go; it's compile and runtime out of bounds access safeguards; lack of universal Min() function; runes vs strings handling and so on. I'm only a beginner go programmer and I am very sorry for wasting everyone's time on this. I guess people didn't have much to do on a Throwback Thursday.

The postings on this site are my own and don't necessarily represent Intel s positions, strategies, or opinions.

17 November 2013

Dmitrijs Ledkovs: libnih (upstart dependency) ported to kFreeBSD/eglibc


Back in September at the Linux Plumbers Conference in New Orleans, James Hunt and I presented Upstart roadmap, which included porting Upstart to kFreeBSD/glibc. The first milestone is now complete.

Libnih is ported to kFreeBSD/eglibc. During the course to get there the following has happened:

  • Later libnih test-suite gets a test case failure, which is then reported to FreeBSD.org. Turns out some syscalls on FreeBSD returned bogus siginfo_t.si_status information in some cases. Luckily there is now a full test-case and kernel patch posted by Jilles Tjoelker. Once that's committed upstream, I'll push for uploads to the affected kernels in Debian.


  • A few small patches applied to libnih, mostly adding POSIX compliant header includes and the like.
So with a patched libc and patched kernel one can compile libnih & run its test-suite. There are however some caveats feature miss-partiy:

  • nih_watch / inotify support is disabled. This is TODO on my list to port nih_watch to use kevent/kqueue EVFILT_VNODE.
  • FreeBSD kernel doesn't have abstract namespace domain sockets, so at the moment I have disabled some abstract namespace test-cases & changed dbus-connection tests to use pathname sockets instead. This is still a TODO because unlink before bind is not done, and the socket is not cleaned up after servers close it at the moment.




With all caveats above, here are the test-suite results:

================================================
libnih 1.0.4: nih-dbus-tool/test-suite.log
================================================

# TOTAL: 2492
# PASS: 2492
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

===========================================
libnih 1.0.4: nih-dbus/test-suite.log
===========================================

# TOTAL: 110
# PASS: 110
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

======================================
libnih 1.0.4: nih/test-suite.log
======================================

# TOTAL: 788
# PASS: 787
# SKIP: 0
# XFAIL: 1
# FAIL: 0
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

PASS: test_watch
================

not ok 1 - nih_watch not supprted yet # TODO
XFAIL: test_watch 1 - nih_watch not supprted yet # TODO

at tests/test_watch.c:1594 (main).
1..1