• [1] http://etbe.coker.com.au/2012/08/08/hard-drives-backup/
• [2] http://etbe.coker.com.au/2012/08/01/cheap-sata-poweredge-t410/
• [3] http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/
• [4] http://protocase.com/products/index.php?e=Backblaze
• [5] http://www.norcotek.com/item_detail.php?categoryid=1&modelno=rpc-4224
• [6] http://www.supermicro.com/products/nfo/storage.cfm

1. Cheap SATA Disks in a Dell PowerEdge T410 A non-profit organisation I support has just bought a Dell...
2. ZFS vs BTRFS on Cheap Dell Servers I previously wrote about my first experiences with BTRFS [1]....
3. Flash Storage and Servers In the comments on my post about the Dell PowerEdge...

• as you might have read in the ftp-master sprint report, we have a new ftp-master: congrats Ansgar! (delegation update pending, but no suspense here)
• we have a new front-desk member (delegation pending + suspense)
• we have a new debian policy team (delegation pending + suspense)
• the tech-ctte is in the process of selecting a new candidate to fill the vacant seat. I've nominated a couple of candidates myself, and I'm now waiting for tech-ctte recommendations as per Constitution 6.2. On related news, the tech-ctte has now dealt with the conflict on Python interpreter maintenership, which was also the longest standing issue brought before them. Having invested quite some energy in that issue myself, I'm delighted to see it finally addressed.

self.assoc_data.update(dict([tuple(ad.strip().split("=")) for ad in data[header_len:].split(",") if ad.find("=") > -1]))

Related articles

• Verifying NTP Reserved Mode Denial of Service Vulnerability (gursevkalra.blogspot.com)
• Time precision between NTP and GPS source (stackoverflow.com)

• Added a note to --node-size that it only affects new B-trees. Thanks, Michael Brown.
• New obnam diff subcommand to show differences (added/removed/modified files) between two generations, by Peter Valdemar M rch.
• obnam backup now logs the names of files that are getting backed up at the INFO level rather than DEBUG.
• The command synopsises for backup, restore, and verify commands now make it clearer that Obnam only accepts directories, not individual files, as arguments. (For now.)
• The output from the show plugin can now be redirected with the --output=FILE option. Affected subcommands: clients, generations, genids, ls, diff, nagios-last-backup-age.

• Notify user of errors during backups.
• The SFTP plugin now manages to deal with repository paths starting with /~/ which already exist without crashing.
• Character and block device nodes are now restored correctly. Thanks to Martin Dummer for the bug report.
• The symmteric key for a toplevel repository directory is re-encrypted when a public key is added or removed to the toplevel using the add-key or remove-key subcommands.
• Manual page typo fix. Thanks, Steve Kemp.

ssize_t
sock_fd_write(int sock, void *buf, ssize_t buflen, int fd)
 
    ssize_t     size;
    struct msghdr   msg;
    struct iovec    iov;
    union  
        struct cmsghdr  cmsghdr;
        char        control[CMSG_SPACE(sizeof (int))];
      cmsgu;
    struct cmsghdr  *cmsg;
    iov.iov_base = buf;
    iov.iov_len = buflen;
    msg.msg_name = NULL;
    msg.msg_namelen = 0;
    msg.msg_iov = &iov;
    msg.msg_iovlen = 1;
    if (fd != -1)  
        msg.msg_control = cmsgu.control;
        msg.msg_controllen = sizeof(cmsgu.control);
        cmsg = CMSG_FIRSTHDR(&msg);
        cmsg->cmsg_len = CMSG_LEN(sizeof (int));
        cmsg->cmsg_level = SOL_SOCKET;
        cmsg->cmsg_type = SCM_RIGHTS;
        printf ("passing fd %d\n", fd);
        *((int *) CMSG_DATA(cmsg)) = fd;
      else  
        msg.msg_control = NULL;
        msg.msg_controllen = 0;
        printf ("not passing fd\n");
     
    size = sendmsg(sock, &msg, 0);
    if (size < 0)
        perror ("sendmsg");
    return size;
 

ssize_t
sock_fd_read(int sock, void *buf, ssize_t bufsize, int *fd)
 
    ssize_t     size;
    if (fd)  
        struct msghdr   msg;
        struct iovec    iov;
        union  
            struct cmsghdr  cmsghdr;
            char        control[CMSG_SPACE(sizeof (int))];
          cmsgu;
        struct cmsghdr  *cmsg;
        iov.iov_base = buf;
        iov.iov_len = bufsize;
        msg.msg_name = NULL;
        msg.msg_namelen = 0;
        msg.msg_iov = &iov;
        msg.msg_iovlen = 1;
        msg.msg_control = cmsgu.control;
        msg.msg_controllen = sizeof(cmsgu.control);
        size = recvmsg (sock, &msg, 0);
        if (size < 0)  
            perror ("recvmsg");
            exit(1);
         
        cmsg = CMSG_FIRSTHDR(&msg);
        if (cmsg && cmsg->cmsg_len == CMSG_LEN(sizeof(int)))  
            if (cmsg->cmsg_level != SOL_SOCKET)  
                fprintf (stderr, "invalid cmsg_level %d\n",
                     cmsg->cmsg_level);
                exit(1);
             
            if (cmsg->cmsg_type != SCM_RIGHTS)  
                fprintf (stderr, "invalid cmsg_type %d\n",
                     cmsg->cmsg_type);
                exit(1);
             
            *fd = *((int *) CMSG_DATA(cmsg));
            printf ("received fd %d\n", *fd);
          else
            *fd = -1;
      else  
        size = read (sock, buf, bufsize);
        if (size < 0)  
            perror("read");
            exit(1);
         
     
    return size;
 

void
child(int sock)
 
    int fd;
    char    buf[16];
    ssize_t size;
    sleep(1);
    for (;;)  
        size = sock_fd_read(sock, buf, sizeof(buf), &fd);
        if (size <= 0)
            break;
        printf ("read %d\n", size);
        if (fd != -1)  
            write(fd, "hello, world\n", 13);
            close(fd);
         
     
 
void
parent(int sock)
 
    ssize_t size;
    int i;
    int fd;
    fd = 1;
    size = sock_fd_write(sock, "1", 1, 1);
    printf ("wrote %d\n", size);
 
int
main(int argc, char **argv)
 
    int sv[2];
    int pid;
    if (socketpair(AF_LOCAL, SOCK_STREAM, 0, sv) < 0)  
        perror("socketpair");
        exit(1);
     
    switch ((pid = fork()))  
    case 0:
        close(sv[0]);
        child(sv[1]);
        break;
    case -1:
        perror("fork");
        exit(1);
    default:
        close(sv[1]);
        parent(sv[0]);
        break;
     
    return 0;
 

void
parent(int sock)
 
    ssize_t size;
    int i;
    int fd;
    fd = 1;
    size = sock_fd_write(sock, "1", 1, -1);
    printf ("wrote %d without fd\n", size);
    size = sock_fd_write(sock, "1", 1, 1);
    printf ("wrote %d with fd\n", size);
    size = sock_fd_write(sock, "1", 1, -1);
    printf ("wrote %d without fd\n", size);
 

void
child(int sock)
 
    int fd;
    char    buf[16];
    ssize_t size;
    sleep(1);
    size = sock_fd_read(sock, buf, sizeof(buf), NULL);
    if (size <= 0)
        return;
    printf ("read %d\n", size);
    size = sock_fd_read(sock, buf, sizeof(buf), &fd);
    if (size <= 0)
        return;
    printf ("read %d\n", size);
    if (fd != -1)  
        write(fd, "hello, world\n", 13);
        close(fd);
     
 
void
parent(int sock)
 
    ssize_t size;
    int i;
    int fd;
    fd = 1;
    size = sock_fd_write(sock, "1", 1, 1);
    printf ("wrote %d without fd\n", size);
    size = sock_fd_write(sock, "1", 1, 2);
    printf ("wrote %d with fd\n", size);
 

void
parent(int sock)
 
    ssize_t size;
    int i;
    int fd;
    fd = 1;
    size = sock_fd_write(sock, "1", 1, -1);
    printf ("wrote %d without fd\n", size);
    size = sock_fd_write(sock, NULL, 0, 1);
    printf ("wrote %d with fd\n", size);
    size = sock_fd_write(sock, "1", 1, -1);
    printf ("wrote %d without fd\n", size);
 

1. read and recvmsg don t merge data across a file descriptor message boundary.
2. failing to accept an fd in the receiver results in the fd being closed by the kernel.
3. a file descriptor must be accompanied by some data.

static void
child(int sock)
 
    uint8_t xreq[1024];
    uint8_t xnop[4];
    uint8_t req;
    int i, reqlen;
    ssize_t size, fdsize;
    int fd = -1, *fdp;
    int j;
    sleep (1);
    for (j = 0;; j++)  
        size = sock_fd_read(sock, xreq, sizeof (xreq), NULL);
        printf ("got %d\n", size);
        if (size == 0)
            break;
        i = 0;
        while (i < size)  
            req = xreq[i];
            reqlen = xreq[i+1];
            i += reqlen;
            switch (req)  
            case 0:
                break;
            case 1:
                if (i != size)  
                    fprintf (stderr, "Got fd req, but not at end of input %d < %d\n",
                         i, size);
                 
                fdsize = sock_fd_read(sock, xnop, sizeof (xnop), &fd);
                if (fd == -1)  
                    fprintf (stderr, "no fd received\n");
                  else  
                    FILE    *f = fdopen (fd, "w");
                    fprintf(f, "hello %d\n", j);
                    fflush(f);
                    fclose(f);
                    close(fd);
                    fd = -1;
                 
                break;
            case 2:
                fprintf (stderr, "Unexpected FD passing req\n");
                break;
             
         
     
 
int
tmp_file(int j)  
    char    name[64];
    sprintf (name, "tmp-file-%d", j);
    return creat(name, 0666);
 
static void
parent(int sock)
 
    uint8_t xreq[32];
    uint8_t xnop[4];
    int i, j;
    int fd;
    for (j = 0; j < 4; j++)  
        /* Write a bunch of regular requests */
        for (i = 0; i < 8; i++)  
            xreq[0] = 0;
            xreq[1] = sizeof (xreq);
            sock_fd_write(sock, xreq, sizeof (xreq), -1);
         
        /* Write our 'pass an fd' request with a 'useless' FD to block the receiver */
        xreq[0] = 1;
        xreq[1] = sizeof(xreq);
        sock_fd_write(sock, xreq, sizeof (xreq), 1);
        /* Pass an fd */
        xnop[0] = 2;
        xnop[1] = sizeof (xnop);
        fd = tmp_file(j);
        sock_fd_write(sock, xnop, sizeof (xnop), fd);
        close(fd);
     
 

The reason for the problem: 4.3.2 Not accepting messages at this time 554- 5.7.1 ERROR: Mail refused <150.101.137.131> See http://sendersupport.senderscore.net

1. A redirection to https://sendersupport.senderscore.net/
2. A badly configured webserver that uses the above URL with a certificate for www.senderscore.net
3. A page that says It works

The reason for the problem: 4.3.2 Not accepting messages at this time 554- 5.7.1 ERROR: Mail refused <150.101.137.131> See http://sendersupport.senderscore.net

1. A redirection to https://sendersupport.senderscore.net/
2. A badly configured webserver that uses the above URL with a certificate for www.senderscore.net
3. A page that says It works

• it's a day care centre "chain". They have kids ranging from infant to pre-K (if I'm getting my grade terminology correct). It's not what I'd call a "school", hence they call themselves a "preschool"
• having actually been to one of the facilities, I can understand the request to not mingle in the lobby. It's tiny, and it blocks the view of the doors from the front office, where an employee is usually buzzing people in from
• I can totally understand the desire by the staff to not hold the door open for other people, even if we "know" them, because of possible changes in family circumstances. I would say that there's a difference between knowing someone by sight, because they're always dropping off or picking up their kid at the same time as you are, and knowing their name, where they live, etc. Having done pickups and drop offs, there are parents in both categories. I'll still hold the door open for a parent that we socialise with outside of "school", but I'll think twice now before doing it for someone who only looks familiar but I don't really know. I don't want to be the one who facilitates a future incident
• I think a lot of the conservative position the day care management has stems from liability. I have no problem with that. If the shoe was on the other foot, and an estranged parent had walked in and taken their child, and the centre had a "locked door" policy, there'd be some uncomfortable questions to answer. I'm paying them a boatload of money, they say they operate under a "locked door" policy, I expect them to keep my kid safe

• Integrated the KGB bot for Debian-IN team repository. Thanks to gregoa Tincho and dam for extending helping hand in getting it configured
• Updated the setup-repository to install a proper post-receive hook when creating new repositories.
• Copied the set-git-hook from pkg-ruby-extras team and modified it slightly to install post-receive hook for KGB integration for already existing repository. With this script I fixed few of the packages to use KGB which I own. Waiting for Kartik to fix remaining repositories as he is the DD for our team :-)
• Configured my personal instance of KGB which now stands in Debian-IN's channel (#debian-in). But bot is named RAW (Research and Analysis Wing) equivalaent of KGB and CIA in India :-P

• Attended the PyconIndia-2012 on Saturday, Got a NFC card :-) which I can't use as my phone is not NFC capable. Only handful of talks were nice rest were all newbie like talk which even lacked proper presentation skills hence didn't attend on second day. Looks like quality of conference in India is decreasing.
• Finally completed answering all questions on Philosophy and Procedures - 1 part of NM process :-). Phew it took me more time than I expected. Waiting for my AM's comment. Hopefully I will get next bunch of questions soon :-)

Good old email If you cannot be bothered with change, then simply email me! My email address is dr@jones.dk like always

Simple webchat If you are in a hurry and do not use Jabber yet, then try use my Simple Groupchat. Use guest login if you are not a user at jones.dk.

Proper Jabber Jabber is similar to email: You need 1) an account somewhere, 2) some client application setup to use that account, and 3) the address of those you want to connect.

1. Register an account at e.g. Indonesian http://jabin.org/ or some other server.
2. Install a Jabber client program :
   • on Windows try install Gajim from http://gajim.org/.
   • on smartphone try app from https://imo.im/.
   • on Ubuntu try use the Me menu in top right corner.
   • on Debian try install Pidgin: aptitude install pidgin.
3. Invite me as your friend/buddy: my Jabber ID is jonas@jones.dk
4. Tell your friends to do the same!

Geeky IRC Many computer geeks hang out on IRC. I find it ugly and clumsy to use, but most of my techie friends are there, so I recommend going there to discuss technical matters. On OFTC my nickname is jonas, and I often hang out at the chatroom #debian-devel (plus a few others).

Why not Facebook? Facebook is a closed system: all activities happen at one central place. Email, Jabber and IRC are open protocols, exchanging between many Free systems (and some non-free ones as well).

Imagine having an overprotecting mother, who allow you only to meet your friends at one spot in town - where she can look after you. You maybe have no secrets from your mom, but probably still appreciate the option of being alone - to have some privacy.

Facebook is that one spot in town. Jabber is "let's go meet somewhere". Regards, Jonas

SMS is used to lock car doors Copenhagen Carpool soon receive a new batch of electric cars. The new cars will contain a cellphone tied to the door locks, to only let into the car the member who booked its current timeslot. I am setting up the system to connect the cars with their booking calendar. The general idea is to

1. Connect a standard cellphone to a computer.
2. Forward sms messages received on the phone to the calendar.
3. Send messages from the calendar through the phone.
4. Leave the system in a corner, turned on 24/7.

Most of my work is in trying to avoid ways things might go wrong which so often happens when computers are involved. What if

• messages are delayed or lost?
• a chain of messages arrive out of order?
• messages are sent faster than they can be delivered?

Reliable conversation Human conversation is most reliable when using plain speak. Slang, sarcasm or jokes raise the risk of misunderstandings. This is quite similar in computer conversation. When surfing the Web, you normally "GET" a web page or "POST" new data like a search query. The words "GET" and "POST" are "plain speak" of the World Wide Web defined as a principle called "REST" or "RESTful design". I use the Kannel tool to talk directly to the phones. When an sms arrives on the phone, Kannel "GET"s it to the calendar. That's weakly expressed better if it had "POST"ed the message instead. Kannel is a quite reliable tool, but not RESTful in conversations. That requires extra attention at other parts of the setup to avoid breakage in special situations. That worries me.

Each car has a blog So I setup a gateway from Kannel to RESTful conversation. And choose classic blogging as style of conversation:

• The carpool fleet is a blogroll
• each car has a blog
• each sms emitted from a car is a blog entry

Blogging comes in two flavors, RSS and Atom. Both cover distribution of messages. I use Atom because it is newest and coolest, and because it also covers creating and editing meesages with the AtomPub extension. AtomPub allows the calendar to create an empty message which then triggers an sms from the car which "fills it in." AtomPub also let the calendar "PUT" tags onto existing messages, to mark them as processed, read etc., and let an admin or a cleanup script "DELETE" outdated entries or blogs. The words "PUT" and "DELETE" are RESTful conversation as well.

The system is dancing! My sms-to-blog gateway is written in Perl, using the Dancer toolkit. Dancer makes it simple to setup a RESTful web conversation, and when I contacted the authors on IRC, they swiftly added support for the non-RESTful quirk that I needed. My sms-to-blog gateway is not yet ready, but basic routines work. The project is public, and code is Free Software.

• InvalidateBuffers events. When the X window size changes, the buffers created by the X server for rendering must change size to match. The problem is that the client is presumably drawing to the old buffers when the new ones are allocated. Delivering an event to the client is supposed to make it possible for the client to keep up, but the reality is that the event is delivered at some random time to some random thread within the application. This leads to general confusion within the application, and often results in a damaged frame on the screen. Fortunately, applications tend to draw their contents often, so the damaged frame only appears briefly.
• No information about new back buffer contents. When a buffer swap happens and the client learns about the new back buffer, the back buffer contents are always undefined. For most applications, this isn t a big deal as they re going to draw the whole window. However, compositing managers really want to reduce rendering by only repainting small damaged areas of the window. Knowing what previous frame contents are present in the back buffer allows the compositing manager to repaint just the affected area.
• Un-purgable stale buffers. Between the X server finishing with a buffer and the client picking it up for a future frame, we don t need to save the buffer contents and should mark the buffer as purgable. With the current DRI2 protocols, this can t be done, which leaves all of those buffers hanging around in memory.
• Driver-specific buffers. The DRI2 buffer handles are device specific, and so we can t use buffers from other devices on the screen. External video encoders/cameras/encoders can t be used with the DRI2 extension.
• GEM flink has lots of issues. The flink names are global, allowing anyone with access to the device to access the flink data contents. There is also no reference to the underlying object, so the X server and client must carefully hold references to GEM objects during various operations.

• No longer need DRI2 additions to manage new GL buffers. Adding HiZ to the intel driver required new DRI2 code in the X server, even though X wasn t doing anything with those buffers at all.
• Eliminate some X round trips currently required for GL buffer allocation.
• Knowing what s in each buffer. Because the client allocates each buffer, it can track the contents of them.
• Size tracking is trivial. The application sends the GL the of the viewport, and the union of all viewports should be the same as the size of the window (or there will be undefined contents on the screen). The driver can use the viewport information to size the buffers and ensure that every frame on the screen is complete.

foldl' :: (s -> x -> s) -> s -> [x] -> s

data LeftFold x a = forall s. LeftFold  
  start :: s,
  process :: s -> x -> s,
  finish :: s -> a
   

runLeftFold :: LeftFold x a -> [x] -> a
runLeftFold (LeftFold st1 p1 f1) xs = f1 (foldl' p1 st1 xs)

instance Functor (LeftFold x) where
  fmap f (LeftFold st1 p1 f2) = LeftFold st1 p1 (f . f2)
instance Applicative (LeftFold x) where
  pure x = LeftFold () const (const x)
  LeftFold st1 p1 f1 <*> LeftFold st2 p2 f2 = LeftFold  
  start  =  st1 :!: st2,
  process = \(s1 :!: s2) x -> p1 s1 x :!: p2 s2 x,
  finish  = \(s1 :!: s2)  -> f1 s1 (f2 s2)
   

sequenceA :: [LeftFold x a] -> LeftFold x [a]

monoidFold :: Monoid m => LeftFold m m
mapElems :: LeftFold y a -> (x -> y) -> LeftFold x a 
filterWith :: (x -> Bool) -> LeftFold (Bool :!: x) a -> LeftFold x a
onSelected :: LeftFold x a -> LeftFold (Bool :!: x) a
onJusts :: LeftFold x a -> LeftFold (Maybe x) a
onAll :: LeftFold x a -> LeftFold (Bool :!: x) a
runOnGroups :: (x -> x -> Bool) -> LeftFold x y -> LeftFold y z -> LeftFold x z
runOnIntervals :: LeftFold x y -> LeftFold y z -> LeftFold (Bool :!: x) z
lfLength :: LeftFold x Int
lfFirst :: LeftFold x (Maybe x)
lfLast :: LeftFold x (Maybe x)
toList :: LeftFold x [x]
concatFold :: LeftFold [x] [x]