Search Results: "Bastian Blank"

28 December 2020

Bastian Blank: Salsa updated to GitLab 13.7

Yesterday, Debian Salsa was updated to the new GitLab 13.7 upstream release. As always, this new release comes with a bunch of new features. GitLab 13.7 GitLab 13.7 includes a bunch of new features. See the upstream release posting for a full list.

22 October 2020

Bastian Blank: Salsa updated to GitLab 13.5

Today, GitLab released the version 13.5 with several new features. Also Salsa got some changes applied to it. GitLab 13.5 GitLab 13.5 includes several new features. See the upstream release postfix for a full list. Shared runner builds on larger instances It's been way over two years since we started to use Google Compute Engine (GCE) for Salsa. Since then, all the jobs running on the shared runners run within a n1-standard-1 instance, providing a fresh set of one vCPU and 3.75GB of RAM for each and every build. GCE supports several new instance types, featuring better and faster CPUs, including current AMD EPICs. However, as it turns out, GCE does not support any single vCPU instances for any of those types. So jobs in the future will use n2d-standard-2 for the time being, provinding two vCPUs and 8GB of RAM.. Builds run with IPv6 enabled All builds run with IPv6 enabled in the Docker environment. This means the lo network device got the IPv6 loopback address ::1 assigned. So tests that need minimal IPv6 support can succeed. It however does not include any external IPv6 connectivity.

30 September 2020

Bastian Blank: Booting Debian on ThinkPad X13 AMD

Running new hardware is always fun. The problems are endless. The solutions not so much. So I've got a brand new ThinkPad X13 AMD. It features an AMD Ryzen 5 PRO 4650U, 16GB of RAM and a 256GB NVME SSD. The internal type identifier is 20UF. It runs the latest firmware as of today with version 1.09. So far I found two problems with it: Disable Secure Boot The system silently fails to boot a signed shim and grub from an USB thumb drive. I used on of the Debian Cloud images, which should properly work in this setup and do on my other systems. The only fix I found was to disable Secure Boot alltogether. Select Linux in firmware Running a Linux 5.8 with default firmware setting produces ACPI errors on each key press.
ACPI BIOS Error (bug): Could not resolve symbol [\_SB.PCI0.GPP3], AE_NOT_FOUND (20200528/psargs-330)
ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_FOUND) (20200528/psparse-529)
This can be "fixed" by setting a strategic setting inside the firmware:
Config > Power > Sleep State to Linux

16 September 2020

Bastian Blank: Salsa hosted 1e6 CI jobs

Today, Salsa hosted it's 1,000,000th CI job. The price for hitting the target goes to the Cloud team. The job itself was not that interesting, but it was successful.

12 August 2017

Bastian Blank: Network caps in cloud environments

Providing working network is not easy. All the cloud providers seem to know how to do that most of the time. Providing enough troughput is not easy either. Here it get's interresting as the cloud providers tackle that problem with completely different results. There are essentially three large cloud providers. The oldest and mostly known cloud provider is Amazon Web Services (AWS). Behind that follow Microsoft with Azure and the Google Cloud Platform (GCP). Some public instances of OpenStack exist, but they simply don't count anyway. So we remain with three and they tackle this problem with widely different results. Now, what network troughput is necessary for real world systems anyway? An old friend gives the advice: 1Gbps per Core of uncongested troughput within the complete infrastructure is the minimum. A generalization of this rule estimates around 1bps per clock cycle and core, so a 2GHz core would need 2Gbps. Do you even get a high enough network cap at your selected cloud provider to fill any of these estimates? Our first provider, AWS, publishes a nice list of network caps for some of there instance types. The common theme in this list is: for two cores (all the *.large types) you get 500Mbps, for four cores (*.xlarge) you get 750Mbps and for eight cores (*.2xlarge) you get 1000Mbps. This is way below our estimate shown above and does not even raise linear with the number of cores. But all of this does not really matter anyway, as the performance of AWS is the worst of the three providers. Our second provider, Azure, seems to not publish any real information about network caps at all. From my own knowledge it is 50MBps (500Mbps) per core for at least the smaller instances. At least is scales linear with instance size, but is still way below our estimates. Our third provider, GCP, documents a simple rule for network caps: 2Gbps per core. This matches what we estimated. Now the most important question: does this estimate really work and can we actually fill it. The answer is not easy. A slightly synthetic test of a HTTP server with cached static content showed that it can easily reach 7Gbps on a 2GHz Intel Skylake core. So yes, it gives a good estimate on what network troughput is needed for real world applications. However we still could easily file pipe that is larger by a factor of three.

11 June 2017

Bastian Blank: New blog featuring Pelican

For years I used a working blog setup using Zope, Plone and Quills. Quills was neglected for a long time and made me stuck at the aging Plone 4.2. There seems to be some work done in the last year, but I did not really care. Also this whole setup was just a bit too heavy for what I actually use it for. Well, static page generators are the new shit, so there we are. So here is it, some new blog, nice and shiny, which I hope to stop neglecting. The blog is managed in a Git repository. This repository is hosted on a private GitLab instance. It uses Pelican to generate shiny pages and is run via the GitLab CI. The finished pages are served by a, currently not highly available, instance of GitLab Pages (yes, this is one of the parts they copied from Github first). So let's see if this setup makes me more comfortable.

9 December 2016

Guido G nther: Debian Fun in November 2016

Debian LTS November marked the nineteenth month I contributed to Debian LTS under the Freexian umbrella. I had 7 hours allocated which I used completely by: Other Debian stuff Some other Free Software activites

3 August 2016

Guido G nther: Debian Fun in July 2016

Debian LTS July marked the fifteenth month I contributed to Debian LTS under the Freexian umbrella. As usual I spent the 8 hours working on these LTS things: Other Debian stuff

14 March 2016

Lunar: Reproducible builds: week 46 in Stretch cycle

What happened in the reproducible builds effort between March 6th and March 12th:

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dfc, gap-openmath, gnubik, gplanarity, iirish, iitalian, monajat, openimageio, plexus-digest, ruby-fssm, vdr-plugin-dvd, vdr-plugin-spider. The following packages became reproducible after getting fixed:
  • adduser/3.114 by Niels Thykier.
  • bsdmainutils/9.0.7 by Michael Meskes.
  • criu/2.0-1 by Salvatore Bonaccorso.
  • genometools/1.5.8+ds-2 by Sascha Steinbiss.
  • gfs2-utils/3.1.8-1 uploaded by Bastian Blank, fix by Christoph Berg.
  • gmerlin/1.2.0~dfsg+1-5 by IOhannes m zm lnig.
  • heroes/0.21-14 by Stephen Kitt.
  • kmc/2.3+dfsg-3 by Sascha Steinbiss.
  • polyml/5.6-3 by James Clarke.
  • sed/4.2.2-7.1 by Niels Thykier.
  • snpomatic/1.0-3 by Sascha Steinbiss.
  • tantan/13-4 by Sascha Steinbiss.
Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #817979 on modernizr by Sascha Steinbiss: sort list of files included in feature-detects.js.
  • #818027 on snapper by Sascha Steinbiss: always use /bin/sh as shell.

tests.reproducible-builds.org Always use all cores on armhf builders. (h01ger) Improve the look of Debian dashboard. (h01ger)

Package reviews 118 reviews have been removed, 114 added and 15 updated in the previous week. 15 FTBFS have been filled by Chris Lamb. New issues: xmlto_txt_output_locale_specific.

Misc. Lunar seeks new maintainers for diffoscope, several mailing lists, and these very weekly reports.

21 March 2014

Petter Reinholdtsen: Video DVD reader library / python-dvdvideo - nice free software

Keeping your DVD collection safe from scratches and curious children fingers while still having it available when you want to see a movie is not straight forward. My preferred method at the moment is to store a full copy of the ISO on a hard drive, and use VLC, Popcorn Hour or other useful players to view the resulting file. This way the subtitles and bonus material are still available and using the ISO is just like inserting the original DVD record in the DVD player. Earlier I used dd for taking security copies, but it do not handle DVDs giving read errors (which are quite a few of them). I've also tried using dvdbackup and genisoimage, but these days I use the marvellous python library and program python-dvdvideo written by Bastian Blank. It is in Debian already and the binary package name is python3-dvdvideo. Instead of trying to read every block from the DVD, it parses the file structure and figure out which block on the DVD is actually in used, and only read those blocks from the DVD. This work surprisingly well, and I have been able to almost backup my entire DVD collection using this method. So far, python-dvdvideo have failed on between 10 and 20 DVDs, which is a small fraction of my collection. The most common problem is DVDs using UTF-16 instead of UTF-8 characters, which according to Bastian is against the DVD specification (and seem to cause some players to fail too). A rarer problem is what seem to be inconsistent DVD structures, as the python library claim there is a overlap between objects. An equally rare problem claim some value is out of range. No idea what is going on there. I wish I knew enough about the DVD format to fix these, to ensure my movie collection will stay with me in the future. So, if you need to keep your DVDs safe, back them up using python-dvdvideo. :)

9 September 2013

Bastian Blank: Setting up Ceph the hard way

Components Ceph consists of two main daemons. One is the monitoring daemon, which monitors the health of the cluster and provides location information. The second is the storage daemon, which maintains the actual storage. Both are needed in a minimal setup.
Monitor The monitor daemons are the heart of the cluster. They maintain quorum within the cluster to check if everything can be used. They provide referrals to clients, to allow them to find the data they seek. Without a majority of monitors nothing will work within the cluster.
Storage The storage daemons maintain the actual storage. One daemon maintains one backend storage device.
Configuration The default config is understandable, but several things will just not work with it.
Monitor on localhost By default the monitor daemon will not work on localhost. There is an (undocumented) override to force it to work on localhost:
[mon.noname-admin]
 mon addr = [::1]:6789
The monitor will be renamed to mon.admin internaly.
IPv6 Ceph supports IP (IPv6) or legacy-IP (IPv4), but never both. I don't really use legacy-IP any longer, so I have to configure Ceph accordingly:
[global]
  ms bind ipv6 = true
One-OSD clusters For testing purposes I wanted to create a cluster with exactly one OSD. It never got into a clean state. So I asked and found the answer in #ceph:
[global]
 osd crush chooseleaf type = 0
Disable authentication While deprecated, the following seems to work:
[global]
 auth supported = none
Complete configuration
[global]
 auth supported = none
 log file = $name.log
 run dir =  
 osd pool default size = 1
 osd crush chooseleaf type = 0
 ms bind ipv6 = true
[mon]
 mon data =  /$name
[mon.noname-admin]
 mon addr = [::1]:6789
[osd]
 osd data =  /$name
 osd journal =  /$name/journal
 osd journal size = 100
[osd.0]
 host = devel
Installation This is currently based on my updated packages. And they are still pretty unclean from my point of view.
Setup All the documentation tells only about ceph-deploy and ceph-disk. This tools are abstractions that need root to mount stuff and do all the work. Here I show how to do a minimal setup without needing root.
Keyring setup For some reason even with no authentication the monitor setup wants a keyring. So just set one up:
$ ceph-authtool --create-keyring keyring --gen-key -n mon.
$ ceph-authtool keyring --gen-key -n client.admin
Monitor setup Monitor setup by hand is easy:
$ mkdir $mon_data
$ ceph-mon -c ceph.conf --mkfs --fsid $(uuidgen) --keyring keyring
After that just start it:
$ ceph-mon -c ceph.conf
$
OSD setup First properly add the new OSD to the internal state:
$ ceph -c ceph.conf osd create
$ ceph -c ceph.conf osd crush set osd.0 1.0 root=default
Then setup the OSD itself:
$ mkdir $osd_data
$ ceph-osd -c ceph.conf -i 0 --mkfs --mkkey --keyring keyring
And start it:
$ ceph-osd -c ceph.conf -i 0
starting osd.0 at :/0 osd_data $osd_data $osd_data/journal
$
Health check The health check should return ok after some time:
$ ceph -c ceph.conf health
HEALTH_OK
$

4 March 2013

Bastian Blank: Using SECCOMP to filter sync operations

Linux includes a syscall filter since a long time. It was restricted to a pre-defined set of syscalls. Since some versions Linux got a more generic filter. Linux can use a BPF-filter to define actions for syscalls. This allows a fine granular specification on which syscalls to act. Also it supports different outcomes assigned to the filter. This filter can be used to filter out sync operations. Debian already got a tool to do this called eatmydata. It is pretty limited as it uses a shared library to drop the library functions. It needs to be available at all times, or it will not do anything. I wrote a small tool that asks the kernel to filter out sync operations for all children. It sets a filter with all currently supported sync-like operations and makes them return success. However it can't filter the O_SYNC-flag from the open-syscall, so it just makes it return an error. It executes the command given on the command-line after that. This is just a proof of concept, but lets see.
/*
 * Copyright (C) 2013 Bastian Blank <waldi@debian.org>
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice, this
 *    list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 *    this list of conditions and the following disclaimer in the documentation
 *    and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
#define _GNU_SOURCE 1
#include <errno.h>
#include <fcntl.h>
#include <seccomp.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define filter_rule_add(action, syscall, count, ...) \
  if (seccomp_rule_add(filter, action, syscall, count, ##__VA_ARGS__)) abort();
static int filter_init(void)
 
  scmp_filter_ctx filter;
  if (!(filter = seccomp_init(SCMP_ACT_ALLOW))) abort();
  if (seccomp_attr_set(filter, SCMP_FLTATR_CTL_NNP, 1)) abort();
  filter_rule_add(SCMP_ACT_ERRNO(EINVAL), SCMP_SYS(open), 1, SCMP_A1(SCMP_CMP_MASKED_EQ, O_SYNC, O_SYNC));
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(fsync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(fdatasync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(msync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(sync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(syncfs), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(sync_file_range), 0);
  return seccomp_load(filter);
 
int main(__attribute__((unused)) int argc, char *argv[])
 
  if (argc <= 1)
   
    fprintf(stderr, "usage: %s COMMAND [ARG]...\n", argv[0]);
    return 2;
   
  if (filter_init())
   
    fprintf(stderr, "%s: can't initialize seccomp filter\n", argv[0]);
    return 1;
   
  execvp(argv[1], &argv[1]);
  if (errno == ENOENT)
   
    fprintf(stderr, "%s: command not found: %s\n", argv[0], argv[1]);
    return 127;
   
  fprintf(stderr, "%s: failed to execute: %s: %s\n", argv[0], argv[1], strerror(errno));
  return 1;
 

2 March 2013

Bastian Blank: LDAP, Insignificant Space and Postfix

For some, LDAP, just like X.500, is black magic. I won't argue against. Sometimes it really shows surprising behavior. It always makes sense, if you think about what LDAP is built for. One surprising behavior is the handling of the "Insignificant Space"
LDAP supports syntax and comparator methods. The syntax specifies how the entry should look. Usually this is some form of text, but numbers or stuff like telephone numbers are supported as well. The comparators specifies how the values are compared. Most of the text comparators are defined to apply the handling of insignificant spaces. Insignificant space handling normalizes the use of spaces. First all leading and trailing spaces are removed. All the internal spaces are normalized to at most two spaces. At the end all strings starts and ends with one space to allow proper sub-string matches. The resulting strings are used for comparisons. This behavior makes sense most of the time. If the user want to find something in the directory, he usually don't cares about spaces, but about content. But I found one occurrence where this produces some grieve. Postfix supports LDAP since some time. And lets say, it does not care about spaces in its queries. This is no problem as e-mail addresses do not contain spaces. Or do they?
Yes, e-mail addresses can contain spaces. This is not widely known, but still allowed. This addresses are quoted and the command looks like RCPT TO:<" test"@example.com>. The local part is quoted and contains a space at the beginning. And this is where the problem starts. Postfix sanitizes the address. It uses a simplified internal representation of the address for all lookups. So the address gets test@example.com. This form is used in all table lookups. The LDAP table uses the internal form of the address. This address is copied verbatim into the query. This query may look this way (mail= test@example.com). It is sent to the server this way. The LDAP server applies the insignificant space modifications. The query is now interpreted and the comparator specified modifications are applied. The query gets effectively changed to (mail=test@example.com). And this is where the fun starts. Postfix accepts undeliverable mails. Depending on the setup, such LDAP queries may be used to check for valid addresses. Because of the space handling, the sender can add spaces to the beginning of the address and it will still be considered valid. In later steps this addresses are not longer valid. Addresses starting with spaces are considered invalid in some locations of Postfix. What surprised me a bit is that virtual alias handling did not map them. The unmodified addressed showed up on the LMTP backend server. That's why they showed up my radar. I would say Postfix is wrong in this case. The LDAP server applies the spec correctly and defines spaces in e-mail addresses as insignificant. Postfix on the other side considers them significant. The easiest fix would be to not allow any spaces in the LDAP table.

25 December 2012

Bastian Blank: New software: LMTP to UUCP gateway

I use UUCP to get my mails. It works fine but lacks support for modern SMTP features like DSN. While it may be possible to bolt support into the the rmail part, both the sendmail interface used to submit mails and the Postfix pipe daemon used to extract mail are not able to do so. So I started a small project to get around this problem. This software uses LMTP to retrieve and SMTP to send all mails. LMTP (a SMTP derivative with support for all extensions) is used to inject mail via a small daemon. The mails are transported using a format similar to batched SMTP to the remote system. It is then injected via SMTP to the local MTA.
Sender LMTP is used to supply mail. As a SMTP derivative, LMTP inherits support for all the available SMTP extensions. The only difference between LMTP and SMTP is the support for one result per recipient after end-of-data. This allow proper handling and mails with multiple recipients without a queue. Mails are supplied to a special LMTP server. This server may currently run from inetd or in foreground by itself. A real daemon mode is not yet implemented. Each mail is submitted to the UUCP queue in its own format. We need to store a lot of meta-data along with the real mail. This data is stored in a custom format.
Protocol All data is transferred using a custom protocol. It is a SMTP derivative, but it is only used in uni-directional communication, so no responses exists. It uses its own Hello command and supports the SMTP commands MAIL, RCPT and DATA. This format allows exactly one mail in each file. An EOF ends the mail transaction. Also all data must be in dot-escaped form like in SMTP.
Hello (UHLO) A sender must start the transaction with this command. It specifies the name of the sender and all requested SMTP extensions. Syntax:
uhlo = "UHLO" SP ( Domain / address-literal ) *( SP ehlo-keyword ) CRLF
The receiver must check if all requested SMTP extensions are available.
Receiver Each mail is submitted by the UUCP system. It calls the supplied receiver tool called rumtp. This tool reads the protocol stream and submits the mail to a local SMTP server. There is no error handling right now in this tool. All errors will create a mail to the local UUCP admin by the UUCP system itself.
License and distribution This package is licensed GPL 3. It is for new distributed via Alioth.

24 December 2012

Bastian Blank: Relaying e-mail over unreliable connections

Introduction I still prefer to handle all my mails at home. I got backup set-up on my own and have anything in direct access. This network is connected to the internet via an end-user DSL connection without a static IP-address and without any useful SLA. However relaying mails over such an unreliable connection is still an unsolved problem. There exists a lot of solutions for this problem. I'm currently building a new e-mail setup, so I tried to collect all the used and maybe possible solutions for the relay problem. I will show some of them.
Don't do it The easiest solution is to just don't try to collect mails at home. This solution is somewhat against the rules, but I know people who prefers it. Access to the mails is usually done with IMAP and a copy is made if needed. This is not really a solution, but it works for many people.
SMTP SMTP is the mail protocol used in the internet. It is used for all public mail exchanges. By itself it can't be used to submit mails to remote destinations With some preparations it can be used to relay mail over unreliable connections. There are three different turn commands in SMTP that can be used to start the mail flow. Two of them will be
VPN or dynamic DNS with TLS with or without extended turn (ETRN) Using SMTP themself is an easy solution. It can relay mails without much hasle But it needs either a fixed or authenticated connection. SMTP can use a fixed connection. This is usually provided by some sort of VPN. The VPN can be encrypted, but does not need to. This allow the MTA to connect to the downstream server if it is available. The other solution is to authenticate the downstream server. Authentication is available via TLS and X.509 certificates. It still needs some way to find it, but with dynamic DNS this is no real problem. Both variants can be combined with the extended turn command. Extended turn allows to request a queue flush for a domain. It can be used to request mails only to be delivered if the downstream server is available at all. This reduces the load on the MTA.
Authenticated turn (ATRN) On-demand mail relay is a rarely supported ESMTP feature. The Authenticated turn command effectively reverses the SMTP connection and allows, after authentication, the flow of mails from the server to the client via standard SMTP. There exists some stand-alone implementations, but no widely used MTA includes an implementation.
POP3/IMAP and fetchmail/getmail All e-mails can be delivered to mailboxes and retrieved by the end-users system and re-injected into the delivery system. Both fetchmail and getmail are able to retrieve mail from POP3 and IMAP servers. They are either directly delivered via a MDA like procmail or maildrop. Otherwise they are submitted via the sendmail interface or SMTP and delivered by the MTA. Neither POP3 nor IMAP have support for meta data like the real recipient or sender.
Mailbox per address The mails for each address are delivered to its own mailbox. This allows proper identification of the recipient address. It still have no real idea of the sender address. Because it needs to poll one mailbox per address, this highers the resources needed on both sides dramatically.
Multi-drop mailbox The mails for all addresses are delivered into one mailbox. The original recipient must be saved in a custom header to allow this information to be restored. Only one mailbox needs to be polled for mail addresses.
UUCP One of the oldest transports used for mail is UUCP. It effectively copies a file to a different system and pipes it into a program. UUCP can be used to transport mails in various ways.
rmail Each mail is copied verbatim to the client. It saves the sender address in form of a "From" pseudo header in the mail itself and supplies the recipient on the command-line. So it have access to both sender and recipient address.
Batched SMTP Batched SMTP transfers old-style SMTP transactions over UUCP The MTA (Exim supports this) or a special tool (in the bsmtpd package) writes this files and after a given time or size they are dispatched via UUCP to the remote system. The bsmtpd packages was removed from Debian some years ago.
Dovecot synchronization Dovecot support bi-directional synchronization of mailbox contents. It holds all data on both sides. The internal log is used to merge changes done on both sides, so it should not loose any data. This synchronization can be used to work with the data on both sides (via Dovecot of cause) or create backups. It needs shell access to the user owning the data on both sides.
Conclusions There is no one size fits all solution for this problem. If you admin the remote system, you can implement any of this solutions If it is managed by someone else you need good luck. Almost all solutions does not support current SMTP features. The one I'm really missing is DSN, aka configurable delivery notices. POP3 and IMAP handle already delivered mail and have no need for this. All UUCP variants does not handle it, because they are much older anyway. Only SMTP itself supports all of its features. I still use rmail over UUCP for my mails at home and it works flawless. UUCP itself runs over SSH, it can compress data on the fly and authenticate using private keys.

1 July 2012

Niels Thykier: Performance bottlenecks in Lintian

Thanks to a heads up from Bastian Blank, I learned that Lintian 2.5.7 and 2.5.8 were horribly slow on the Linux binaries. Bastian had already identified the issue and 2.5.9 fixed the performance regression. But in light of that, I decided to have a look at a couple of other bottlenecks. First, I added a simple benchmark support to Lintian 2.5.10 (enabled with -dd) that prints the approximate run time of a given collection. As an example, when running lintian -dd on lintian 2.5.10, you can see something like:
N: Collecting info: unpacked for source:lintian/2.5.10 ...
[...]
N: Collection script unpacked for source:lintian/2.5.10 done (0.699s)
When done on linux-image, the slowest 3 things with 2.5.10 are (in order of appearance):
[...]
N: Collection script strings for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (12.333s)
N: Collection script objdump-info for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (15.915s)
[...]
N: Finished check: binaries (5.911s)
[...]
(The mileage (and order) probably will vary a bit.) These 3 things makes up about 22 seconds of a total running time on approximately 28-30s on my machine. Now if you wondering how 12, 16 and 6 becomes 22 the answer is parallelization . strings and objdump-info are run in parallel so only the most
expensive of the two counts in practise (with multiple processing units). The version of linux-image I have been testing (3.2.20-1, amd64) has over 2800 ELF binaries (kernel modules). That makes the runtime of strings and objdump-info much more dominating than in your average package . For the fun of it I have done a small informal benchmark of various Lintian versions on the binary. I have used the command line:
# time is the bash shell built-in and not /usr/bin/time
$ time lintian -EvIL +pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null
# This was used with only versions that did not accept -L +pedantic
$ time lintian -EvI --pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null
With older versions of Lintian (<= 2.5.3) Perl starts to emit warnings; these have been manually filtered out. I used lintian from the git repository (i.e. I didn t install the packages, but checked out the relevant git tags). I had libperlio-gzip-perl installed (affects the 2.5.10 run). Most results are only from a single run, though I ran it twice on the first version (hoping my kernel would cache the deb for the next run). The results are:
2.5.10
real  0m28.836s
user  0m36.982s
sys  0m3.280s
2.5.9
real  1m9.378s
user  0m33.702s
sys  0m11.177s
2.5.8
real  4m54.492s
user  4m0.631s
sys  0m30.466s
2.5.7 (not tested, but probably about same as 2.5.8)
2.5. 0..6 
real  1m20s  - 1m22s
user  0m19.0s - 0m20.7s
sys  0m5.1s  - 0m5.6s
I think Bastian s complaint was warranted for 2.5. 7,8 . :) While it would have been easy to attribute the performance gain in 2.5.10 on the new parallelization improvements, it is simply not the case. These improvements only apply to running collections when checking multiple packages. On my machine, the parallelization limit for a package is effectively determined by the dependencies between the collections on my machine. Instead the improvements comes from reducing the number of system(3) (or fork+exec) calls Lintian does. Mostly through using xargs more, even if it meant slightly more complex code. But also, libperlio-gzip-perl shaved off a couple of seconds on binaries check. But as I said, linux-image is not your average package . Most of the improvements mentioned here are hardly visible on other packages. So let s have a look at some more other bottlenecks. In my experience the following are the worst offenders : But enough Lintian for now time to fix some RC bugs!

17 June 2012

Bastian Blank: DDoS against DNS-Servers

Today I found out that the DNS-Server at home is used for a DDoS against a DNS-Server. The attackers send a small query, "ripe.net ANY" in this case, with a faked sender IP. The DNS-Server answers with a much larger packet to the (faked) sender. Because the domain is not local, the nameserver should have only produced an error. But the bind-config allowed everyone to get answers to cached answers:
allow-query-cache   0.0.0.0/0; ::/0;  ;
With proper restriction, the server only returns errors now. Restricting answer for cached entries does not help if the queried nameserver is authoritative for the domain. In this case it can help to drop queries for the ANY type. In ferm this looks like this:
proto udp dport domain mod string from 40 algo bm hex-string ' 0000ff0001 ' DROP;

27 May 2012

Bastian Blank: Installing Wheezy on Thinkpad X130e

I installed a Lenovo Thinkpad X130e with Wheezy.
UEFI The X130e does not want to boot via BIOS emulation. The Wheezy D-I installs only BIOS Grub. It does not provide the selection to use EFI at all. So the initial installation was BIOS only and EFI refused to boot it without an error. BIOS emulation simply don't want to work on the X130e. Partman does not align partitions in GPT. For some reasons the partitions created by partman are not aligned in the partition table at all. This is bad for performance. Also it makes it impossible to create an EFI partition on it that is recognizable by Grub. EFI Grub needs the EFI partition. This partition needs to be properly aligned and formatted with FAT16. It must be mounted on /boot/efi. Grub gets installed on this with the following commands:
apt-get install grub-efi-amd64
grub-install --bootloader-id=GRUB --removable
Grub Grub fails to load EFI console stuff. Grub needs extra modules to allow the kernel to write on the console. Otherwise the kernel just hangs without output. This can be done via /boot/grub/custom.cfg to load them:
insmod efi_gop
insmod efi_uga
Radeon The Radeon card needs a firmware. It just produces random output. To get a usable output, Radeon modeset must be disabled on the kernel command line:
radeon.modeset=0
Now the firmware can be installed:
apt-get install firmware-linux
Wireless card The Broadcom wireless card needs a firmware:
apt-get install firmware-brcm80211

7 April 2012

Bastian Blank: Booting Debian via AoE (ATA over Ethernet) and PXE

AoE is one of the protocols supported by Linux to access storage via network. It uses plain Ethernet for communication and includes a discovery mechanism to find all available targets. I use it to provide disk space to VMs running on different machines. Next step is to boot via AoE. It is no real problem to use AoE in running systems. However with some help it is even possible to actually boot disk-less machines via AoE. The PXE implementation iPXE provides the AoE support to actually boot from. I will describe the necessary parts.
Setup vblade The AoE target used it vblade. vblade needs access to raw sockets. As I prefer to not have run anything as root if it is not necessary, I use filesystem capabilities to allow it access to the network.
setcap cap_net_raw+ep /usr/sbin/vblade
vblade gets the mac address of the initiator, the shelf and slot number, the network device and the block device.
/usr/sbin/vblade -m $mac 0 0 eth0 $dev
Setup a tftp server and iPXE
apt-get install atftpd ipxe
ln -s /usr/lib/ipxe/undionly.kpxe /var/lib/tftpboot
Setup ISC dhcpd The dhcp server needs to be configured. It needs to hand out two distinct parameter sets. The first is used to chain-load iPXE into the normal PXE stack. The second is for iPXE and sets the root path to the AoE device. They are selected on the iPXE marker in the request.
if exists user-class and option user-class = "iPXE"  
  filename "";
  option root-path "aoe:e0.0";
  else  
  filename "undionly.kpxe";
 
Support AoE via initramfs-tools The initramfs needs to initialize AoE support. It needs to enable the network device used for communication with the AoE server and wait until it is up. After that it needs to load the aoe module and run aoe-discover. We should have all devices now. The root device can now be used like any other normal device. After the AoE device is initialized, it can be found via UUID and all the other ways. So no further modifications are necessary over the usage of local disks. The initramfs finds the device as usual and boots from it. The initramfs support is still a prototyp, but seems to work. For initramfs-tools it needs a hook to include all necessary stuff in the initramfs and a script to actually do the work. Both are shown here. /etc/initramfs-tools/hooks/aoe:
#!/bin/sh
case $1 in
prereqs)
  echo "udev"
  exit 0
  ;;
esac
. /usr/share/initramfs-tools/hook-functions
copy_exec /sbin/aoe-discover
manual_add_modules aoe
/etc/initramfs-tools/scripts/local-top/aoe:
#!/bin/sh
case $1 in
prereqs)
  echo "udev"
  exit 0
  ;;
esac
ifconfig eth0 up
sleep 10
modprobe aoe
aoe-discover
udevadm settle --timeout=30
Problems Not all parts of this works 100%. Some parts works not for all hardware.
  • My old notebook is not able to run Linux with this setup. grub loads the kernel via AoE and nothing comes later.
  • The network may need more time. Especially in large environments with spanning tree enabled, it may need half a minute until any packets will flow.
Some of the problems can be addressed later. Some can't.

12 March 2012

Stefano Zacchiroli: debian contributions to the linux kernel

The statistics of the "who wrote Linux x.y.z" series date back to at least 2.6.20. According to my experience talking with users and Free Software enthusiasts, those statistics really make a dent in the public perception of who is giving back upstream. Obviously, one should not take a single upstream, even if it is as important as the Linux kernel, as a measure of how much a given Free Software entity is giving back upstream overall. But users still seem to be fascinated by them. As a result, I have often had to answer the question: why Debian doesn't show up on those statistics?. My answer has always been something along the lines that Debian Developers who maintain Linux kernel packages, the almighty Debian Kernel Team, do that mostly as part of their volunteer engagement in Debian. As a consequence, they do not earmark their contributions as if they worked for a company and they add up to the hobbyist count instead (although you can you can routinely spot individual Debian Kernel Team members among the most active contributors for specific Linux releases). The above is the true and honest answer. But every time I've given it, I couldn't help feeling that the user who asked went home with a "yeah, well" afterthought. If you don't want to take my word of it, fine. Here is what Greg K-H had to say about Debian contributions in a recent blog post about the stable Linux kernel:
I would personally like to thank the Debian kernel developers, specifically Ben Hutchings, Maximilian Attems, Dann Frazier, Bastian Blank, and Moritz Muehlenhoff. They went above and beyond what any "normal" developer would have done, ferreting patches out of the kernel.org releases and the different vendor kernels and bug tracking systems, backporting them to the 2.6.32 kernel, testing, and then forwarding them on to me. Their dedication to their user community is amazing for such a "volunteer" group of developers. I firmly believe that without their help, the 2.6.32 kernel would not have been the success that it was. The users of Red Hat and SuSE products owe them a great debt. Buy them a beer the next time you see them, they more than deserve it.
I'll take good care of following his wise advice. Please do the same.
(Thanks to Sylvestre for pointing me to Greg's blog post.)

Next.