Enrico Zini: Migrating from procmail to sieve
Anarcat's "procmail considered harmful" post
convinced me to get my act together and finally migrate my venerable procmail based setup to sieve.
My setup was nontrivial, so I migrated with an intermediate step in which sieve
scripts would by default pipe everything to procmail, which allowed me to
slowly move rules from
I did not find an obvious way in sieve to create montly mailboxes, so I
redesigned that system using Postfix's
I added this to the local dovecot configuration:
This makes Dovecot ready to receive mail from Postfix via a lmtp unix socket
created in Postfix's private chroot.
It also activates the sieve plugin, and uses
and then created a script for it:
And I can have a sieve script that delegates processing to procmail:
Activate the postfix side
These changes switched local delivery over to Dovecot:
Without
Then I created a filter script:
And now what was previously:
Can become:
Updates
Ansgar mentioned that it's possible to replicate the monthly mailbox using the
variables and
date
extensions, with a hacky trick from the extensions' RFC:
procmailrc to sieve until nothing remained in
procmailrc.
Here's what I did.
Literature review
https://brokkr.net/2019/10/31/lets-do-dovecot-slowly-and-properly-part-3-lmtp/
has a guide quite aligned with current Debian, and could be a starting point to
get an idea of the work to do.
https://wiki.dovecot.org/HowTo/PostfixDovecotLMTP is way more terse, but
more aligned with my intentions. Reading the former helped me in understanding
the latter.
https://datatracker.ietf.org/doc/html/rfc5228 has the full Sieve syntax.
https://doc.dovecot.org/configuration_manual/sieve/pigeonhole_sieve_interpreter/
has the list of Sieve features supported by Dovecot.
https://doc.dovecot.org/settings/pigeonhole/ has the reference on Dovecot's
sieve implementation.
https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-extprograms.txt
is the hard to find full reference for the functions introduced by the
extprograms plugin.
Debugging tools:
- doveconf to dump dovecot's configuration to see if what it understands matches what I mean
- sieve-test
parses sieve scripts:
sieve-test file.sieve /dev/nullis a quick and dirty syntax check
BACKUP="/srv/backupts/test- date +%Y-%m-d .mbox"
:0c
$BACKUP
always_bcc
feature, piping everything to an archive user.
I'll then recreate the monthly archiving using a
chewmail script that I can simply run
via cron.
Configure dovecot
apt install dovecot-sieve dovecot-lmtpd
service lmtp
unix_listener /var/spool/postfix/private/dovecot-lmtp
user = postfix
group = postfix
mode = 0666
protocol lmtp
mail_plugins = $mail_plugins sieve
plugin
sieve = file:~/.sieve;active=~/.dovecot.sieve
~/.sieve as a sieve script.
The script can be a file or a directory; if it is a directory,
~/.dovecot.sieve will be a symlink pointing to the .sieve file to run.
This is a feature I'm not yet using, but if one day I want to try enabling UIs
to edit sieve scripts,
that part is ready.
Delegate to procmail
To make sieve scripts that delegate to procmail, I enabled the
sieve_extprograms plugin:
plugin
sieve = file:~/.sieve;active=~/.dovecot.sieve
+ sieve_plugins = sieve_extprograms
+ sieve_extensions +vnd.dovecot.pipe
+ sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe
+ sieve_trace_dir = ~/.sieve-trace
+ sieve_trace_level = matching
+ sieve_trace_debug = yes
mkdir -p /usr/local/lib/dovecot/sieve-pipe/
(echo "#!/bin/sh'; echo "exec /usr/bin/procmail") > /usr/local/lib/dovecot/sieve-pipe/procmail
chmod 0755 /usr/local/lib/dovecot/sieve-pipe/procmail
require "vnd.dovecot.pipe";
pipe "procmail";
--- a/roles/mailserver/templates/dovecot.conf
+++ b/roles/mailserver/templates/dovecot.conf
@@ -25,6 +25,8 @@
+auth_username_format = %Ln
+
diff --git a/roles/mailserver/templates/main.cf b/roles/mailserver/templates/main.cf
index d2c515a..d35537c 100644
--- a/roles/mailserver/templates/main.cf
+++ b/roles/mailserver/templates/main.cf
@@ -64,8 +64,7 @@ virtual_alias_domains =
-mailbox_command = procmail -a "$EXTENSION"
-mailbox_size_limit = 0
+mailbox_transport = lmtp:unix:private/dovecot-lmtp
auth_username_format = %Ln dovecot won't be able to understand
usernames sent by postfix in my specific setup.
Moving rules over to sieve
This is mostly straightforward, with the luxury of being able to do it a bit at
a time.
The last tricky bit was how to call spamc from sieve, as in some situations I
reduce system load by running the spamfilter only on a prefiltered selection of
incoming emails.
For this I enabled the filter directive in sieve:
plugin
sieve = file:~/.sieve;active=~/.dovecot.sieve
sieve_plugins = sieve_extprograms
- sieve_extensions +vnd.dovecot.pipe
+ sieve_extensions +vnd.dovecot.pipe +vnd.dovecot.filter
sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe
+ sieve_filter_bin_dir = /usr/local/lib/dovecot/sieve-filter
sieve_trace_dir = ~/.sieve-trace
sieve_trace_level = matching
sieve_trace_debug = yes
mkdir -p /usr/local/lib/dovecot/sieve-filter/"
(echo "#!/bin/sh'; echo "exec /usr/bin/spamc") > /usr/local/lib/dovecot/sieve-filter/spamc
chmod 0755 /usr/local/lib/dovecot/sieve-filter/spamc
:0 fw
/usr/bin/spamc
:0
* ^X-Spam-Status: Yes
.spam/
require "vnd.dovecot.filter";
require "fileinto";
filter "spamc";
if header :contains "x-spam-level" "**************"
discard;
elsif header :matches "X-Spam-Status" "Yes,*"
fileinto "spam";
require "date"
require "variables"
if currentdate :matches "month" "*" set "month" "$ 1 ";
if currentdate :matches "year" "*" set "year" "$ 1 ";
fileinto :create "$ month -$ year ";
My monthly report covers a large part of what I have been doing in the free software world. I write it for 
.
Further I used the upload of poppler (DLA 446-1) to test the workflow of the new wheezy-security upload. Uploading and building packages worked perfectly. Unfortunately the push to the security mirrors was a bit delayed (it only happened after an upload of the security team). But this seems to be fixed by Ansgar now.
Last but not least I had a look at PHP5. I think I will start my regular uploads in May.
Other stuff
As I had to deal with non-Debian stuff this month, I didn t do lots of other things. I only uploaded node-uml
Data is from 
What happened in the 
Portability toward other systems has been improved: old versions of GNU diff are now supported (Mike McQuaid), suggestion of the appropriate locale is now the more generic 
For the concrete work I did during DebConf, I can only claim two schroot uploads to fix the lack of support of the new overlay filesystem that replaces aufs in the official Debian kernel, and some Distro Tracker work (fixing an issue that some people had when they were logged in via Debian s SSO).
While the numerous discussions I had during DebConf can t be qualified as work , they certainly contribute to build up work plans for the future:
As a Kali developer, I attended multiple sessions related to derivatives (notably the 
Following an idea by Ansgar Burchardt, I've done some digging on version numbers in Debian:
Most common version numbers:
Playing the 
Big release notes since 1.0:
We ve got a new list 
liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never 
This year our efforts have paid off and despite there being more mentoring organizations than there were in 2011 (175 in 2011 and 180 in 2012), this year in Debian we got 81 submissions versus 43 submissions in 2011.
The result is this year we ll have 15 students in Debian versus 9 students last year! Without further ado, here is the list of projects and student who will be working with us this summer: