Setting up a mail server

Background Some terminology:

MTA

Mail Transport Agent

MUA

Mail User Agent

MDA

Mail Delivery Agent

SMTP

Simple Mail Transfer Protocol

MX

Mail eXchange

POP

Post Office Protocol

IMAP

Internet Message Access Protocol

With SMTP you connect to a server and send two things: envelope and message. The envelope looks like this:

MAIL FROM: <enrico@enricozini.org>
RCPT TO: <rms@fsf.org>
RCPT TO: <linus@linux.org>

The message looks like this:

From: <enrico@enricozini.org>
To: <rms@fsf.org>
Cc: <linus@linux.org>
Message-ID: <1234567@enricozini.org>
Subject: Test mail
Hi Richard,
this is a test mail.  I'm also writing
Linus to show how to send to more people.
Cheers,
Enrico

There is no authentication. There is no encryption. Two usual types of access control:

1. Outbound e-mail is normally only accepted from an internal network
2. Inbound e-mail is normally accepted from anywhere

The DNS is used to find the SMTP server to use to send a message:

$ host -t MX yahoo.com
yahoo.com MX 10 smtp1.yahoo.com
yahoo.com MX 20 smtp2.yahoo.com
yahoo.com MX 20 smtp3.yahoo.com

The process of sending an E-Mail:

1. Enrico writes an E-Mail:

   From: Enrico Zini <enrico@enricozini.org>
   To: Richard Stallman <rms@fsf.org>
   Subject: Hello from Addis
   Hi Richard,
   Addis is a wonderful city, even if
   it rains a lot.
   Bye,  Enrico
   

2. Enrico's MUA connects to the SMTP server (for example, port 25 of smtp.aau.edu.et):

   HELO enricozini.org
   200 OK Hello enricozini.org
   MAIL FROM: <enrico@enricozini.org>
   200 OK Mail from enrico@enricozini.org
   RCPT TO: <rms@fsf.org>
   

   Here, the SMTP server performs relay control: "do we relay mail to rms@fsf.org?":

   • Outbound e-mail is normally only accepted from an internal network
   • Inbound e-mail is normally accepted from anywhere

   A target address could be refused:

   413 ERR I don't relay for rms@fsf.org
   

   In this case, the destination is not local but the recipient is accepted because I'm inside the local network:

   200 OK Destination rms@fsf.org
   DATA
   200 OK Please send message body
   From: Enrico Zini <enrico@enricozini.org>
   To: Richard Stallman <rms@fsf.org>
   Subject: Hello from Addis
   Date: Mon, 17 Jul 2006 09:49:45 +0300
   Message-ID: <124372643@enricozini.org>
   Hi Richard,
   Addis is a wonderful city, even if
   it rains a lot.
   Bye,  Enrico
   .
   200 OK Message accepted
   QUIT
   200 OK Bye.
   

3. The SMTP server needs to find out where to send the message, using the DNS:

   $ host -t MX fsf.org
   fsf.org MX 10 mail.fsf.org
   fsf.org MX 20 mail.gnu.org
   

4. So the SMTP server tries the first one and connects to port 25 of mail.fsf.org:

   HELO smtp.aau.edu.et
   200 OK Hello smtp.aau.edu.et
   MAIL FROM: <enrico@enricozini.org>
   200 OK Mail from enrico@enricozini.org
   RCPT TO: <rms@fsf.org>
   

   The destination is accepted because it's for a local user:

   200 OK Destination rms@fsf.org
   DATA
   200 OK Please send message body
   From: Enrico Zini <enrico@enricozini.org>
   To: Richard Stallman <rms@fsf.org>
   Subject: Hello from Addis
   Date: Mon, 17 Jul 2006 09:49:45 +0300
   Message-ID: <124372643@enricozini.org>
   Received: by mail.aau.edu.et
     on Mon, 17 Jul 2006 09:55:53 +0300
     from 10.4.15.158
   Hi Richard,
   Addis is a wonderful city, even if
   it rains a lot.
   Bye,  Enrico
   .
   200 OK Message accepted
   QUIT
   200 OK Bye.
   

5. Now, mail.fsf.org will invoke a MDA to write the mail in Richard Stallman's mailbox.

Example of problems with mail handling:

• Accepting inbound connections:
  • Malicious input:
    • logic errors
    • buffer overflows
    • DoS (Denial Of Service) attacks
    • Connection floods
• Performing outbound connections:
  • Programming errors:
    • Flooding of connections
• Performing routing:
  • Unauthorised relays
  • Mail loops
• Writing to the local hard drive:
  • Filling up the hard drive
  • Writing to the wrong files
• Writing to the local hard drive as root:
  • In case of error or attack, any file in the system can potentially be compromised

RFC-822 is the original standard for E-mail. RFCs are standard Internet documents. Have a look at RFC documents released the 1st of April.

postfix Common setup: "Internet site with smarthost". More difficult to maintain: "Internet site". A smarthost is a machine that will relay e-mail for you. Questions asked with "Internet site with smarthost":

• Mail name: aau.edu.et (name used to publicly identify the mail server)
• Smarthost name: smtp.telecom.net.et (SMTP server that will relay our e-mail)

To test a mail server:

$ telnet localhost 25
HELO me
MAIL FROM: <a@b.c>
RCPT TO: <mail@of.a.local.user>
DATA
hi
.
QUIT

By default, you find locally delivered mail in /var/mail/username. Postfix configuration files:

/etc/postfix/master.cf

configures how all the postfix components run together (man 5 master)

/etc/postfix/main.cf

Main postfix configuration (man 5 postconf)

To rewrite addresses:

1. In /etc/postfix/main.cf:

   canonical_maps = hash:/etc/postfix/canonical
   

2. Then in /etc/postfix/canonical you can add the rewrite rules, like:

   enrico   enrico@enricozini.org
   

3. When /etc/postfix/canonical is modified you need to regenerate the index:

   sudo postmap canonical
   

   (same is when you change the alias file: sudo postalias /etc/aliases)

(see file:///usr/share/doc/postfix/html/ADDRESS_REWRITING_README.html) Manipulating the message queue:

mailq - List the mail queue.

Example:

mailq

postqueue - Postfix queue control

Examples:

# Like mailq
postqueue -p
# Tries to send every message in the queue
postqueue -f
# Tries to send every message in the queue for that site
postqueue -s site

postsuper - Postfix superintendent

Examples:

# Deletes one message
sudo postsuper -d 7C4D2EC0F5D
# Deletes all messages held in the queue for later delivery
sudo postsuper -d ALL deferred

Different mail queues in postfix:

incoming

mail who just entered the system

active

mail to be delivered

deferred

mail to be delivered later because there were problems

hold

mail that should not be delivered until released from hold

Mail logs are in:

/var/log/mail.log
/var/log/mail.err
/var/log/mail.info
/var/log/mail.warn

Mail delivery Mailbox formats:

• mbox: single file, mail separated by "From " lines
• maildir: one directory per folder, one file per mail
• mh: similar to maildir, but not really used

Alternate MDA: procmail: allows to filter mail automatically into different folders. Mail forwarding with ~/.forward: allows to redirect mail to a different address: just put the address you want to send to in the file ~/.forward.

POP or IMAP server Installation:

apt-get install dovecot

Configuration is in:

/etc/dovecot/dovecot.conf

The main thing that is needed is to enable the mail protocols you want:

protocols = imaps

Server monitoring To make all sorts of graphs:

apt-get install munin munin-node

Example: http://munin.ping.uio.no To compute more statistics:

• anteater
• isoqlog
• mailgraph

Monitor system logs: logcheck:

• sends you mail with abnormal log lines
• It's important to customize what is normal and you do it with regular expressions

Filtering viruses and spam clamav - Virus scanner Virus scanning:

• Postfix gives the mail to clamav that scans it and gives it back if it's clean.
• Strategies for infected mail:
  • silently delete it
  • refuse the mail and send a notification to the sender
  • refuse the mail and send a notification to the receiver
  • quarantine the e-mail
  • refuse delivery with a SMTP error
  • deliver with an extra header that says that it's a virus

spamassassin - Spam filter Spam scanning:

• Postfix gives the mail to spamd that scans it and gives it back with some spam information.
• Strategies for spam mail:
  • silently delete it
  • refuse the mail and send a notification to the sender
  • refuse the mail and send a notification to the receiver
  • quarantine the e-mail
  • refuse delivery with a SMTP error
  • deliver with an extra header that says that it's spam
• New techniques:
  • greylisting: when you receive a mail from a host you've never seen before, refuse it with a temporary error, and accept it the second time (after some time delay). Spammers normally don't retry, and implementing retry would increase their cost of sending e-mail.
  • crossassassin: if more than some amount of your users receive a mail with the same message ID, throw it away. Sending mails with different headers would increase the cost of sending e-mail.

Man pages and sections Man pages are divided in sections:

• man man shows all the sections of the manpages
• man 5 postconf shows the postconf manpage in the "configuration file" section
• Normally manpages are referred as manpage(section) (e.g. postconf(5) )

drwxr-xr-x   2 root root    38 2006-07-14
 
+- Is a directory
drwxr-xr-x   2 root root    38 2006-07-14
 ---
   
  +- User permissions (u)
drwxr-xr-x   2 root root    38 2006-07-14
    ---
      
     +- Group permissions (g)
drwxr-xr-x   2 root root    38 2006-07-14
       ---
         
        +- Permissions for others (o)
drwxr-xr-x   2 root root    38 2006-07-14
               ----
                 
                +- Owner user
drwxr-xr-x   2 root root    38 2006-07-14
                    ----
                      
        Owner group -+

• 4000 Set user ID:
  • For executable files: run as the user who owns the file, instead of the user who runs the file
  • For directories: I think it's not used
• 2000 Set group ID:
  • For executable files: run as the group who owns the file, instead of the group of the user who runs the file
  • For directories: when a file is created inside the directory, it belongs to the group of the directory instead of the default group of the user who created the file
• 1000 Sticky bit:
  • For files: I think it's not used anymore
  • For directories: only the owner of a file can delete or rename the file

• chown - change file owner and group
• chgrp - change group ownership
• chmod - change file access permissions
• sudo adduser enrico www-data adds the user enrico to the group www-data.

# Create the group 'students'
mkdir /var/www/students
chgrp students /var/www/students
chmod 2775 /var/www/students
# If you don't want other users to read the files of the students:
chmod 2770 /var/www/students
adduser www-data students
 (this way the web server can read the
  pages)
# when you add a user to a group, it does not affect running processes:
 - users need to log out and in again
 - servers need to be restarted

• To install apache2 without a graphical interface:

  apt-cache search apache2   less
  sudo apt-get install apache2
  

• By default, /var/www is where is the static website.

• By default, ~/public_html is the personal webspace for every user, accessible as: http://localhost/~user

• By default, /usr/lib/cgi-bin contains scripts that are executed when someone browses http://website/cgi-bin/script

• By default, apache reads the server name from the DNS. If we don't have a name in the DNS and we want to use the IP, we need to set:

  ServerName 10.4.15.158
  

  in /etc/apache/apache2.conf (set it to your IP address)

• To access the Apache manual: http://localhost/doc/apache2-doc/manual/

• http://localhost/doc/apache2-doc/manual/mod/mod_access.html The access control module

• http://localhost/doc/apache2-doc/manual/mod/mod_auth.html The user authentication module

• To edit a user password file, use:

  htpasswd - Manage user files for basic authentication
  

• Example .htaccess file to password protect a directory:

  AuthUserFile /etc/apache2/students
  AuthType Basic
  AuthName "Students"
  Require valid-user
  

• Information about .htaccess is in http://localhost/doc/apache2-doc/manual/howto/htaccess.html

• If you need to tell apache to listen on different ports, add a Listen directive to /etc/apache2/ports.conf. Then you can use:

  <VirtualHost www.training.aau.edu.et:9000>
  [...]
  </VirtualHost>
  

• To setup an HTTPS website:

  • Documentation is in http://localhost/doc/apache2-doc/manual/ssl/

  • How to create a certificate: http://www.tc.umn.edu/~brams006/selfsign.html

  • Create a certificate:

    /usr/sbin/apache2-ssl-certificate -days 365
    

  • Create a virtual host on port 443:

    <VirtualHost www.training.aau.edu.et:443>
    [...]
    </VirtualHost>
    

  • Enable SSL in the VirtualHost:

    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/apache.pem
    

  • Enable listening on the HTTPS port (/etc/apache2/ports.conf):

    Listen 443
    

• check that there are no errors in the configuration file:

  apache2ctl configtest
  

  This it is always a good thing to do before restarting or reloading apache.

• read logs in /var/log/apache2/

• if you made a change but you don't see it on the web, it can be that you have the old page in the cache of the browser: try reloading a few times.

• apt-get install libapache2-module-php5

• then by default, every file .php is executed as php code

• Small but useful test php file:

  <? phpinfo() ?>
  

• apt-get install mysql-client mysql-server

• for administration run mysql as root:

  • Create a database with:

    create database students
    

• Give a user access to the database:

  # Without password
  grant all on students.* to enrico;
  # With password
  grant all on students.* to enrico identified by "SECRET";
  

• More information can be found at http://www-css.fnal.gov/dsg/external/freeware/mysqlAdmin.html

apt-get install php5-mysqli php5-mysql

• the apache2 manual in /usr/share/doc/manual can only be viewed using apache because it uses MultiView. So you need to have a working apache to read how to have a working apache.
• chmod does not have examples in the manpage.

• ifconfig - configure a network interface

• dnsmasq - Simple DNS and DHCP server

• host - DNS lookup utility

• route - show / manipulate the IP routing table

• arping - send ARP REQUEST to a neighbour host

• mii-tool - view, manipulate media-independent interface status (IOW, see if the cable works)

• nmap - Network exploration tool and security / port scanner Examples:

  # Look at what machines are active in the local network:
  nmap -sP 10.5.15.0/24
  # Look at what ports are open in a machine:
  nmap 10.5.15.26
  

• tcpdump - dump traffic on a network It can be used to see if there is traffic, and to detect traffic that shouldn't be there.

# Convert a unix timestamp to a readable date
date -d @1152841341

1. type the address www.google.com in the browser
2. the browser needs the IP address of the web server:

1. look for the DNS address in /etc/resolv.conf (/etc/resolv.conf is created automatically by the DHCP client)
2. try all the DNS servers in /etc/resolv.conf until one gives you the IP address of www.google.com
3. take the first address that comes from the DNS (in our case was 64.233.167.104)

3. figure out how to connect to 64.233.167.104:

1. consult the routing table to see if it's in the local network:

1. if it's in the local network, then look for the MAC address (using ARP - Address Resolution Protocol)
2. if it'd not in the local network, then send through the gateway (again using ARP to find the MAC address of the gateway)

4. Send out the HTTP request to the local web server or through the gateway, using the Ethernet physical protocol, and the MAC address to refer to the other machine.

1. See if the network driver works:

1. With ifconfig, see if you see the HWaddr:. If you do not see it, then the linux driver for the network card is not working. Unfortunately there's no exact way to say that it works perfectly

2. See if you have an IP address with ifconfig. If you find out that you need to rerun DHCP (for example, if the network cable was disconnected when the system started), then you can do it either by deactivating/reactivating the Ethernet interface using System/Administration/Networking or, on a terminal, running:

   # ifdown eth0
   # ifup eth0
   

   If you don't get an IP, try to see if the DHCP server is reachable by running:

   $ arping -D [address of DHCP server]
   

3. See if the local physical network works:

1. With sudo mii-tool, see if the cable link is ok. If it's not, then it's a problem in the cable or the plugs, or simply the device at the other end of the cable is turned off.
2. Try arping or ping -n on a machine in the local network (like the gateway) to see if the local network works.

4. See if the DNS works:

1. Find out the DNS address:

   cat /etc/resolv.conf
   

2. If it's local, arping it

3. If it's not local, ping -n it

4. Try to resolve a famous name using that DNS:

   $ host [name] [IP address of the DNS]
   

5. Try to resolve the name of the machine you're trying to connect. If you can resolve a famous name but not the name you need, then it's likely a problem with their DNS.

5. If you use a proxy, see if the proxy is reachable: check if the proxy name resolves to an IP, if you can ping it, if you can telnet to the proxy address and port:

   $ telnet [proxy address] [proxy port]
   

   you quit telnet with ^]quit.

6. If you can connect directly to the web server, try to see if it answers:

   $ telnet [address] 80
   

   If you are connected, you can confirm that it's a web server:

   GET / HTTP/1.0 (then Enter twice)
   

   If it's a web server, it should give you something like a webpage or an HTTP redirect.

1. check that it's running:

   $ ps aux   grep dnsmasq
   

2. check that it's listening on the right port:

   $ sudo netstat -lp
   

3. check that it's listening from the outside:

   $ nmap [hostname]
   

4. check for messages in /var/log/daemon.log or /var/log/syslog

5. check that the configuration is correct and reload or restart the server to make sure it's running with the right configuration:

   # /etc/init.d/dnsmasq restart
   

By default: works as a DNS server that serves the data in /etc/hosts. By default: uses /etc/resolv.conf to find addresses of other DNS to use when a name is not found in /etc/hosts. To enable the DHCP server, uncomment:

dhcp-range=192.168.0.50,192.168.0.150,12h

in /etc/dnsmasq.conf and set it to the range of addresses you want to serve. Pay attention to never put two DHCP servers on the same local network, or they will interfere with each others. To test if the DHCP server is working, use dhcping (not installed by default on Ubuntu). To communicate other information like DNS, gateway and netmask to the clients, use this piece of dnsmasq.conf:

# For reference, the common options are:
# subnet mask - 1
# default router - 3
# DNS server - 6
# broadcast address - 28
dhcp-option=1,255.255.255.0
dhcp-option=3,192.168.0.1
dhcp-option=6,192.168.0.1
dhcp-option=28,192.168.0.255

• changing the name of the local machine in /etc/hosts breaks sudo, and without sudo it's impossible to edit the file. The only way to fix this is a reboot in recovery mode.
• dhclient -n -w is different than dhclient -nw

# Create an archive
tar zcvf nmap.tar.gz *.deb
# Extract an archive
tar zxvf nmap.tar.gz
# Look at the contents of an archive
tar ztvf nmap.tar.gz

# To send
nc -l -p 12345 -q 1 < nmap.tar.gz
# To receive
nc 10.5.15.123 12345 > nmap.tar.gz
# To repeat the send command 20 times
for i in  seq 1 20 ; do nc -l -p 12345 -q 1 < nmap.tar.gz ; done