How to Install Spamassassin with Postfix and Dovecot on Ubuntu/Debian Server

Spam free email with SpamAssassin

If you are running your own mailbox on a server using Postfix and Dovecot then it is imperative to install an anti-spam software to protect your mailbox from getting bombed with incoming spam mails.

And for this SpamAssassin is one of the best solutions available out there. It is quite effective and one of the most popular.

In this post we shall see how to setup spamassassin with postfix and dovecot for spam protection.

We assume that you have already setup Postfix and Dovecot on your server for handling incoming mails.

And that you are using Dovecot LMTP for delivering mails for user accounts. The process had been outlined in a previous post here -

Setup a mail server with Postfix and Dovecot on Ubuntu / Debian

Spamassassin scans incoming emails to identify spam and if a mail is found to be potentially a spam mail then Spamassassin adds headers to that email to mark it a spam.

Spam checking happens at the Postfix level. Postfix is configured to talk to Spamassassin as a content_filter.

In the next step, Dovecot analyses these headers to identify spam messages and puts them in a separate Spam folder, just like Gmail.

1. Install Spamassassin

The first thing would be to install spamassassin packages.

$ apt-get install spamassassin spamc

2. Setup a user account and group for spamd service

# groupadd spamd
# useradd -g spamd -s /bin/false -d /var/log/spamassassin spamd
# mkdir /var/log/spamassassin
# chown spamd:spamd /var/log/spamassassin

3. Configure Spamassassin

Edit /etc/default/spamassassin so these options are set:

# Change to one to enable spamd
ENABLED=1

SAHOME="/var/log/spamassassin/"

# Options
# See man spamd for possible options. The -d option is automatically added.

# SpamAssassin uses a preforking model, so be careful! You need to
# make sure --max-children is not set to anything higher than 5,
# unless you know what you're doing.

OPTIONS="--create-prefs --max-children 5 --helper-home-dir --username spamd \
-H ${SAHOME} -s ${SAHOME}spamd.log"

Also find the option called CRON at the bottom and change it to 1. This will enable automatic updating of spamassassin rules as they are released.

# Cronjob
# Set to anything but 0 to enable the cron job to automatically update
# spamassassin's rules on a nightly basis
CRON=1

4. Start the Spamassassin daemon (spamd)

# service spamassassin start
Starting SpamAssassin Mail Filter Daemon: spamd.
root@localhost:/etc/default# 
root@localhost:/etc/default# service spamassassin status
 * spamd is running
root@localhost:/etc/default#

Spamassassin starts a background service, so use the netstat command to ensure that spamassassin is running fine or not.

# netstat -nlp4 | grep spam
tcp        0      0 127.0.0.1:783           0.0.0.0:*               LISTEN      1123/spamd.pid

4. Configure Postfix to use Spamassassin as a filter

Postfix needs to be configured to use spamassassin service to filter all incoming mails
The postfix configuration file master.cf needs to be edited to add spamassassin filter. Open the file.

# cat /etc/postfix/master.cf

Add the following at the end of the file

spamassassin unix -     n       n       -       -       pipe
        user=spamd argv=/usr/bin/spamc -f -e  
        /usr/sbin/sendmail -oi -f ${sender} ${recipient}

After adding the above lines to master.cf the next task is to restart postfix and make sure that it works fine.

$ service postfix restart

5. Configure Spamassassin to Mark Spam Messages

After configuring Postfix to use Spamassassin, its time to configure Spamassassin to modify emails according to spam checks. The configuration file is - /etc/spamassassin/local.cf

$ nano /etc/spamassassin/local.cf

Uncomment the rewrite_header line to modify the Subject lines of emails that are identified as spam.

#   Add *****SPAM***** to the Subject header of spam e-mails
#
# rewrite_header Subject *****SPAM*****

rewrite_header Subject [***** SPAM _SCORE_ *****]

The above line would add the Spam score with the asterisk marks to indicate that the message looks like spam. Higher scores would indicate higher probability of the message being a spam.

Spam Score

The minimum required score to mark the message as spam. The default is 5.0, but now a days you might need a more strict score level. So 4.0 works good.

#   Set the threshold at which a message is considered spam (default: 5.0)
#
required_score 4.0

Check out the other options in the file and edit them as required.

Restart Spamassassin - After configuring all parameters in the file, save the file and restart spamassassin.

# service spamassassin restart
Restarting SpamAssassin Mail Filter Daemon: spamd.

6. Check Spamassassin log

If at any point you need to debug something or view whats going in the background of spamassassin, feel free to check its look file.

# tail -f /var/log/spamassassin/spamd.log 
Fri Feb  5 20:11:43 2016 [1123] info: prefork: child states: II
Fri Feb  5 20:23:02 2016 [1123] info: spamd: server killed by SIGTERM, shutting down
Fri Feb  5 20:23:02 2016 [1404] info: logger: removing stderr method
Fri Feb  5 20:23:03 2016 [1406] info: zoom: able to use 360/360 'body_0' compiled rules (100%)
Fri Feb  5 20:23:03 2016 [1406] info: spamd: server started on IO::Socket::INET6 [127.0.0.1]:783, IO::Socket::INET6 [::1]:783 (running version 3.4.0)
Fri Feb  5 20:23:03 2016 [1406] info: spamd: server pid: 1406
Fri Feb  5 20:23:03 2016 [1406] info: spamd: server successfully spawned child process, pid 1407
Fri Feb  5 20:23:03 2016 [1406] info: spamd: server successfully spawned child process, pid 1408
Fri Feb  5 20:23:03 2016 [1406] info: prefork: child states: IS
Fri Feb  5 20:23:03 2016 [1406] info: prefork: child states: II

7. Test a Spam mail

Send an email to your mail system from outsite. Include the following piece of text in the email's body

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

Its called GTUBE - Generic Test for Unsolicited Bulk Email. Its a simple way to test if your mail server is using spamassassin to filter mails or not.

Learn more about it here -
http://spamassassin.apache.org/gtube/

SpamAssassin adds headers to all emails with spam scores > 0

Mail with spam score 12.9 This is considered spam according to our configuration of minimum score of 3

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on localhost
X-Spam-Flag: YES
X-Spam-Level: ************
X-Spam-Status: Yes, score=12.9 required=3.0 tests=FROM_LOCAL_NOVOWEL,
	HTML_FONT_LOW_CONTRAST,HTML_FONT_SIZE_LARGE,HTML_MESSAGE,HTML_OBFUSCATE_05_10,
	MIME_HTML_ONLY,MISSING_MID,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_BRBL_LASTEXT,
	RCVD_IN_PBL,RCVD_IN_PSBL,RCVD_IN_XBL,RDNS_NONE autolearn=no
	autolearn_force=no version=3.4.0

Spam Score 1.7, Not considered spam

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on localhost
X-Spam-Level: *
X-Spam-Status: No, score=1.7 required=3.0 tests=DEAR_SOMETHING,FREEMAIL_FROM,
	HTML_MESSAGE,RCVD_IN_MSPIKE_H2,URIBL_BLOCKED autolearn=no autolearn_force=no
	version=3.4.0

These headers shall be used to send emails into a separate Spam folder via Dovecot LMTP transport.

Configure Dovecot with separate Spam folder

Dovecot is the mail reading and managing system, POP3 and IMAP server. It is Dovecot that organises mails into separate folders depending on criterias.

Dovecot can be configured to read mail headers and move spam mails to a separate folder called Spam. Much like you see with free mail services like Gmail and Outlook.

Install and Configure Sieve Plugin

Dovecot needs the Sieve interpreter plugin to use Sieve to read mails. Install the Sieve packages

# apt-get install dovecot-sieve dovecot-managesieved

Edit
/etc/dovecot/conf.d/20-lmtp.conf

Configure dovecot to enable the sieve plugin

protocol lmtp {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins sieve
}

Finally its time to restart dovecot

# service dovecot restart

Now check if the sieve server is running on port 4190. Use the netstat command. The service will be named as dovecot.

# netstat -nltp | grep 4190
tcp        0      0 0.0.0.0:4190            0.0.0.0:*               LISTEN      9843/dovecot    
tcp6       0      0 :::4190                 :::*                    LISTEN      9843/dovecot

Setup Sieve filtering

To learn more about sieve filters, check out the following page - http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples

# mkdir /var/lib/dovecot/sieve/

Create a file called default.sieve and fill it with the following rule sets

require ["fileinto", "mailbox"];

if header :contains "X-Spam-Flag" "YES" {
        fileinto :create "Spam";
}

Now compile the .sieve script file using the sievec command.

# sievec default.sieve

If you have multiple .sieve files in the directory, then you can compile all of them in one go like this -

# sievec sieve/

Now change the ownership of the sieve files to the dovecot specific user so that dovecot can read them. In this case the username is vmail.

/var/lib/dovecot# chown -R vmail:vmail sieve/*

Restart Dovecot again.

# service dovecot restart

Check dovecot logs

Dovecot log files shall help you diagnose any problems that might come along the way.

Find the location of dovecot log files with the following command

# doveadm  log find
Debug: /var/log/dovecot.log
Info: /var/log/dovecot.log
Warning: /var/log/dovecot.log
Error: /var/log/dovecot.log
Fatal: /var/log/dovecot.log

Then open the file using the tail command

# tail -f /var/log/dovecot.log

Feb 06 11:17:16 imap-login: Info: Login: user=<[email protected]>, method=PLAIN, rip=122.163.23.27, lip=173.255.230.5, mpid=10370, TLS, session=<RfgSgBMr1QB6oxcb>
Feb 06 11:17:37 lmtp(10379): Info: Connect from local
Feb 06 11:17:37 lmtp(10379, [email protected]): Error: uMMvHPmItVaLKAAA0J78UA: sieve: main script: failed to stat sieve script: stat(/var/lib/dovecot/sieve/default.sieve) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +x perm: /var/lib/dovecot, dir owned by 0:0 mode=0750)
Feb 06 11:17:37 lmtp(10379, [email protected]): Error: uMMvHPmItVaLKAAA0J78UA: sieve: failed to access default user script /var/lib/dovecot/sieve/default.sieve
Feb 06 11:17:37 lmtp(10379, [email protected]): Info: uMMvHPmItVaLKAAA0J78UA: msgid=<CAKfwS=YH3TJ2XCxE0RzQqisBO54-QmYe9QJWAQd8-i207JN_1Q@mail.gmail.com>: saved mail to INBOX
Feb 06 11:17:37 lmtp(10379): Info: Disconnect from local: Successful quit
Feb 06 11:18:31 lmtp(10379): Info: Connect from local
Feb 06 11:18:31 lmtp(10379, [email protected]): Error: vMMvHPmItVaLKAAA0J78UA: sieve: msgid=<CAKfwS=YLa+1DYyX3HiQ7qQN=4PkmzCfX_1vB28A1gHT9xB4xjA@mail.gmail.com>: failed to store into mailbox 'Spam': Mailbox doesn't exist: Spam
Feb 06 11:18:31 lmtp(10379, [email protected]): Info: vMMvHPmItVaLKAAA0J78UA: sieve: msgid=<CAKfwS=YLa+1DYyX3HiQ7qQN=4PkmzCfX_1vB28A1gHT9xB4xjA@mail.gmail.com>: stored mail into mailbox 'INBOX'
Feb 06 11:18:31 lmtp(10379, [email protected]): Error: vMMvHPmItVaLKAAA0J78UA: sieve: execution of script /var/lib/dovecot/sieve/default.sieve;name=main script failed, but implicit keep was successful
Feb 06 11:18:31 lmtp(10379): Info: Disconnect from local: Successful quit
Feb 06 11:26:27 lmtp(10479): Info: Connect from local
Feb 06 11:26:27 lmtp(10479, [email protected]): Info: 3h4eBwuLtVbvKAAA0J78UA: sieve: msgid=<CAKfwS=b6cEU-WwhOkxaBjZcxb61dAZomyC4x98T4drLVFgeTxw@mail.gmail.com>: stored mail into mailbox 'Spam'
Feb 06 11:26:27 lmtp(10479): Info: Disconnect from local: Successful quit
Feb 06 11:26:57 imap-login: Info: Login: user=<[email protected]>, method=PLAIN, rip=122.163.23.27, lip=173.255.230.5, mpid=10483, TLS, session=<OreuohMrXgB6oxcb>
Feb 06 11:26:58 imap-login: Info: Login: user=<[email protected]>, method=PLAIN, rip=122.163.23.27, lip=173.255.230.5, mpid=10484, TLS, session=<xr6+ohMrXwB6oxcb>
Feb 06 11:28:30 imap-login: Info: Login: user=<[email protected]>, method=PLAIN, rip=122.163.23.27, lip=173.255.230.5, mpid=10499, TLS, session=<n7Q2qBMrCwB6oxcb>

Change the permissions of the dovecot directory

/var/lib/dovecot# chmod a+x .

Boost Spamassassin spam filtering with Pyzor

Pyzor will act just like other rules.
https://wiki.apache.org/spamassassin/Rules/PYZOR_CHECK

It will add an extra spam score of about 1.985 according to the tests table found at - http://spamassassin.apache.org/tests_3_3_x.html

# sudo apt-get install pyzor

Discover the Pyzor servers

# pyzor --homedir /etc/mail/spamassassin/pyzor discover

Make the servers file readable by all

/etc/spamassassin/pyzor# chmod a+r servers

Run a check -

# echo "test" | spamassassin -D pyzor 2>&1 | less

You should see some output like this

Feb  9 16:24:58.824 [23979] dbg: pyzor: network tests on, attempting Pyzor
Feb  9 16:24:59.787 [23979] dbg: pyzor: pyzor is available: /usr/bin/pyzor
Feb  9 16:24:59.788 [23979] dbg: pyzor: opening pipe: /usr/bin/pyzor check < /tmp/.spamassassin23979z26KDDtmp
Feb  9 16:24:59.841 [23979] dbg: pyzor: [23981] finished: exit 1
Feb  9 16:24:59.842 [23979] dbg: pyzor: check failed: no response

Add the following lines to the end of /etc/spamassassin/local.cf

pyzor_options --homedir /etc/mail/spamassassin/pyzor

And restart spamassassin daemon

# service spamassassin restart

Install Razor

sudo apt-get install razor

/etc/spamassassin# mkdir razor

Now run the following commands to register with the razor distributed network.

# razor-admin -home=/etc/spamassassin/razor -register
Register successful.  Identity stored in /etc/mail/spamassassin/razor/identity-rudkVFCIxs
# razor-admin -home=/etc/spamassassin/razor -create
# razor-admin -home=/etc/spamassassin/razor -discover

Add the following line at the end of /etc/spamassassin/local.cf file

razor_config /etc/spamassassin/razor/razor-agent.conf

Finally restart spamassassin

check razor working with this command

# echo "test" | spamassassin -D razor2 2>&1 | less

something like this should show up

Feb  9 17:11:34.701 [26028] dbg: razor2: razor2 is available, version 2.84
 Razor-Log: read_file: 15 items read from /etc/mail/spamassassin/razor/razor-agent.conf

Custom scores for higher scores

This is a little bit advanced way to increase the accuracy of spamassassin to catch spam better. It involves increasing the score levels for certain spam-tests inside of spamassassin. These are the spam tests that are known to have a very high level of accuracy in detecting spam.

So if some of these high accuracy tests indicate a mail as spam, we should focus more on it.

http://toao.net/566-improving-spamassassin-accuracy-with-statistics

Conclusion

That was the process to setup SpamAssassin on your mail server to detect spam mails and keep your mailbox clean.
The process is a bit lengthy but it works great if you need your own mail server with anti-spam system in place.

If you have any feedback or questions, please let us know in the comments below.

References and Resources

Simple Spamassassin setup with Postfix and Dovecot on Ubuntu Breezy
http://www.townx.org/index.php?q=blog/elliot/simple_spamassassin_setup_with_postfix_and_dovecot_on_ubuntu_breezy

How To Install and Setup Spamassassin on Ubuntu 12.04
https://www.digitalocean.com/community/tutorials/how-to-install-and-setup-spamassassin-on-ubuntu-12-04

Instructions on installing Pyzor and Razor with Spamassassin
https://kura.io/2011/09/22/spamassassin-razor-pyzor/

Setting up Sieve filtering with Dovecot
https://easyengine.io/tutorials/mail/server/sieve-filtering/

Spamassassin Tips : Ultimate Setup Guide
http://www.spamtips.org/p/ultimate-setup-guide.html

How to Improve SpamAssassin Accuracy
https://wiki.apache.org/spamassassin/ImproveAccuracy

On the effects of lowering your SpamAssassin threshold
http://taint.org/2008/02/29/155648a.html

Improving SpamAssassin accuracy on cPanel (or any other) mail servers - with statistics
http://toao.net/566-improving-spamassassin-accuracy-with-statistics

This post outlines simple hacks that you can use to change score levels for some of the spamassassin tests thereby improving spam filtering.

Advanced Tips for Using SpamAssassin
https://www.andrewferrier.com/oldpages/spamassassin_tips.html