We feel like we could almost write a whole "book" on SPAM - SPAM is the scourge of most e-mail providers and is a surprisingly complex issue to deal with effectively. Rather thank "write the book however" we cover just a few tactical issues related to SPAM.
NeatClubs provides mail hosting capabilities. For users who are dealing with a significant amount of SPAM, we can optionally setup software from the open-source Apache “Spam Assassin” project to filter incoming e-mail for individual problematics accounts. Spam Assassin uses a spectrum of sophisticated algorithms and techniques to identify SPAM e-mail. Our preferred approach is to not filter all e-mail for SPAM since this results in high levels of load on the server, and we risk potentially mis-identifying valid e-mail messages as SPAM. Current e-mail clients including Outlook 2003 and higher do an excellent job of filtering SPAM with their own PC resident algorithms.
Although the SPAM configuration details are managed entirely by NeatWorx, the relevant information is provided here for anyone wishing to use SPAM Assassin in a similar manner on their own servers.
On Fedora core (on which the NeatWorx servers are based), individual mailboxes are hosted under:
/var/qmail/mailname/<domainname>/<username>
The .qmail configuration file provides an opportunity to introduce a mail filter before depositing incoming mail into a mailbox. For customers hosting their e-mail with NeatWorx, the POP or IMAP mail client will retrieve e-mail from this mailbox via the POP or IMAP mail server.
The normal .qmail configuration while will be of the form:
| true
./Maildir/
We replace this configuration file with a configuration that will pipe all incoming mail through the spamassassin mail fliter:
| /usr/bin/spamassassin –P | /usr/local/bin/maildir ./Maildir/
The -P switch on the spamassassin command is actually deprecated - we leave it in only because it still works with later versions of spamassassin and is required for earlier versions. With recent versions the -P can be removed since operated in "pipe mode" is the default.
“maildir” is not a part of the Fedora core linux distribution – rather it is distributed as a part of “safecat”. The safecat source code has been included in /var/www/vhosts/addons. Safecat provides a safe mechanism to write mail messages to the qmail Maildir directory associated with individual mail users.
The actual spamassassin configuration rules are beyond the scope of this document. Note that the e-mail subject re-write rule and threshold for SPAM detection is provided in the file:
/etc/mail/spamassassin/local.cf
The contents are of the form:
report_safe 0
rewrite_header subject [*NWX:SPAM*]
required_score 7.00
By setting the required score lower, SPAM filter is made more sensitive and we are more likely to misidentify valid e-mail as SPAM. We have intentionally set this threshold conservatively.
For the people (like me) who just need to know how everything works, a detailed list of how SPAM messages are scored can be found at:
http://spamassassin.apache.org/tests_3_2_x.html
after making any changes to the SPAM Assassin configuration, the qmail MTA and Spam Assassin software daemons should be restarted as follows:
# /etc/init.d/spamassassin restart
# /etc/init.d/qmail restart
Configuring Q-Mail to Discard SPAM
There are a few solutions to do this, but this document explains the "from scratch" solution that does rely on the commercial PLESK spamassassin implementation.
We have elected to provide "per mailbox" filtering since this gives of more flexibility than doing server-wide filtering. It is also a little more tedious to administer however. The authors of spamassassin recommend against discarding messages classified as spam since no filter is 100% accurate, however we've found spamassassin to be very good and thus wanted to provide that option.
Our solution for SPAM filtering relies on a few software components - a korn shell script called "ifspamh". It relies on a package called mess822 - the procedures for obtaining and installing mess822 is provided below (with my gratitude to Mohammed!)
http://mohammednv.wordpress.com/?s=mess822
Sure enough in building the binaries for mess822, we relied on the same issue related to errno.h cited on the web blog above. We were able to resolve it easily using the provided solutions. (turns out this is an FAQ related to the "ifspamh" script.
For our purposes, we modified the "ifspamh" korn shell script installed in /usr/bin so a complete copy of the script is re-produced below:
#!/bin/ksh
# A wrapper for SpamAssassin suitable for calling from a dot-qmail file
# Version: 1.5
#
# Usage:
# |ifspamh mail-address
#
# Mail will be reinjected to the given mail-address, with annotatations
# if SpamAssassin thinks that it is a likely spam message.
#
# For example, in your .qmail file:
# |ifspamh <user>-isspam
# ./Mailbox
# (or however you want to deliver the mail if it isn't a spam message)
#
# This will forward anything that SpamAssassin deems spam to the
# given address.
#
# in your .qmail-isspam file:
# ./Mail/isspam
# (or wherever you want to save the Spam mail)
#
# For more details on SpamAssassin, how to set preferences, whitelists,
# etc, see 'Mail::SpamAssassin::Conf(3)' - and put the preferences
# in ~/.spamassassin/user_prefs
#
# For more detail on .qmail files, see dot-qmail(5) and qmail-command(8)
#
# Author: James R Grinter
# jrg@watching.org 24/03/2002
# Last Update: 28/02/2003
# URL: http://www.gbnet.net/~jrg/qmail/ifspamh/
#
# Requires: spamc, spamd, qmail and 822field (from DJB's mess822 package)
# to be installed and working.
#
# Also requires your /bin/sh to have a "printf" available
# (most do, if yours doesn't then consider fetching ksh-93
# from http://www.research.att.com/sw/download/, or using bash)
#
# N.B.
# If you are using vpopmail, make sure you are using at least
# version 5.3.6.
#
# If you want to run SpamAssassin globally for every email, see the
# qmail-spamc/qmail-scanner approach in the qmail/ subdirectory of the
# SpamAssassin distribution
#
# spamc - client location
SPAMC=/usr/bin/spamc
#SPAMC=/usr/bin/spamassassin
# qmail's forward program location
FORWARDBIN=/var/qmail/bin/forward
# mess822 822field location
M822FIELD=/usr/local/bin/822field
#################################################################
# nothing beyond here should require adjustment
#################################################################
FORWARD="$1"
if [ -z "$FORWARD" ]; then
echo "Usage: ifspamh [address]"
exit 111
fi
# we used to skip 'From ' - the 1st line that got added by spamd/spamc
# but it seems newer versions don't add it (and SA 2.50 adds a 3 line
# Received header)
output="`$SPAMC | sed '1{/^From .*/d;}'`"
exitcode=$?
# spamc will not process a "large email"
msize=`echo "$output" | wc -c`
# there's also a sizelimit with some shells that triggers around the 512kB mark
# with an external printf and arguments size so we stop at this point
if [ $msize -gt 250000 ]; then
# probably deemed too large anyway .. let it through
exit 0
fi
flagvalue=`printf "%s\n" "$output" | $M822FIELD X-Spam-Flag | sed 's/^ //'`
# X-Spam-Flag might contain "YES"
if [ "$flagvalue" = "YES" ]; then
# match - likely spam
# no match - not a spam
# forward on the email
printf "%s\n" "$output" | $FORWARDBIN $FORWARD
if [ $? -eq 0 ]; then
# so qmail will not do any further deliveries in .qmail file
exit 99
fi
# problem calling inject - temp failure
exit 111
else
# spamc doesn't distinguish temporary failure
# if we're passing through the message
# so look for signs that spamd provided the output..
flag2value=`printf "%s\n" "$output" | $M822FIELD X-Spam-Status`
exitcode2=$?
#if [ $exitcode2 -ne 0 ]; then
# X-Spam-Status header not present in message -> failure of spamc/spamd?
# spamc will not process a "large email", but this is accounted for above
# so indicate temporary failure
# echo "spamc returned temporary failure"
# exit 111
#fi
# indicate that qmail should continue processing dot-qmail file
printf "%s\n" "$output"
exit 0
fi
The lines shown in a "maroon" color are the big changes. We adapted the script so that the filtered e-mail is passed to stdout so that we can pipe it to separate shell script programs. Also, we've suppressed the error detection when we to catch a non-zero exit code when looking for X-Spam-Status.
Some example .qmail files (under /var/qmail/mailnames) for various scenarios are shown below:
Discarding SPAM from a native mailbox based on spamassassin's determination of SPAM
Labeling SPAM from a native mailbox based on spamassassin's determination of SPAM
"Passing SPAM through" to a distribution list with no filtering
Discarding SPAM based on a forward rule
Just to prove that sometimes it really is worth paying for commercial software, on the NeatClubs servers licensed to configure SPAMASSASSIN with PLESK, life is definitely much easier. PLESK users (on licensed servers) will see the Spam Filter icon below when they access their mail setup.
Selecting the filter allows properties to be configured on a per mailbox basis. Note that we can re-label SPAM in the subject line and we can automatically delete SPAM if we choose to do so.
Changing the web interface above makes changes to the corresponding .qmail configuration file on the server as shown below:
