Your sme server receives a lot of email with virus infected attachments and you want to reject it before it enters your server's mail system.

You want to block email with certain types of file attachments to improve security of your server or reduce bandwidth use caused by unwanted or undesired large multimedia files.

Current methods typically use Anti Virus detection software. These methods are processor & memory intensive and can annoy recipients with excessive virus detection notification messages.

There is presently no system for blocking multimedia or other file attachments

This contribution for sme server 6.0 or 6.0.1, adds a feature to mailfront/smtpfront-qmail, which allows incoming & outgoing messages to be rejected if the attached file has executable content, which matches specific file type patterns.

A default pattern matching database is created with common executable file patterns, which cover the majority of currently known Windows type executable viruses.

Patterns can be created for any file types to allow multimedia or other attachments to be rejected where the system management policy considers it appropriate.

Note that this contribution is NOT compatible with sme server 6.0beta, 5.6, or earlier releases.

For use on sme server 6.5 please see specific installation section below.

Email messages are rejected if the attachment content matches an entry in the patterns database. By default this includes the majority of *.exe files, older v1.0 *.zip files and some *.gif files.

This blocking applies to both incoming and outgoing smtp email messages, including the local network, in order to stop virus propagation.

If these file types need to be sent using email, they should be compressed using WinZip (v2.0 format) or WinRAR or other suitable compression software, or alternatively shared on the local network use filesharing. Note that recent releases of compression software use the v2.0 zip format.

Messages with attachments that match the patterns database are rejected by the mail system, and as a result there is no further processing. In practice a large number of virus infected messages will be rejected, perhaps 95 % or more, depending on the type of virus infections you receive and your system exposure (email addresses).

In conjunction with RBL list blocking of spam messages (see separate HOWTO) my server has experienced a reduction in virus detections by Clamavis-ng from hundreds per week to one message in three weeks. The use of RBL list spam blocking also helps reduce virus infected email messages entering the server, probably due to the fact that some virus infected messages come from similar sources as spam messages.

This method works for servers configured as either Server & Gateway or Server Only as long as the mail server components are enabled (smtpfront-qmail & qmail) and the server has access to the Internet via another sme server or firewall.

WARNING: Installation of these rpms will make major changes to your server's mail system. Please follow all instructions carefully and ensure that an anti-relay test is carried out after implementation to ensure your server is not inadvertantly misconfigured as an open mail relay. This is only a precaution, but a necessary one.

Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. Once these new viruses have been analysed, additional patterns can be created and added to the patterns database as required. It is envisaged that new patterns would be added infrequently.

See separate section below for information on analysing, determining & adding file patterns.

This pattern matching feature should be used in conjunction with virus scanning software and spam filtering software, although these programs will have a lot less work to do.

It has been tested on a sme 6.0 server with Clamavis-ng and Spamassassin installed and works very effectively. It also works OK with Jesper Knudsens Antivirus & Spam Filter panels installed.

Pattern blocking should be compatible with other brands of virus & spam software based programs. They generally scan or filter the message after it has been accepted by the servers mail system.

Pattern blocking occurs before the message is accepted, and if a matching occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely.

An additional feature I recommend to implement is "RBL List blocking" using smtpfront-qmail, to reject spam messages from senders that are included on RBL lists. This technique will dramatically reduce the amount of spam entering the server. See separate HOWTO for details.

The RBL feature has been incorporated into Jesper Knudsens Spam Filter panel.

This HOWTO is based on devinfo posts, emails, and rpms developed by Gordon Rowell, in conjunction with my own investigations & testing. Thanks also to Gordon Rowell & Charlie Brady for their assistance & suggestions and the work they did to integrate this feature into sme 6.0 and to Russell Nelson for the initial work done on qmail pattern matching and Bruce Guenter who added the feature to mailfront (& also wrote mailfront). Additional information re v6.5 from forum post by Shad Lords.

Please note these rpms have been incorporated into the contribs.org release of sme server v6.5. You DO NOT need to carry out the steps outlined in this HOWTO, simply enable Pattern matching using the following command.

/sbin/e-smith/config setprop smtpfront-qmail Patterns enabled

If you wish to enable pattern matching on the secure email interface do the following command.

/sbin/e-smith/config setprop ssmtpfront-qmail Patterns enabled

Then configure the file types to be blocked in the Server Manager Email panel and Save.

You will first need to decide which patterns you wish block. By default the majority of Windows executable, zip (v1.0) & some gif files will be blocked. The default is generally suitable for most users. Note that zip (v2.0 format) will NOT be blocked by default, but can be enabled with a simple command.

Additional patterns can be added to the database after install is completed.

Also see separate section below for information on analysing, creating & adding patterns.

WARNING: Enable additional patterns with care. Verify that the patterns do not block attachment types that you wish to receive.

Check and note the versions of rpms you currently have installed. This information will be required if you wish to undo the installation later. A standard install of sme 6.0 or 6.0.1 may not have all these rpms installed.

Do

rpm -q mailfront e-smith-email e-smith-mailfront

Output for sme 6.0 Output for sme 6.0.1

mailfront-0.81-1.i386.rpm mailfront-0.81-1.i386.rpm

e-smith-email-4.14.0-06.noarch.rpm e-smith-email-4.14.0-07.noarch.rpm

e-smith-mailfront-1.3.0-11.noarch.rpm e-smith-mailfront-1.3.0-11.noarch.rpm

Download to an empty folder the following rpms from ibiblio or other mirror sites. Please note that later versions may have been released, which will replace the files mentioned in this section. Dependencies require that perl-perl-ldap, perl-Net-Server, perl-libnet & sortspam rpms also be installed.

mailfront-0.91-3es.i386.rpm

wget ftp://ftp.ibiblio.org/pub/linux/distributions/e-smith/contrib/GordonRowell/RPMS/noarch/e-smith-email-4.15.0-07gr07.noarch.rpm

wget ftp://ftp.ibiblio.org/pub/linux/distributions/e-smith/contrib/GordonRowell/RPMS/noarch/e-smith-mailfront-1.5.0-13gr06.noarch.rpm

1.i386.rpm

1.i386.rpm

1.i386.rpm

wget ftp://ftp.ibiblio.org/pub/linux/distributions/e-smith/devel/RPMS/i386/sortspam-1.1.0-

05.i386.rpm

/etc/init.d/smtpfront-qmail stop

cp /etc/tcprules/tcp.smtp /tmp

rpm -Uvh *.rpm

/sbin/e-smith/db mailpatterns delete ZIPV1 (This command is only necessary if you previously tested an earlier version of the contrib as this pattern has changed).

/sbin/e-smith/db mailpatterns delete PATTERNTYPE (This command is only necessary if you previously tested or created other pattern types that are in the default database, where PATTERNTYPE is the pattern name you created).

/etc/e-smith/events/actions/initialize-default-databases

/sbin/e-smith/expand-template /etc/tcprules/tcp.smtp

diff /etc/tcprules/tcp.smtp /tmp/tcp.smtp

WARNING: If there are any differences except blank lines, undo the installation by reverting to the previously installed versions of the rpms.

Here is an example of diff output showing "blank line" changes, this can be safely ignored:

2,4d1
<
<
<
7d3
<

During testing, a custom template created by the installation of the smeserver-spamassassin-1.0.0-01 rpm, conflicted with the revised tcprules/tcp.smtp template and caused some differences to be observed. As the function of the custom template was being duplicated, it was safe to remove the custom template and proceed with the installation. After again running the diff command above, no differences other than blank lines were seen.

If you have any problems review any custom templates in your system and remove them where necessary.

See section below for further info on the tcp.smtp file contents

/sbin/e-smith/db configuration setprop smtpfront-qmail Patterns enabled

/sbin/e-smith/expand-template /etc/tcprules/tcp.smtp

diff /etc/tcprules/tcp.smtp /tmp/tcp.smtp

Note: Each line should now have a PATTERNS entry, but no other changes when compared to the previous diff output. See section below for further info on the tcp.smtp file contents.

/sbin/e-smith/signal-event email-update

/etc/init.d/smtpfront-qmail start

Furthermore to ensure that all .qmail files are updated after package changes do the following

/sbin/e-smith/signal-event post-upgrade

/sbin/e-smith/signal-event reboot

A menu box is added to the server manager Email panel, which allows executable content blocking to be enabled or disabled. It is enabled by default. Use "Ctrl click" to highlight or unhighlight the various groups of file types, and then click the Save button to enable/disable pattern matching.

There are two ways this can be done

At the server's command prompt do:

telnet relay-test.mail-abuse.org

Note: This only works if the outgoing address is also your mailserver address, which is usually the case in standard installations. This may not be true in some more complex network setups.

Alternatively browse to

www.abuse.net/relay.html

and enter your server details and perform the test

Depending on the configuration of your server the contents of this file may vary from the examples shown below.

The /etc/tcprules/tcp.smtp file without Clamavis-ng installed and before Pattern Matching has been installed looks like this

(Note local IP = 192.XXX.X.X)

(Note gateway IP = xxx.xxx.xxx.xxx)

127.0.0.1:allow,RELAYCLIENT=""

192.XXX.X.X:allow,RELAYCLIENT=""

xxx.xxx.xxx.xxx:allow

192.XXX.X.:allow,RELAYCLIENT=""

:allow,MAILRULES="/var/qmail/control/mailrules.default"

The /etc/tcprules/tcp.smtp file with Clamavis-ng installed and before Pattern Matching has been installed looks like this

(Note local IP = 192.XXX.X.X)

(Note gateway IP = xxx.xxx.xxx.xxx)

127.0.0.1:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

192.XXX.X.X:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

xxx.xxx.xxx.xxx:allow,MAILRULES="/var/qmail/control/mailrules.default",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

XXX.XXX.X.:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

XXX.XXX.X.:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

:allow,MAILRULES="/var/qmail/control/mailrules.default",QMAILQUEUE="/usr/bin/qmail-queue.amavis"

My server's /etc/tcprules/tcp.smtp file with Clamavis-ng installed and after Pattern Matching has been installed & enabled looks like this

(Note local IP = 192.XXX.X.X)

(Note gateway IP = xxx.xxx.xxx.xxx)

127.0.0.1:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

192.XXX.X.X:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

xxx.xxx.xxx.xxx:allow,MAILRULES="/var/qmail/control/mailrules.default",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

192.XXX.X.:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

192.XXX.X.:allow,RELAYCLIENT="",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

:allow,MAILRULES="/var/qmail/control/mailrules.default",QMAILQUEUE="/usr/bin/qmail-queue.amavis",PATTERNS="/var/qmail/control/patterns.default"

The standard patterns enabled by default are:

Extra patterns not included in default database that may be enabled if required for blocking of new viruses

A recent pattern identified for the virus

Worm.SomeFool.P

is

TVoAAD8AA

Identified as MS-DOS executable

Extra patterns not included in default database that may be enabled if required for blocking of multimedia files etc (long & short versions listed)

Note that these have not been tested and may need further refinement to ensure they accurately represent the signature pattern for all occurrences of the particular file type

***** Feedback is welcomed on the correctness of these patterns *****

SCR screen saver files - MS-DOS executable (EXE)

PIF1 - data

PRlRcSFQ5OFxIVDk4LkVY

PIF3 - data

RcSFJcSFIwMC5FWEUA

wLgAAAAAAAAAAAAAAAAAA

QAUAAAIBAwADAAAAzgAA

DEAwAAIRgBAAAAAAAA

ADADCBTZWgAAAAAAAAAeeIB

MPG mpeg1 video file - MPEG system stream data

M2P mpeg2 video file - MPEG system stream data

AVI video file - RIFF (little-endian) data

To find out what the pattern or signature or magic for a file is, we need to run it through a base64 encoding routine and look for the appropriate strings in the first line of the output. That is, for "sane" files which have "magic" numbers at the start. We also can decode the file to find out what type of file it is. Published file specifications (where available) could also be referred to.

Copy a file to a folder on your sme server, say filename.zip

At the command prompt do

perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' <filename.zip | head -1

we get an output of

UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV

We need to pick a suitable substring to use as the pattern for this file type, for example:

UEsDBAoAA

We want the pattern string to be long enough to avoid "false positives" and short enough to catch all of that file type. Running the above command across a few files of a particular type will usually clearly show the appropriate substring.

To find out the file type details do

then run "file" on the result

file /tmp/17.exe

the output is

/tmp/17.exe: Zip archive data, at least v1.0 to extract

which identifies the type of file

An alternative way of identifying the file pattern or signature for users of Clamavis-ng is to view the quarantined messages in /var/spool/amavis-ng/quarantine

Let's say we want to add a pattern to the existing EXEFILES type (which you should do if you discover new patterns for common new MSDOS Executable type viruses)

A pattern analysed from email messages received is

TVoAAD8AA

As we wish to add this to the existing db entry we would do:

/sbin/e-smith/db mailpatterns set EXEFILES pattern Body TVqQAAMAA,TVpQAAIAA,

TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,

TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA Description "MS-DOS executables"

Glob yes LineStart yes Status enabled

(the above is all on one line, watch the spaces are correct)

/sbin/e-smith/signal-event email-update

To check the entry is correct do:

/sbin/e-smith/db mailpatterns show EXEFILES

which gives an output of

EXEFILES=pattern

Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA

Description=MS-DOS executables

Glob=yes

LineStart=yes

Status=enabled

Let's say we want to enable the pattern for PIF2 type files (which you should do if you wish to block some PIF attachments)

A pattern being tested for this file type is

AMlIbDk5Lm

So we do:

(the above is all on one line)

/sbin/e-smith/signal-event email-update

To check the entry is correct do:

/sbin/e-smith/db mailpatterns show PIF2

which gives an output of

PIF2=pattern

Description=PIF2 data

Glob=yes

LineStart=yes

Status=enabled

The fields are as follows:

pattern – the type of the entry in the database (currently only the "pattern" type is used)

Body – the substring to match

Description – free format text to describe this pattern. This text will be used to display a menu

of patterns to enable/disable in a later version

Glob – whether to apply a wildcard match after the pattern

LineStart – whether to only match this pattern at the start of the line

Status – whether this pattern is currently enabled (i.e. blocked)

To disable the pattern do:

/sbin/e-smith/db mailpatterns setprop PIF2 Status disabled

/sbin/e-smith/signal-event email-update

The alternative but more correct approach is as follows:

The initialize-default-databases loads the db with fragments from /etc/e-smith/db. When new patterns are added to the master rpm, new fragments are also added.

To add a pattern to the default set in the rpm, we do:

then do

/etc/e-smith/events/actions/initialize-default-databases

which will load the default settings

To show all the patterns in the mailpatterns database & their status (enabled or disabled) do

/sbin/e-smith/db mailpatterns show

which will give an output similar to

(Note the last entry for ZIPV2 is disabled)

/sbin/e-smith/db mailpatterns show

EXEFILES=pattern

Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA

Description=MS-DOS executables

Glob=yes

LineStart=yes

Status=enabled

GIF01=pattern

Body=R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy

Description=GIF file from old virus

Glob=yes

LineStart=yes

Status=enabled

ZIPV1=pattern

Body=UEsDBAoAA

Description=Zip archive data, at least v1.0 to extract

Glob=yes

LineStart=yes

Status=enabled

ZIPV2=pattern

Body=UEsDBBQAA

Description=Zip archive data, at least v2.0 to extract

Glob=yes

LineStart=yes

Status=disabled

By reviewing /var/log/smtpfront-qmail/current and var/log/smtpfront-qmail/* you can see the entries for rejected messages and generally enough information as to why the rejection occurred, and therefore see the effectiveness of Pattern Matching blocking.

Note that you will only see these types of entries after blocking has been enabled and messages have been rejected.

If you do not see all of the types of entries shown below, it would either be due to not having the particular Pattern enabled or not receiving attachments with that type of content.

You can view date formatted logs using the Server Manager View log files panel

To see ALL the log entries do

grep "" /var/log/smtpfront-qmail/current | tai64nlocal

To see only the rejected message entries and the reason for rejection do

grep "We don't accept email with executable content" /var/log/smtpfront-qmail/current | tai64nlocal

(the above is all on one line)

Here is an example of some typical entries

Note: you will only see these entries after some messages have been rejected

2004-04-15 12:32:11.892522500 smtpfront-qmail[23392]: 554 We don't accept email with executable content ZIPV1 (#5.3.4)

2004-04-15 15:23:40.765202500 smtpfront-qmail[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)

2004-04-15 15:33:08.132041500 smtpfront-qmail[29241]: 554 We don't accept email with executable content EXE12 (#5.3.4)

2004-04-15 15:33:09.021650500 smtpfront-qmail[29265]: 554 We don't accept email with executable content PIF (#5.3.4)

Alternatively you could filter on the pattern type code to see how many messages with a particular type of executable content are being rejected eg

grep EXE01 /var/log/smtpfront-qmail/current | tai64local

2004-04-15 15:23:40.765202500 smtpfront-qmail[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)

2004-04-15 15:33:08.132041500 smtpfront-qmail[29241]: 554 We don't accept email with executable content EXE01 (#5.3.4)

2004-04-15 15:33:09.021650500 smtpfront-qmail[29265]: 554 We don't accept email with executable content EXE01 (#5.3.4)

2004-04-15 15:33:24.986426500 smtpfront-qmail[29274]: 554 We don't accept email with executable content EXE01 (#5.3.4)

These links may be of interest. Note that they do not specifically apply to sme server, so DO NOT implement them. They are listed for background information only.

http://qmail.planetmirror.com/top.html

http://qmail.planetmirror.com/top.html#microsoft

http://qmail.planetmirror.com/qmail-smtpd-viruscan-1.3.patch