AITS System Administration Group

ENCS mail delivery with procmail

Procmail is a very powerful utility, which can be configured to do almost anything you'd like with your incoming mail. We will touch upon some very basic features here. For more information see the man pages procmail(1), procmailrc(5), and for some examples, procmailex(5). Also, see the Procmail FAQ.

ALWAYS test your mail delivery after making ANY CHANGES to your .procmailrc file. Any mistakes could lead to all future incoming mail being lost until the mistake is found. ENCS IT staff will support only basic Procmail configurations.

Contents

Basic set-up

The actions of Procmail are controlled by the file .procmailrc on your Unix home directory (your U: drive if using Windows). The first thing you need to do is to create this file using our template; this doesn't change your mail processing. The next step will be to edit this file to enable features that do affect your mail processing.

Please note: the basic set-up is for folks who do not already have a .forward or .procmailrc file. If you do have such files already on your account, please remove them before following this procedure.

If you are one of the handful of people who use mh to read their mail, you'll need to modify your .procmailrc file specially; please read procmail for mh users.

The easy way (Unix)

Invoke the following command; it will create a new .procmailrc file for you based on our template:


  /encs/bin/set-up-vacation off
  

The easy way (Windows)

Download our template, and move it to your U: drive under the name .procmailrc (the leading dot is important).

To do this:

  • Right click on the "our template" link above. Select "Save Link As..." (assuming Mozilla/Forefox). (If you're using the unsupported Explorer browser, select "Save Target As...".)
  • Under "Save as type", select "All Files".
  • Rename the file name as .procmailrc (remember that leading dot!) and save it under the U: drive.

Another way (Unix)

If you prefer to copy or adapt our template explicitly, you will find it on most Unix systems in /encs/Share/mail/procmailrc.

Configuring your mail delivery

Basic concepts

To configure your mail delivery, in most cases, you must edit the .procmailrc file that you have just installed. Make sure that you edit it without reformatting the lines, or switching to Windows line endings. Under Unix, you can use pico -w. Under Windows, use an editor that supports "Unix Format" for files; we recommend Crimson Editor, which is available on ENCS-managed Windows machines. Another possibility is EditPad Lite.

In .procmailrc, the character "#" is used to indicate a comment, and we use it to disable a line. When we ask you to enable a line by "uncommenting" it, it means that you must remove the "#" at the beginning of that line. For example, this spam-fighting code is disabled:


  # Uncomment this to refile probable spam to your spam folder.
  # SpamAssassin scores: at least 8
  #
  # -=-=-=- START de-spamming-8 -=-=-=-
  #:0:
  #* ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\*
  #$SPAM
  # -=-=-=- END   de-spamming-8 -=-=-=-
  

... while in this example, is has been "uncommented", and thus enabled:


  # Uncomment this to refile probable spam to your spam folder.
  # SpamAssassin scores: at least 8
  #
  # -=-=-=- START de-spamming-8 -=-=-=-
  :0:
  * ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\*
  $SPAM
  # -=-=-=- END   de-spamming-8 -=-=-=-
  

Note that only the lines between the "marker lines" ("-=-=-=-") were uncommented. Got that? Good, let's proceed.

Spam filtering

The mail system tries to determine the likelihood of a message being spam using a complex set of rules. Some messages are blocked outright. All remaining messages are delivered to you, but a 'spam score' is calculated for the message, and if the score is higher than 5, a header is added to the message to show the score. You can choose to filter your mail so that any message with a score higher than a value you choose is placed into a different mailbox, or discarded (though we don't recommend discarding messages, since a legitimate message occasionally receives a high spam score).

To filter spam with a score of eight or higher into a separate mailbox for spam, uncomment the "de-spamming-8" section of your .procmailrc file, like so:


  # -=-=-=- START de-spamming-8 -=-=-=-
  :0:
  * ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\*
  $SPAM
  # -=-=-=- END   de-spamming-8 -=-=-=-
  

Make sure you uncomment all three lines.

The number of "\*" in the above recipe determines the minimum score that a message must have for you to consider it to be spam. In general, values between 5 and 10 will be most useful. If you want a value higher or lower than 8, just add or remove "\*" until you have the desired number.

What this does

When the recipe above is enabled, messages that have been assigned a spam score of 8 or higher are not delivered to your regular mailbox; instead, they are placed in a folder called "spam" (but see below to change the name). Please remember to empty out your spam folder every week or so, after checking it to make sure that no legitimate messages were accidentally caught.

Your spam folder

This line near the top of your .procmailrc file tells Procmail where to put spam when the recipe above in enabled:


  SPAM=spam
  

If you want to call your spam folder something other than "spam", go right ahead and edit it in your .procmailrc file, for example like this:


  SPAM=eight-asterisks
  

Avoiding the 87 MB quota limit (deliver to home directory)

By default, Procmail delivers your mail into your "INBOX", which is a special folder that lives outside your home directory diskspace. Your mail folder has a quota of 87MB, which cannot be increased. If you find that this is too small (for example, because your mailbox fills up when you take vacation), you can arrange to have Procmail deliver your mail directly into your home directory, where you have a bigger disk quota.

You should do that only if you really need to; if someone sends you lots of huge attachments, for example, it's usually better to run out of INBOX quota and bounce mail, as opposed to risking overrunning your home directory quota.

If you still want to do this, here's how: just uncomment the "homedir-delivery" section:


  # -=-=-=- START homedir-delivery -=-=-=-
  DEFAULT=Incoming_mail
  # -=-=-=- END   homedir-delivery -=-=-=-
  

What this does

This tells Procmail to deliver mail (other than any mail that you explicitly file otherwise, such as spam) to a folder called Incoming_mail (in your IMAP directory mail).

Note that you must configure your mail client to explicitly subscribe to the folder Incoming_mail in order to pick up your new mail there.

You can change the Incoming_mail folder to some other name if you like. Just remember to subscribe to the right folder in your e-mail client software.

If you want to reverse this, and go back to receiving your mail in the system INBOX area, just comment out the DEFAULT entry (by putting a "#" in front of it).

Mail delivery logging

If you need to see a log trace of your messages, uncomment one or both lines in the "delivery-log" section, like so:


  # -=-=-=- START delivery-log -=-=-=-
  LOGFILE=$HOME/delivery.log
  #VERBOSE=on
  # -=-=-=- END   delivery-log -=-=-=-
  

What this does

If you turn on logging, a log file called delivery.log is created in in your Unix home directory (U: drive), and it contains a three-line summary for every message received, indicating when and from whom it was received, the subject, and which folder it was placed into. If there were errors during the delivery, they appear here as well.

The VERBOSE option is rarely needed. If you leave logging on, then please arrange to rotate (clean out) your delivery.log file regularly.

"Vacation" auto-responses

Once you are using our .procmailrc template (see basic set-up), it is easy to enable or disable vacation mail processing. To enable it, simply create a file called vacation-yes on your Unix home directory (U: drive). Important note for Windows users: that's a filename of vacation-yes with no filename extension. To make sure you get this right, you might want to disable the hiding of extensions for known file types.

Once you have created U:vacation-yes, then the next message that arrives on your account will automatically trigger the creation of the required "vacation database" and the "vacation message", if they do not already exist. If you need that to happen right away (for example because you want to edit the outgoing message), just send yourself an e-mail message!

Alternatively, if you are using Unix, invoke this command to set up everything you need:

/encs/bin/set-up-vacation on

What this does

When vacation processing is enabled, the mail system will send a reply to each e-mail message received on your account from that point on. However, you should be aware that responses are sent only once per week per address, and also, that responses are sent only to messages which "look as though" they come from human and are addressed specifically to you. In particular, various tests are used to avoid sending auto-responses to mailing lists and mail bounces, for example.

Disabling vacation autoresponses

When you return from your time off, just remove the file vacation-yes; this will turn off the automated responses. You don't need to remove the vacation message or database; they will be useful next time you are away.

Alternatively, if you are using Unix, you can invoke this command to remove the file:

/encs/bin/set-up-vacation off

The vacation message

The outgoing message placed on your account will be a very general one, whereas you may prefer to specify when you will return and whom to contact in your absence, and you may want to add your signature to the message. To change the outgoing message, simply edit the file .vacation.msg (note the leading dot in the filename) which you will find on your Unix home directory (U: drive). If that file is not already there after you have enabled vacation mail processing, send yourself a message to trigger its creation.

Important note for Windows users: the filename .vacation.msg must have no additonal filename extension. To make sure you get this right, you might want to disable the hiding of extensions for known file types.

Another important note for Windows users: the file .vacation.msg must continue to have "Unix line endings", which might be incorrectly converted to DOS line endings if you edit the file with the wrong text editor. We recommend that you use the Crimson editor, which is free, and already available on AITS-installed Windows desktops. If you're working from your own machine, you can get this program from the Crimson web site.

The vacation database

The vacation database, a file called .vacation.db, stores the list of addresses that have been sent a reply message. If you want to find out who was sent a response and when, you'll need to log into a Linux machine (such as login.encs), and issue this command:

vacation -l

(That's a lowercase letter "L", for "list".)

Other ways to do this

You may know that you can pipe e-mail to the "vacation" program using a .forward file; this works, but if you do it, your Procmail processing will not take effect; .forward overrides any other mail processing.

Alternatively, you can create a recipe to pipe a copy of your mail to the vacation program directly, without using our template and its special vacation-related INCLUDERC lines; in that case, the vacation-yes file would not apply. Here is a sample recipe if your prefer not to use our template:


  :0c
  | /local/bin/vacation $LOGNAME
  

Don't lose the |; it tells Procmail to hand the message to a program.

Spam from specific addresses

Occasionally spam gets past the system-wide filters. If you'd like mail from a specific source to be filed as spam, you can arrange for this to happen. For example:


  :0:
  * ^To:.*annoyingperson@some.where
  $SPAM
  

To create your own spam recipe, take the three lines above, insert them into your .procmailrc file at the end, and edit the second line to replace annoyingperson@some.where with the address whose mail you want to consider as spam.

"Friend" and mailing list exceptions

Occasionally you will find that mail from certain addresses tends to get a high spam score; this happens to some mailing lists, for example. You can "rescue" those messages from being filed as spam by catching them before the spam checks, and explicitly filing them into your inbox. Here's an example:


  :0:
  * ^To:.*reallyfastcars@mailing.list.server
  $DEFAULT
  

To create your own exception recipe, take the three lines above, insert them into your .procmailrc file where it says "If you need to make exceptions to the de-spamming, do it here", and edit the second line to replace reallyfastcars@mailing.list.server with the address whose mail you want to "rescue".

What this does

Putting a special "filing" recipe like the one shown above into your .procmailrc file before the despamming code will make sure that mail from the address you specify doesn't get mistakenly filed as spam.

Other delivery recipes (refiling)

Many people like to keep certain messages separate from their regular mail, either to make sure they don't miss it (for example, mail from the boss), or to allow them to read it later (for example, mail from busy mailing lists).

Procmail is able to filter mail based on any aspect of the mail message. The most common filters use the From:, To:, or Subject: headers. For instance, if you want all mail from your friend johndoe@ece.concordia.ca to be saved in a mailbox called john, you could use a Procmail recipe like this:


  :0:
  * ^From:.*johndoe@ece.concordia.ca
  john
  

The best place to insert such a recipe is usually after all de-spamming and vacation auto-replying have taken place; the spot is indicated in our template with "If you want any special refiling, do it here".

What this does

The example recipe above says that if a line in the mail headers starts with (^) the word From:, followed by johndoe@ece.concordia.ca, with anything else in between (.*), then the mail should be put in the folder john. That folder is in your IMAP directory ($HOME/mail), assuming that you used our template, which sets the MAILDIR variable.

The reason for the .* ("anything else in between") is to make certain that you get a proper match, even if the real From: header looks like this:


  From: "John Q. Doe" <johndoe@ece.concordia.ca>
  

More examples

Sorting mail from a mailing list is almost the same. Mailing list messages are normally sent 'To:' the list name. So if you're subscribed to reallyfastcars@mailing.list.server, you could sort mail from that list into the fastcars folder using a Procmail recipe like this:


  :0:
  * ^To:.*reallyfastcars@mailing.list.server
  fastcars
  

You can always check your old e-mail messages to find a good header to match on. If you have turned on delivery logging), you can check the Procmail log file (delivery.log in our example) after you make any changes to your .procmailrc file, in order to make certain you mail is being filtered the way you expect. Otherwise, you can determine it based on whether or not future messages land in the correct folder.

You can put in as many of these recipes as you like, to filter your incoming email by matching messages to certain conditions, and placing them in the appropriate mailbox.

Forwarding your mail

You may already know that you can forward all your mail using a .forward file, but that's not compatible with using Procmail. You can use Procmail to forward some of your mail, for example, only the non-spam, or only messages from a particular person - what gets forwarded depends on where in your .procmailrc file you place the forwarding recipe.

Here's how to forward your mail; there will be no copy kept in your ENCS mailbox for any message forwarded this way:


  :0
  ! self@other.addr.ess
  

Of course you must replace self@other.addr.ess with the address you want to forward to. Don't lose the "!"; this means "forward it" to Procmail.

Here's how to forward just a copy of your mail; this way, another copy of the message is delivered to your ENCS mailbox. The only difference is the presence of the c flag on the first line, which means "do this to a copy of the message":


  :0c
  ! self@other.addr.ess
  

Finally, here's an example showing only mail for the "reallyfastcars" mailing list being forwarded:


  :0
  * ^To:.*reallyfastcars@mailing.list.server
  ! self@other.addr.ess
  

A crash course on recipe creation

A Procmail recipe is a block of code giving Procmail an instruction of the form "if the message matches this pattern, do this with it". Procmail recipes are used in the order in which they appear; once a message has been "delivered" (placed into a mail folder) by a recipe, then processing stops.

A recipe looks like this:

Form Meaning Example What the example means
:0 <flags> Recipe begins. :0: Start recipe, and use a lock (:) when delivering
* <pattern> If the message matches this pattern... * From:.*profx@encs.concordia.ca If the message is from Professor X...
<action> ... then do this thing with it. important ... then place it into a folder called "important".

For more details about exactly what all those lines mean, see the man page for procmailrc(5), and remember to consult the Procmail FAQ.

Author: Anne Bennett
Credits: Chris O'Regan, Daniel Morrison
Last update: Anne Bennett -- 2014-10-20
Valid CSS Valid XHTML 1.0 Transitional