How do I configure ENCS email delivery with procmail?
As of June 2022, the use of procmail is deprecated. It will continue to be available indefinitely, but if you can, you are encouraged to switch to SIEVE. For information about SIEVE, please see:
https://www.concordia.ca/ginacody/aits/support/faq/sieve.html
Try not to use both; for information about the interaction between procmailrc and SIEVE, please see:
https://www.concordia.ca/ginacody/aits/support/faq/procmail_and_sieve.html
Procmail is a very powerful utility, which can be configured to do almost anything you'd like with your incoming mail. We will touch upon some very basic features here. For more information see the man pages procmail(1), procmailrc(5), and for some examples, procmailex(5). Also, see the Procmail FAQ.
ALWAYS test your mail delivery after making ANY CHANGES to your .procmailrc file. Any mistakes could lead to all future incoming mail being lost until the mistake is found. GCS IT staff will support only basic procmail configurations.
Documentation
The actions of procmail are controlled by the file .procmailrc on your Unix home directory (your U: drive if using Windows). The first thing you need to do is to create this file using our template; this doesn't change your mail processing. The next step will be to edit this file to enable features that do affect your mail processing.
Before you start, make sure that you don't already have either a .forward file or a .procmailrc file.
If you are one of the handful of people who use mh to read their mail, you'll need to modify your .procmailrc file specially; please read “special considerations for MH users” below.
The easiest way to set up procmail is using the GCS Webmail interface; from the top menu, select “Forwards”, then “Unset/remove the forward”, and submit. Now you have the template .procmailrc file.
(Alternatively, if you're logged into a Unix system, just copy /encs/Share/mail/procmailrc to ~/.procmailrc.)
How to edit .procmailrc
To configure your mail delivery, in most cases, you must edit the .procmailrc file that you have just installed. Make sure that you edit it without reformatting the lines, or switching to Windows line endings. Under Unix, you can use pico -w. Under Windows, use an editor that supports “Unix Format” for files; we recommend Notepad++, which is
available on ENCS-managed Windows machines. Other possibilities are Crimson Editor and EditPad Lite.
Comments in .procmailrc
In .procmailrc, the character “#” is used to indicate a comment, and we use it to disable a line. When we ask you to enable a line by “uncommenting” it, it means that you must remove the “#” at the beginning of that line. For example, this spam-fighting code is disabled:
# Uncomment this to refile probable spam to your spam folder. # SpamAssassin scores: at least 8 # # -=-=-=- START de-spamming-8 -=-=-=- #:0: #* ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\* #$SPAM # -=-=-=- END de-spamming-8 -=-=-=-
… while in this example, is has been “uncommented”, and thus enabled:
# Uncomment this to refile probable spam to your spam folder. # SpamAssassin scores: at least 8 # # -=-=-=- START de-spamming-8 -=-=-=- :0: * ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\* $SPAM # -=-=-=- END de-spamming-8 -=-=-=-
Note that only the lines between the “marker lines” (“-=-=-=-”) were uncommented. Got that? Good, let's proceed.
A crash course on recipe creation
A procmail recipe is a block of code giving procmail an instruction of the form "if the message matches this pattern, do this with it". Procmail recipes are used in the order in which they appear; once a message has been "delivered" (placed into a mail folder) by a recipe, then processing stops.
A recipe looks like this:
| Form | Meaning | Example | What the example means |
|---|---|---|---|
| :0 <flags> | Recipe begins. | :0: | Start recipe (":0"), and use a lock (the second ":") when delivering |
| * <pattern> | If the message matches this pattern... | * From:.*\<profx@encs.concordia.ca\> | If the message is from Professor X... |
| <action> | ... then do this thing with it. | important | ... then place it into a folder called "important". |
For more details about exactly what all those lines mean, see the man page for procmailrc(5), and remember to consult the Procmail FAQ.
The mail system tries to determine the likelihood of a message being spam using a complex set of rules. Some messages are blocked outright. All remaining messages are delivered to you, but a 'spam score' is calculated for the message, and if the score is higher than 5, a header is added to the message to show the score. You can choose to filter your mail so that any message with a score higher than a value you choose is placed into a different mailbox, or discarded (though we don't recommend discarding messages, since a legitimate message occasionally receives a high spam score).
To filter spam with a score of eight or higher into a separate mailbox
for spam, uncomment the “de-spamming-8” section of your
.procmailrc file, like so:
# -=-=-=- START de-spamming-8 -=-=-=- :0: * ^X-CU-Score-Spam: by .*: spamassassin: \*\*\*\*\*\*\*\* $SPAM # -=-=-=- END de-spamming-8 -=-=-=-
Make sure you uncomment all three lines.
The number of “\*” in the above recipe determines the minimum score that a message must have for you to consider it to be spam. In general, values between 5 and 10 will be most useful. If you want a value higher or lower than 8, just add or remove “\*” until you have the desired number.
What this does
When the recipe above is enabled, messages that have been assigned a spam score of 8 or higher are not delivered to your regular mailbox; instead, they are placed in a folder called "spam" (but see below to change the name). Please remember to empty out your spam folder every week or so, after checking it to make sure that no legitimate messages were accidentally caught.
Your spam folder
This line near the top of your .procmailrc file tells
procmail where to put spam when the recipe above in enabled:
SPAM=spam
If you want to call your spam folder something other
than “spam”, go right ahead and edit it in your
.procmailrc file, for example like this:
SPAM=eight-asterisks
By default, procmail delivers your mail into your “INBOX”,
which is a special folder that lives outside your home directory
diskspace. Your mail folder has a relatively small quota, which
cannot be increased. If you find that this is too small (for
example, because your mailbox fills up when you take vacation),
you can arrange to have procmail deliver your mail directly
into your home directory, where you have a bigger disk quota.
You should do that only if you really need to; if someone sends
you lots of huge attachments, for example, it's usually better to run
out of INBOX quota and bounce mail, as opposed to risking
overrunning your home directory quota.
If you still want to do this, here's how: just uncomment the “homedir-delivery” section:
# -=-=-=- START homedir-delivery -=-=-=- DEFAULT=Incoming_mail # -=-=-=- END homedir-delivery -=-=-=-
What this does
This tells procmail to deliver mail (other than any mail that you
explicitly file otherwise, such as spam) to a folder called
Incoming_mail (in your IMAP directory mail).
Note that you must configure your mail client to explicitly
subscribe to the folder Incoming_mail in order to pick up
your new mail there.
You can change the Incoming_mail folder to some other
name if you like. Just remember to subscribe to the right folder in
your e-mail client software.
If you want to reverse this, and go back to receiving your mail in
the system INBOX area, just comment out the DEFAULT
entry (by putting a “#” in front of it).
If you need to see a log trace of your messages, uncomment one or both lines in the “delivery-log” section, like so:
# -=-=-=- START delivery-log -=-=-=- LOGFILE=$HOME/delivery.log #VERBOSE=on # -=-=-=- END delivery-log -=-=-=-
What this does
If you turn on logging, a log file called delivery.log
is created in in your Unix home directory (U: drive), and
it contains a summary for every message received, indicating
when and from whom it was received, the subject, and which folder it was
placed into. If there were errors during the delivery, they appear here
as well.
The VERBOSE option is rarely needed. If you leave logging
on, you might want to arrange to rotate (clean out) your
delivery.log file regularly.
First, the very easiest way to set up mail processing is using the GCS Webmail interface; from the top menu, select “Vacation” and fill in the required information. Alternatively, you can set up vacation processing manually, as shown here.
Once you are using our .procmailrc template, it is easy to manually enable or disable vacation mail processing. To enable it, simply create a file called vacation-yes on your Unix home directory (U: drive). Important note for Windows users: that's a filename of vacation-yes with no filename extension.
Once you have created vacation-yes, then the next message that arrives on your account will automatically trigger the creation of the required “vacation database” and the “vacation message”, if they do not already exist. If you need that to happen right away (for example because you want to edit the outgoing message), just send yourself an e-mail message!
Alternatively, if you are using Unix, invoke this command to set up everything you need:
/encs/bin/set-up-vacation on
What this does
When vacation processing is enabled, the mail system will send a reply to each e-mail message received on your account from that point on. However, you should be aware that responses are sent only once per week per address, and also, that responses are sent only to messages which “look as though” they come from human and are addressed specifically to you. In particular, various tests are used to avoid sending auto-responses to mailing lists and mail bounces, for example.
Disabling vacation autoresponses
When you return from your time off, just remove the file vacation-yes; this will turn off the automated responses. You don't need to remove the vacation message or database; they will be useful next time you are away.
Alternatively, if you are using Unix, you can invoke this command to remove the file:
/encs/bin/set-up-vacation off
The vacation message
The outgoing message placed on your account will be a very general one, whereas you may prefer to specify when you will return and whom to contact in your absence, and you may want to add your signature to the message. To change the outgoing message, simply edit the file .vacation.msg (note the leading dot in the filename) which you will find on your Unix home directory (U: drive). If that file is not already there after you have enabled vacation mail processing, send yourself a message to trigger its creation.
Important note for Windows users: the filename .vacation.msg must have no additional filename extension.
Another important note for Windows users: the file .vacation.msg must continue to have “Unix line endings”, which might be incorrectly converted to DOS line endings if you edit the file with the wrong text editor. We recommend that you use the Crimson editor, which is free, and already available on AITS-installed Windows desktops. If you're working from your own machine, you can get this program from the Crimson web site.
The vacation database
The vacation database, a file called .vacation.db, stores the list of addresses that have been sent a reply message. If you want to find out who was sent a response and when, you'll need to log into a Linux machine (such as login.encs), and issue this command:
vacation -l
(That's a lowercase letter “L”, for “list”.)
Other ways to do this
You may know that you can pipe e-mail to the “vacation” program using a .forward file; this works, but if you do it, your procmail processing will not take effect; .forward overrides any other mail processing.
Alternatively, you can create a recipe to pipe a copy of your mail to the vacation program directly, without using our template and its special vacation-related INCLUDERC lines; in that case, the vacation-yes file would not apply. Here is a sample recipe if your prefer not to use our template:
:0c | /local/bin/vacation $LOGNAME
Don't lose the |; it tells procmail to hand the message to a program.
Occasionally spam (or other unwanted mail) gets past the system-wide filters. If you'd like mail from a specific source to be filed as spam, you can arrange for this to happen. For example:
:0: * ^From:.*\<annoyingperson@some.where\> $SPAM
To create your own spam recipe, take the three lines
above, insert them into your .procmailrc
file at the end, and edit the second line to replace
annoyingperson@some.where with the address whose
mail you want to consider as spam.
Occasionally you will find that mail from certain addresses tends to get a high spam score; this happens to some mailing lists, for example. You can “rescue” those messages from being filed as spam by catching them before the spam checks, and explicitly filing them into your inbox. Here's an example:
:0: * ^To:.*\<reallyfastcars@mailing.list.server\> $DEFAULT
To create your own exception recipe, take the three lines above, insert them into your .procmailrc file where it says “If you need to make exceptions to the de-spamming, do it here”, and edit the second line to replace reallyfastcars@mailing.list.server with the address whose mail you want to “rescue”.
If you're rescuing mail from a friend, you'll want to match the “From” address instead of the “To” address, for example:
:0: * ^From:.*\<my.friend@some.other.server\> $DEFAULT
What this does
Putting a special “filing” recipe like the one shown above into your .procmailrc file before the despamming code will make sure that mail from (or to) the address you specify doesn't get mistakenly filed as spam, but instead lands in your default mailbox.
Many people like to keep certain messages separate from their regular mail, either to make sure they don't miss it (for example, mail from the boss), or to allow them to read it later (for example, mail from busy mailing lists).
Procmail is able to filter mail based on any aspect of the mail message. The most common filters use the From:, To:, or Subject: headers. For instance, if you want all mail from your friend johndoe@ece.concordia.ca to be saved in a mailbox called john, you could use a procmail recipe like this:
:0: * ^From:.*\<johndoe@ece.concordia.ca\> john
The best place to insert such a recipe is usually after all de-spamming and vacation auto-replying have taken place; the spot is indicated in our template with “If you want any special refiling, do it here”. But if that mail tends to get tagged as spam, then put your refiling recipe before the de-spamming.
What this does
The example recipe above says that if a line in the mail headers starts with (^) the word From:, followed by johndoe@ece.concordia.ca, with anything else in between (.*), then the mail should be put in the folder john. That folder is in your IMAP directory ($HOME/mail), assuming that you used our template, which sets the MAILDIR variable.
The reason for the .* (“anything else in between”) is to make certain that you get a proper match, even if the real From: header looks like this:
From: "John Q. Doe" <johndoe@ece.concordia.ca>
More examples
Sorting mail from a mailing list is almost the same. Mailing list messages are normally sent 'To:' the list name. So if you're subscribed to reallyfastcars@mailing.list.server, you could sort mail from that list into the fastcars folder using a procmail recipe like this:
:0: * ^To:.*\<reallyfastcars@mailing.list.server\> fastcars
You can always check your old e-mail messages to find a good header to match on. If you have turned on delivery logging, you can check the procmail log file (delivery.log in our example) after you make any changes to your .procmailrc file, in order to make certain your mail is being filtered the way you expect. Otherwise, you can determine it based on whether or not future messages land in the correct folder.
You can put in as many of these recipes as you like, to filter your incoming email by matching messages to certain conditions, and placing the messages into the appropriate folder.
You may already know that you can forward all your mail using a
.forward file, but that's not compatible with using
procmail. You can use procmail to forward some of your mail, for
example: only the non-spam, or only messages from a particular
person. What gets forwarded depends on where in your
.procmailrc file you place the forwarding recipe.
Here's how to forward all of your mail; there will be no copy kept in your ENCS mailbox for any message forwarded this way:
:0 ! self@other.addr.ess
Of course you must replace self@other.addr.ess with
the address you want to forward to. Don't lose the “!”;
this means “forward it” to procmail.
Here's how to forward just a copy of your mail; this way, another copy
of the message is delivered to your ENCS mailbox. The only difference
is the presence of the c flag on the first line, which
means “do this to a copy of the message”:
:0c ! self@other.addr.ess
(If you do this, make sure you continue checking your mail at GCS, otherwise your mailbox may fill up, which will cause all sorts of problems with your forwarding.)
Finally, here's an example showing only mail for the “reallyfastcars” mailing list being forwarded:
:0 * ^To:.*reallyfastcars@mailing.list.server ! self@other.addr.ess
If you use mh to read your mail, then you must deliver mail not into regular IMAP-style folders (where several mail messages are stored in a single file), but into mh-style directories (that contain one mail message per file).
Procmail can do this for you if you modify each “delivery” line (the last line of each “recipe”) to tell procmail to deliver to an mh-style directory. The “DEFAULT” folder can also be set as an MH folder.
We assume that only advanced mail users use mh; please be advised that it is not officially supported (though several sysadmins use it).
The simple way to deliver to mh directories
The easiest way is to append “/.” to the name of your mail folder, which must be a directory, and must already exist. Here's an example:
DEFAULT=$HOME/Mail/inbox/.
:0: * ^To:.*\<reallyfastcars@mailing.list.server\> $HOME/Mail/fastcars/.
Remember to make this change for the “delivery line” of every recipe.
The fancy way to deliver to mh directories
If you want the “unseen sequence” to be set, then you need to use rvcstore to deliver your mail, which means you need to take special precautions during delivery for locking and waiting for the program to finish. Also, it's best to leave the DEFAULT alone, but put a “catchall” recipe at the end of your .procmailrc file. This is for advanced users only; we provide examples, but you're basically on your own to figure this out.
First, make sure that you have
Unseen-Sequence: unseen
in your .mh_profile file. Then, to make later lines a bit easier to write, create this variable in .procmailrc:
FILE="/encs/bin/rcvstore -create -unseen"
You won't be able to count on procmail to automatically determine the right lockfile name, so you can either specify a lockfile as part of every recipe, or use global locking. Global locking can cause problems if you receive a lot of mail at once, but it is necessary if you do any fancy logging, so, near the top of .procmailrc, put:
LOCKSLEEP=2 LOCKFILE=$HOME/Mail/procmail.lock
Make sure that the above two lines occur before any LOGFILE=blah line, so that you also protect your logfile (if used) against simulatenous access by multiple instances of procmail.
Finally, you can now write your recipes, making sure to use the “w” flag to wait for the status of the delivery (so that procmail can fall back to a default if something goes wrong):
:0w * ^To:.*\<reallyfastcars@mailing.list.server\> |$FILE +fastcars
