Spam Prevention Methods:

E-mail Message Flow to the Outlook Junk E-mail Folder

Dwight Jesseman has gone to great trouble to create detailed and accurate flowcharts of the complicated path which messages take from the point they hit the UF SMTP server all the way through until they end up in either the Inbox or the Junk E-mail Folder. These flowcharts are available in two formats:

• E-mail Flowcharts in Visio format
• E-mail Flowcharts in PDF format

The descriptions which follow are meant to supplement these flowcharts and to provide OU Admins information to best understand, monitor, and influence how spam is handled within our e-mail system.

Centralized Spam Management via SpamAssassin

All incoming email is first processed centrally by the Open Systems Group. As part of this process it is scored by the UF's SpamAssassin implementation which marks the message headers accordingly. This service, which is actively managed by the Open Systems Group, is our best (and perhaps only really useful) weapon against spam. Once those messages arrive at our Exchange server, we use SPAMtracker's Assassin2Exchange filter to convert those scores into SCL (Spam Confidence Level) scores within our email system. SMTPTracker is set to change SpamAssassin scores of 5 or more to "9", which causes such messages to be placed into the Junk E-mail folder. SMTPTracker also sets SCL scores to "0" for messages originating from the IFAS and UF listservs, from Gatorlink and from Blackberry as shown below.

Spam Management via IMF in Exchange 2003

Prior to getting this filter and utilizing central scanning facilities, we used only Exchange's Intelligent Message Filtering (IMF) for our spam control. Infrequent updates for that, however, made it a poor tool by itself. The current configuration settings for that are shown below.

Note that there is a typo in the IMF configuration dialog box. The text "Move messages with an SCL rating greater than or equal to:" should read "Move messages with an SCL rating greater than:". IMF still provides SCL scoring for us, but the additional SpamAssassin integration improves things considerably. SMTPTracker plugs into the process after the initial IMF scoring.

The complete flowchart of "Email Message Flow to the Outlook Junk E-mail Folder" (thanks Dwight!) helps greatly in understanding our complex overall mail handling processes.

Client-side Control

Setting Outlook to display the SCL scores for the inbox and the Junk E-mail folder and monitoring those is pretty much a necessary prerequisite to understanding how our complex scoring methods operate in actual practice. When doing so, you may want to add the SCL score as the first column--it takes little display space and is readily apparent when displayed that way. That in combination with the above-mentioned flowchart helps tremendously in understanding all this.

Scores and their consequences

Once you configure Outlook to show the SCL scores for the Junk E-mail Folder and the Inbox you should note:

Junk E-mail Folder - Messages can end up here in one of four ways...

1. Manually moved there by the user - always a possibility regardless of score
2. Moved due to IMF scoring - generally the case for SCL scores > 4 and < 9 *
3. Moved due to SpamAssassin scoring - generally the case for SCL scores = 9 *
4. Moved due to client-side scoring - requires that client is in cached mode **

* especially if Outlook Junk E-mail Options are set to "No Automatic Filtering" or if the client is not in cached mode

** dependent on optional level settings as in dialog box shown below--also may be due to the Blocked Senders list

Inbox - Messages can end up here in one of four ways...

1. Manually moved there by the user from the Junk E-mail folder - always a possibility regardless of score
2. Left there due to SMTPTracker whitelisting - often the case for SCL scores = 0 *
3. Left there due to low score on via either IMF, SpamAssassin, or local filter scoring - generally the case for SCL scores > 0 and < 5
4. Moved there by client-side Safe Senders rules - Messages with an SCL score > 4

* (can be trumped by local client-side filtering)

You may also look at the e-mail header information to determine the SpamAssassin score of a message. To do that:

1. Open the e-mail message in its own window by double-clicking it in the list of messages
2. Select "Options" from message's "View" menu
Scroll through the text box labeled "Internet headers:", look for lines looking something like
X-Spam-Level: ******
X-UFL-Spam-Level: ******

The SpamAssassin score is represented by the number of asterisks listed. Note that greylisting details are also included in the header. That information is useful for diagnosing Bounced E-mail from Senders.

Outlook Junk E-mail Options

In Steve's experience, trying to change Outlook Junk E-mail options from the recommended setting on the client side (shown below)

causes more harm than good--often dramatically increasing false positives w/o helping much/if any in catching real spam. The biggest risk is the potential override of SMTPTracker whitelisting of IFAS and UF listserv messages.

Safe Senders List

When false positives are noted, it is useful to add e-mail addresses (and possibly domains) to the Safe Senders list as documented here. Another easy way to accomplish this is to select the message that was erroneously marked as spam, right-click it, and choose "Mark as Not Junk" from the resultant menu. This will move the message back into your inbox and also add the sender to the Safe Senders List.

Note: the Safe Senders, Safe Recipients and Blocked Senders lists are all processed regardless of Outlook mode and can be set in OWA or Outlook.

Using Rules

Rules can be configured by users in Outlook to move messages to the Junk E-mail folder. Due to the amount of work needed to do that effectively, with constantly changing spam profiles, it may not be worth it to most. Also, rules involve a complex interaction between the client and server that is not readily apparent.

Reporting false positives/negatives

You are encouraged to report false positives/negatives to help the Open Systems group better mark similar messages in the future:

"If a message marked by the system as spam in error, the user 
should forward the entire message (including the report) to 
report-ham@ufl.edu. If a message that the user thinks is SPAM 
is not marked as such, they should forward the message to 
report-spam@ufl.edu."

How infected messages are handled: the Quarantine folder

You and your users may on occasion see messages with the subject "Unwanted message body detected, entire message has been moved to quarantine folder~" and a corresponding message in the quarantine folder. These message are due to our anti-virus protection systems.

There are two levels of anti-virus protection affecting a message:

1. McAfee GroupShield on the Exchange server and
2. your desktop McAfee anti-virus, specifically the On-Delivery E-Mail Scanner.

Until roughly August 2007, GroupShield has been configured to "Delete the Item" when it processed a message with a virus. Such a setting did not delete the message itself, but rather the attached virus. After virus removal, the message was then sent to the recipient.

When the recipient received the message, the On-Delivery E-Mail Scanner would scan the message and determine that it looked like a phishing e-mail. At this point the e-mail would be placed in the quarantine folder and a corresponding message would be placed in the mailbox indicating that a message had been moved to the quarantine folder. The On-Delivery E-Mail Scanner takes action on the message after the Outlook server side rules.

Because of the possible client confusion surrounding a quarantine e-mail message the GroupShield configuration has now been changed. GroupShield is now configured to "Delete the Message" when it processed a message with a virus. This change does not eliminate the possibility of messages being placed in the quarantine folder, but reduces it greatly.