Intelligent Message Filter, Content Filter, can do more...

WinDeveloper IMF Tune
WinDeveloper IMF Tune

Anti-Spam Golden Rules – Reject if You Can

Alexander Zammit

Alexander Zammit Photo

Software Development Consultant. Involved in the development of various Enterprise software solutions. Today focused on Blockchain and DLT technologies.

  • Published: Mar 18, 2008
  • Category: Anti-Spam
  • Votes: 4.4 out of 5 - 13 Votes
Cast your Vote
Poor Excellent

Which is the most appropriate method to filter emails classified as spam? We answer this question comparing the characteristics of rejection, deletion, centralized quarantining and per mailbox junk folders.

Most spam filters present a standard set of options for isolating emails classified as spam. Typically the choice includes blocking emails at the gateway, depositing to a central quarantine and filtering to per mailbox junk folders.

One question does always come up. Which of these is the most appropriate? Why pick one and not the other? Today I make the case for rejecting emails.

Rejection in Simple Terms

In our discussion a spam filter uses rejection to terminate the SMTP session of an email that is classified as spam. The sender is not allowed to completely submit the email. Instead the session ends with a rejection response composed of a numeric code and some text describing the rejection reason.

Rejection can only deliver its benefits, if the filter is running on the SMTP server handling foreign emails directly (at the network perimeter). Furthermore a filter can only trigger rejection if it plugs directly to the SMTP conversation of inbound emails. The first requirement depends on your setup. The second depends on how the filter is implemented.

Before delving into the merits of rejection let's just quickly look at other alternative filtering mechanisms:

  • Deletion - This is the process of accepting the email and deleting it. Deletion could be silent, leaving no indication of the filtered email. Being silent, the sender believes the email was delivered successfully. However the recipient never sees it. Alternatively deletion could generate an NDR so as to notify the email sender. We will be seeing the ramifications of this shortly.

  • Quarantine - The email is accepted but routed to some sort of quarantine repository. From here the administrator can review filtered emails.

  • Junk folder - A junk folder isolates filtered emails depositing them to a mailbox folder dedicated for this purpose. This is similar to Quarantine except that the review process is left up to the individual users.

Rejection has two characteristics that set it apart from the other filtering mechanisms:

  1. The email is not accepted.

  2. The sending host is automatically notified of this fact.

Non-Delivery Reports

Rejection frees our server from generating Non-Delivery Reports NDRs. RFC 2821 gives us the rule for NDR generation:

"If an SMTP server has accepted the task of relaying the mail and later finds that the destination is incorrect or that the mail cannot be delivered for some other reason, then it MUST construct an "undeliverable mail" notification message and send it to the originator of the undeliverable "

Rejection blocks the email without accepting it, thus freeing us from this obligation. It is true that very often anti-spam filters won't generate an NDR even if dropping an email after accepting it. However there is an important difference. Even though rejection frees us from NDR generation, the email sender still receives an NDR. In this scenario the responsibility of NDR generation is simply shifted from our server to the sending server. Of course here we are assuming that the sending server is RFC compliant and that the sender has not somehow blocked NDR generation at his end.

Compare this to the case when an email is accepted and subsequently deleted. If we wanted to notify the sender that email delivery was aborted, we would have to generate an NDR ourselves. Considering the huge amount of spam an average server handles, the NDR load is not insignificant.

So the obvious question is: Why would we want to notify a spammer that his email failed delivery? We certainly don't care for spammers. But we do care for legitimate senders whose email is incorrectly classified as spam. In this case the NDR would alert the sender to contact us through some other channel. This is where the rejection response text comes into play, as it provides the space for specifying alternative contact details.

Can't a spammer use the rejection response text to his advantage? Even with a blank response, doesn't the rejection in itself supply a spammer with useful information on what works and what doesn't work when emailing us?

To clarify this point it is best to categorize a bit email senders. In general spammers are just flooding the internet with their emails using a huge list of recipient addresses. They are not really attacking us. There is nothing of personal. They just want to deliver their emails. These spammers won't do much out of a rejection response. The diversity in the recipient list makes any kind of tuning based on rejection responses not feasible.

There is a second more problematic category of senders. This is when someone is specifically targeting our servers. A directory harvesting attack falls under this category. In this case other counter measures exist that allow us to defend ourselves without sacrificing the benefits of rejection. This was discussed in Tar Pitting Directory Harvesting Attacks.

We already highlighted the fact that generating NDRs at our server is in itself a waste of resources. This is not the end of the story. The NDR generation can be abused to launch a distributed denial of service attack DDoS. The attack works as follows. An attacker wants to launch a DDoS against domain.com. Our server generates NDRs in response to spam. Thus the attacker simply needs to send spam spoofing domain.com to us. Our server will then respond with an NDR addressed to the spoofed sender i.e. domain.com.

Apart from unwarily lending a hand to the attacker we can also end up listed on Backscatterer. Although not a list of spammers this can be used as an RBL, causing our own emails to be blocked. Here is the description of Backscatterer taken from their site:

"Email servers should be configured to provide Non-Delivery Reports (bounces) to local users only. Unacceptable email from anywhere else should be rejected.

If you send NDRs (bounces) to innocent people out there, this is considered abusive because it can amount to a DDOS. "

To recap this discussion let us stress the key facts. NDRs are good for notifying legitimate senders of false detection. However we should not generate the NDRs ourselves. Rejection enables us to do just that. If the sender wants an NDR, we just reject the email. The NDR will then be generated by the sending server.

Rejection vs. Quarantine and Junk Folders

In the following section I will be referring to both centralized quarantines and per mailbox junk folders as quarantine systems. This is because both give the opportunity to review filtered emails.

The comparison between Rejection and Deletion is easy to do. However the same type of comparison against a quarantine system is less appropriate. Rejection and Quarantine very often are employed side by side rather than in exclusion of each other.

The manual review of filtered emails is sometimes preferred as it allows humans to feel more in control. MS Exchange IMF/Content Filter users are certainly aware of the SCL 1 to 9 rating system. Here the lower the SCL, the higher the risk of false positives. In this context manual review can be used to verify classification of emails with lower SCL ratings. For example SCLs 8 and 9 are normally considered to require no manual review. However with SCLs 7 to 5 we move towards the grey area where review is more appropriate.

So why don't we just quarantine everything? In practice a Quarantine system can easily end up flooded with spam. Finding a false positive quickly becomes like finding a needle in a haystack. Many quarantines end up being purged blindly with the hope that everything was well classified. However a quarantine that is purged without proper review is nearly as good as an automatic server level silent deletion.

So the solution is very often that of finding a balance. Reject the emails where the likeliness of false positives is very low. This will reduce the amount of spam going to the quarantine increasing the chances of this being properly reviewed.

Final Tips

Today we looked at how email rejection helps dealing with spam. Rejection alerts legitimate senders of false positives without burdening our own servers with NDRs. Rejection also fits nicely with quarantine systems reducing spam flooding.

Another point that I intentionally kept for last is Legal Compliance. We have no responsibility for emails that were never allowed to enter our servers i.e. rejected. If we allow spam to enter our network we might actually be legally obliged to keep that in order to make sure not to delete legitimate emails. Compliancy regulations differ from one country to another so this may not be relevant in your case. However I do find it interesting how the benefits of rejection go beyond the basic spam filtering problem.

References

Tar Pitting Directory Harvesting Attacks

What's in the Junk Email Folder?

RFC 2821 - Simple Mail Transfer Protocol

Backscatterer

User Comments - Page 1 of 1

Look at my anti-spam solution :-) 6 Aug 2008 07:01
thx for the good EXMERGE (exchange 2007) article ;-p


Here's my solution against spam :-)
http://www.gpit.fr/Accueil/tabid/36/EntryID/3379/Default.aspx

other related informations on antispam powershell scripts included in exchange 2007:

http://www.networknet.nl/apps/wp/archives/335
Duminy 25 Jun 2008 03:43
Great Article .. we generally use Trend's Scanmail together with Exchange 2003 IMF, since ScanMail v8 doesn't have a reject option for SPAM.
skypilott2 2 Jun 2008 12:56
Excellent article. Very clear and concise, and quite helpful in explaining why reject is the better option on this filter.
Thanks.
OptimusBob 15 May 2008 06:14
Can I assume there is a way that IMF Tune hooks into IMF? Does IMF have a way for a bolt on application to do further processing? If it does, can you share or point me in the right direction?
Alexander Zammit 6 May 2008 13:20
In Exchange 2007 there is no disk archiving functionality. Ex07 has the quarantine functionality however the emails are quarantined to a mailbox not to disk.

For Archiving to disk you would need an add-on like IMF Tune if I may add that :))
OptimusBob 6 May 2008 11:17
Good article!

I use Exchange 2003 SP2 with IMF on Windows 2003 Server. I use IMF in the Archive mode with an SCL of 1. Yes, it seems extreme, but I have written my own IMF Archive Processor (IMFAP). We are hammered with spam. I use all the features of IMF and then build on them. IMFAP uses white lists, black lists, subject lists, body lists, ... I started working on it back in mid-2006. Now I only review less than 50 questionable emails a day. Without IMF and IMFAP we would have a flood of thousands of spam emails a day. I logged over 10,000 spam attempts in one day.

I have a question, I use the Archive directory with Exchange 2003, is there an archive or quarantine directory with Exchange 2007? If there is, do the emails get saved in the same EML file format that they did with Exchange 2003? Someday we may need to move to Exchange 2007 and I don't want to have completely rewrite IMFAP.
Alexander Zammit 26 Mar 2008 00:42
Since I am from WinDeveloper it is quite obvious I recommend IMF Tune.

However let's keep the comments focused on the article topic itself. IMF Tune does allow you to reject emails as explained above.

Brett 22 Mar 2008 22:52
Hi,

This article really does make sense!

I have 2003 windows server, with Exchange 07, & symantec mail security.
My biggest problem at the moment is SPAM!

What antispam program/s do you recommend??
(i currently DONT have the Spam agent for the mail security)

thanks,
Brett
Copyright © 2005 - 2024 All rights reserved. ExchangeInbox.com is not affiliated with Microsoft Corporation