I never made a secret of my support for the Intelligent Message Filter. Very often it works great. However just like any other filter, IMF is not perfect and could do a better job when dealing with certain spam. The latest spam wave that caught IMF unprepared brought up this issue giving us the opportunity to discuss what I consider to be IMF Achilles' heel. Since being a supporter does not mean loosing objectivity, I dedicate this article to some healthy IMF criticism.
This article largely applies to both the Exchange 2003 IMF and its Exchange 2007 successor that was unimaginatively renamed to Content Filter. Hereafter I will refer to both as IMF.
Another Spam Wave
In the last few days quite a few spam emails of the type shown below, landed into my inbox. Clearly the message is contained in an image, which of course I don't allow Outlook to download.
The first thing I did was to test these emails on both Exchange 2003 and Exchange 2007 Standard Edition. As expected both filter versions have difficulty classifying these emails. Most of the spam variants are assigned SCLs of 1 and 2 with just a few getting an SCL of 5 or higher.
You would have probably noticed the fact that I tested against Exchange 2007 Standard Edition and not Enterprise. This is not a minor detail as we shall see shortly.
Next we take a closer look at the raw email. As we already know, all the "information" is contained within an image. Additionally the email also includes loads of garbage in an attempt to trick filtering. Unfortunately the combination of garbage and content is in this case succeeding to trick IMF. Here is a little snippet, highlighting some interesting content including an HTML image and a Java Script.
The Java Script is in fact enclosed within an HTML comment block. Thus it is pure garbage. However since it is common in all spam emails I had in my Inbox, I will make use of this to identify the spam.
IMF Achilles' Heel
The real problem here is not the fact that an email remained unfiltered. Every filter gives some false negatives (unfiltered spam). These are not so terrible as long as the filter is able to fight back promptly.
The key issue concerns the IMF reaction time. IMF in both Exchange 2003 and Exchange 2007 Standard edition relies on two monthly updates for their filtering intelligence to be refreshed. In practice most spam waves are similar enough allowing IMF to keep up the filtering between refreshes. However clearly this is not the case here.
It is a known fact that spammers try to craft emails in the attempt to bypass filtering. Spammers have been doing this against SpamAssassin for a long time because of its widespread adoption. I would not be surprised if this email was tuned against IMF, catching it unprepared waiting for the next update.
Users of Exchange 2007 Enterprise Edition should not be in the same waters. In this case updates are available on a daily basis. I am saying "should" because I haven't verified this point.
Temporary Measures
Until the next update is available, administrators have some tools to help them mitigate this issue. Exchange 2003 provides the XML custom weights file, whereas Exchange 2007 provides the Custom Words list. These two solutions only allow for simple keyword and phrase matching. However they do allow us to match against raw HTML bodies.
Referring back to the raw email content shown earlier, in this particular example we could block emails based on this phrase:
"text/javascript"
For Exchange 2003 the XML file content would look like this:
Blocking emails containing scripts is quite normal. However since the script is within an HTML comment block this is just garbage that the spammer could replace any time. We will discuss this point a bit further in the concluding section.
It is easy to make mistakes when authoring XML without the necessary tools. So to avoid risks you can just get a copy of the XML file from the article download section. The file must be named MSExchange.UceContentFilter.xml and saved to the IMF directory under:
Drive_Letter:\Program Files\Exchsvr\Bin\MSCFV2\<latest update>
If this is the first time the XML is being created then you will need to restart the SMTP Service for it to be picked up. Otherwise you will need to merge the XML into the one currently configured. In the latter case no service restart is required.
Configuring Exchange 2007 is a matter of going to the Content Filter configuration and entering the phrase under the Custom Word list. For more details I suggest you to look at my earlier article, The Exchange 2007 Content Filter Agent.
Final Tips
Despite the stopgap solution presented here, the way IMF deals with new spam could certainly be improved. Our custom word is a very primitive filter. Spammers could break this with just a little variation. However we only need this to work until the next update is available. At that point hopefully IMF will be able to do the filtering straight away.
One reason for not using a more complex matching phrase is due to the limited functionality the custom words feature provides. We could be a lot more selective if we could combine multiple phrases with AND, OR, NOT operators just like we do with search engines. For that a 3rd party tool such as IMF Tune would however be necessary.
References
Welcome to IMF Regular Updates
Exchange 2007 - Anti-Spam Updates
The Exchange 2007 Content Filter Agent