The Intelligent Message Filter performance counters provide a good picture of the general filtering activity. Today I loaded IMF with a few thousand emails taken from a live environment in order to see what interesting points come out.

Loading the Performance Counters

Here are the steps to get the counters in place:

1. Start the Performance Monitor from Administrative Tools.

2. You will see the standard counters loaded at the lower pane. Select and delete these.

3. Click on the plus toolbar button to open the Add Counter dialog.

4. Under Performance object select:
   MSExchange Intelligent Message Filter

   

   In case you are following this on a test machine, note that you need to receive at least one email before this object is exposed.

5. You may now choose from the list of counters.

   

   The most interesting being:

   • Total Messages Assigned an SCL Rating of x
   • Total messages scanned for UCE
   • Total and percentage UCE detected

6. Select the counters to monitor and click on Add.

Looking at the Counter Data

The data presented in this article was obtained by re-submitting a few thousands emails grabbed from a live environment. The system was composed of Exchange 2003 SP2 without any IMF updates.

The performance monitor provides three views for presenting the counter data; a graph, a histogram and a report. The graph is most useful when analyzing data against time. In this case I find the other views to be the most interesting. Here is what the Performance Monitors showed once the email processing completed:

Some of these counters measure the UCE detection level. These are based on the IMF gateway threshold configured at the Exchange System Manager. In my setup the gateway threshold was set to 7. Thus the 'Total UCE Messages Acted Upon' counter shows the sum of messages assigned SCL 7, SCL 8 and SCL 9.

The UCE detection measure, does not account for emails routed to the Junk Email folder. This is controlled by the store threshold, configured to match emails with lower SCL ratings.

It is interesting to see how the detection level increases as we lower the reference threshold. I compiled this data in the table that follows:

Consider a configuration with a store threshold of 4 and no gateway action. This would cover all messages with SCL ratings 5 to 9. Given my test emails 66.43% of these would be routed to the Junk Email folder.

SCL Assignment Distribution

Probably the most interesting result of this exercise was the distribution of SCL assignments. As you can immediately see from the histogram, SCL2 and 3 get a fairly small proportion of hits. This is good. These are the SCL ratings where email classification is most uncertain. In turn, this uncertainty is what leads to most false positives.

Thus I can safely allow such emails to go to the Inbox. In any case most spam will get a higher rating. Lowering the threshold further does not justify the increased risk of false positives.

At this point you may want to check a table I presented in an earlier article. In 'IMF SCL Configuration - getting it right' under the sub-heading 'What does the SCL really mean?', I compiled a table showing the % confidence level associated with each SCL rating. This was written more than a year ago for IMF v1, but in general the values presented should still be indicative. This data shows how the email classification confidence drops sharply when hitting the lower SCL ratings. Again this reinforces the idea of staying away from the lower SCL ratings when setting thresholds.

Final Tips

The Performance Monitor data may initially seem difficult to interpret. Once we attach a meaning to the presented data, this information becomes a lot more useful. Whether you want to visualize the Intelligent Message Filter filtering activity, or even analyze a possible threshold adjustment, the Performance Monitor can provide valuable feedback.