WinDeveloper IMF Tune

WinDeveloper IMF Tune
WinDeveloper IMF Tune

Looking at IMF through the Performance Monitor

Alexander Zammit

Alexander Zammit Photo

Software Development Consultant. Involved in the development of various Enterprise software solutions. Today focused on Blockchain and DLT technologies.

  • Published: May 02, 2006
  • Category: Anti-Spam
  • Votes: 4.8 out of 5 - 4 Votes
Cast your Vote
Poor Excellent

Whether you want to visualize the Intelligent Message Filter filtering activity, or even analyze a possible threshold adjustment, the Performance Monitor can provide valuable feedback.

The Intelligent Message Filter performance counters provide a good picture of the general filtering activity. Today I loaded IMF with a few thousand emails taken from a live environment in order to see what interesting points come out.

Loading the Performance Counters

Here are the steps to get the counters in place:

  1. Start the Performance Monitor from Administrative Tools.

  2. You will see the standard counters loaded at the lower pane. Select and delete these.

  3. Click on the plus toolbar button to open the Add Counter dialog.

  4. Under Performance object select:
    MSExchange Intelligent Message Filter

    IMF Performance Object

    In case you are following this on a test machine, note that you need to receive at least one email before this object is exposed.

  5. You may now choose from the list of counters.

    IMF Performance Counters

    The most interesting being:

    • Total Messages Assigned an SCL Rating of x
    • Total messages scanned for UCE
    • Total and percentage UCE detected
  6. Select the counters to monitor and click on Add.

Looking at the Counter Data

The data presented in this article was obtained by re-submitting a few thousands emails grabbed from a live environment. The system was composed of Exchange 2003 SP2 without any IMF updates.

The performance monitor provides three views for presenting the counter data; a graph, a histogram and a report. The graph is most useful when analyzing data against time. In this case I find the other views to be the most interesting. Here is what the Performance Monitors showed once the email processing completed:

Performance Monitor Histogram

Performance Monitor Report

Some of these counters measure the UCE detection level. These are based on the IMF gateway threshold configured at the Exchange System Manager. In my setup the gateway threshold was set to 7. Thus the 'Total UCE Messages Acted Upon' counter shows the sum of messages assigned SCL 7, SCL 8 and SCL 9.

The UCE detection measure, does not account for emails routed to the Junk Email folder. This is controlled by the store threshold, configured to match emails with lower SCL ratings.

It is interesting to see how the detection level increases as we lower the reference threshold. I compiled this data in the table that follows:

SCL Level Total Messages Assigned SCL x Total UCE Messages Acted Upon %UCE Out of Total Messages
SCL 0 978 6484 99.94
SCL 1 810 5506 84.86
SCL 2 74 4696 72.38
SCL 3 86 4622 71.24
SCL 4 226 4536 69.91
SCL 5 776 4310 66.43
SCL 6 631 3534 54.47
SCL 7 1229 2903 44.74
SCL 8 1450 1674 25.80
SCL 9 224 224 3.45

Consider a configuration with a store threshold of 4 and no gateway action. This would cover all messages with SCL ratings 5 to 9. Given my test emails 66.43% of these would be routed to the Junk Email folder.

SCL Assignment Distribution

Probably the most interesting result of this exercise was the distribution of SCL assignments. As you can immediately see from the histogram, SCL2 and 3 get a fairly small proportion of hits. This is good. These are the SCL ratings where email classification is most uncertain. In turn, this uncertainty is what leads to most false positives.

Thus I can safely allow such emails to go to the Inbox. In any case most spam will get a higher rating. Lowering the threshold further does not justify the increased risk of false positives.

At this point you may want to check a table I presented in an earlier article. In 'IMF SCL Configuration - getting it right' under the sub-heading 'What does the SCL really mean?', I compiled a table showing the % confidence level associated with each SCL rating. This was written more than a year ago for IMF v1, but in general the values presented should still be indicative. This data shows how the email classification confidence drops sharply when hitting the lower SCL ratings. Again this reinforces the idea of staying away from the lower SCL ratings when setting thresholds.

Final Tips

The Performance Monitor data may initially seem difficult to interpret. Once we attach a meaning to the presented data, this information becomes a lot more useful. Whether you want to visualize the Intelligent Message Filter filtering activity, or even analyze a possible threshold adjustment, the Performance Monitor can provide valuable feedback.

Copyright © 2005 - 2024 All rights reserved. ExchangeInbox.com is not affiliated with Microsoft Corporation