Exchange 2003 has a rich set of tools covering all kind of administrative tasks. Considering their quality, some of these could have easily been a successful commercial application. The Best Practices Analyzer is one that immediately springs to my mind.
Other tools, although less polished, also provide excellent functionality. Today we take a general look at stress testing and tools useful in this area. This includes the Exchange Server Stress and Performance (ESP) and the Load Simulator (LoadSim). LoadSim is a true classic. It has been around for a long time and survived many Exchange version transitions. Check References for download links.
LoadSim is a MAPI based application that simulates internal Exchange user activity. Amongst others, it sends emails, makes appointments, creates contacts, posts to public folders, generates browsing activity etc. Indeed it is a very comprehensive simulation of internal activity typical of Outlook users.
LoadSim is great at simulating MAPI activity. Nevertheless a live Exchange environment must handle work loads from various other interfaces for which LoadSim does not cater. This includes inbound SMTP traffic and access through Mobile devices. Depending on the organization type, POP3 and IMAP4 may also be in use. Finally one should not forget the load from custom applications and gateways to foreign systems.
Microsoft also provides the Exchange Server Stress and Performance tool. This is able to load Exchange by interacting through various protocols. When combined with LoadSim the simulation becomes much closer to the live scenario. Both of these should only be used in test environments. The aim here is to drive Exchange to its limit.
It is a fact that many Exchange deployments do not undergo this type of testing. Following best practices is often enough to achieve good results. Once the design gets more complex and the number of users rises, having a good understanding of the system limits becomes more important.
To be useful, a simulation must closely replicate the intended live scenario. One should match both the hardware specifications and the software usage characteristics. From the Exchange side some key parameters include the number of users, type of client devices in use, the expected load per user and the load distribution. There can indeed be many variables. The simulation tools provide plenty of configurability. Amongst others you can specify the number of user mailboxes, identify the interfaces to target, specify the load level etc.
Apart for considering the above mentioned Exchange elements, for a comprehensive simulation, one should not forget third party extensions. There is a wide variety of these. Their tight integration enables a seamless, consistent experience, but often also plays an important role in performance and resource consumption considerations. Thus including any server side extensions is also important in these tests.
The role of third party applications introduces extra challenges. The knowledge about Exchange is fairly wide-spread and there is plenty of readily available support. Third-party applications may require some more digging. It might require contacting the specific vendors to ensure a good understanding of how to best load the application.
As an example we can look at Exchange integrated anti-virus applications. These are expected to analyze emails before users access them. This can be achieved through different integration openings. One could plug into the Exchange transport blocking delivery of infected emails. Others could integrate at the store level, just before users are able to access the emails.
Thus in some cases extensions may introduce a delivery delay. If the filter is unable to handle large email volumes, it causes a bottleneck. If the application runs directly on the Exchange server machine, the resources consumed also become relevant.
Clearly including the extensions at the simulation stage takes us a long way in reproducing the live environment. Being aware of how the application plugs to Exchange highlights the affected interfaces. Thus we can better interpret the test results.
If we want to be very accurate all of this might still not be enough. If we look at the way applications are developed it is certain that these are optimized to only absorb resources when real work is necessary. In an anti-virus application this means that indeed the content of the test emails can make a big difference. For example executable attachments might be handled differently from text attachments, and compressed files open yet another level of complexity.
From this discussion we can see that apart from generating load, we also need to consider its quality. Submitting random content or using a small set of sample emails might not be effective enough. Random emails are fine only as long as the processing involved is content neutral. Indeed in professional test environments the use of real live data is very important. This typically requires saving live data and then re-using it on the test network.
I hope this article helped better understanding the factors involved in testing Exchange server configurations. I intentionally kept the discussion as general as possible and avoided getting into the specifics of the available tools. Both ESP and LoadSim include detailed documentation covering these tools in depth.
I dedicated most space to the less obvious role played by third party extensions since this is the area where I have most experience. Luckily most deployments do not require this level of testing and often following best practices is enough.
References
Tools for Exchange Server 2003
Using Exchange Server 2003 Stress Tools in a Test Lab
Exchange Server Stress and Performance 2003
Microsoft Exchange Server 2003 Load Simulator (LoadSim)