Most organizations employ retention policies so as to be compliant with legal requirements. The simple fact that all kind of information concerning an Organization is delivered via email, should be enough to appreciate why legislators demand that this communication channel is correctly managed.

The need to satisfy legal requirements is worrisome to many administrators, once they appreciate the key role they play. IT should be about technology. Instead if lucky, we end up with an attorney advising us on how to manage our servers. Many won't even have this luxury and are expected to do everything on their own.

At the end of the day being compliant involves enforcing email retention policies. The retention period provides a time window allowing us to go back and recover emails. On one hand the policy defines the length of time during which we can hold on to information without breaching Data Protection and Privacy laws. On the other hand the policy limits our obligation to supply information in case of an investigation or litigation.

It is certainly not my intention to delve into matters such as Data Protection, Privacy and Litigation. My topic today is retention policies. More specifically I want to discuss the scope of such policies and how in some cases emails might be falling out of the policy scope at our own peril.

Which Emails should we retain?

Very often the mailbox store, the location where emails are saved, is considered to be the perfect place to enforce a retention policy. After all, the mailbox is where received emails are deposited and new emails are created. Thus if the policy allows us to recover all emails within a mailbox (including those deleted) we should be fine. In this context, a mailbox database archiving solution is normally considered to be an integral part of email retention. At the same time the database is defining the physical scope of the retention policy.

In this scenario, is the retention policy truly covering all emails? To answer this, we need to better define which emails our server is responsible for. When receiving an email over SMTP we have the option to either accept or reject the email. Rejection is the standard method to refuse responsibility for an email. On the other hand if we accept the email at SMTP transport level we are expected to deliver it to the intended recipient. If we later abandon delivery, the SMTP specifications require us to notify the original sender with an NDR. Thus from a technical point of view we have a clear definition of how an email server accepts and refuses responsibility for email delivery.

With this definition, we are broadening the scope our retention policies are required to cover. To understand this point let's have a look at this setup:

The internet facing SMTP transport will immediately decide between accepting and rejecting emails. The Email Hygiene stage could also advise the SMTP Transport, flagging rejection of spam and malware infected emails. If not rejected the email is delivered to the Mailbox Store. This is the ideal scenario; emails are either rejected or delivered. Retention policies enforced at the Mailbox Store correctly cover all emails.

In practice many email servers do not follow the SMTP specifications to the letter. Here is an example:

In this case the Email Hygiene layer, apart from causing email rejection may also silently delete emails. This is a very common option in anti-spam filters where emails are deleted without generating any NDR. The NDR is not generated in order to avoid backscatter problems. So now we have emails that are accepted but never delivered. The email never reaches the mailbox and thus the retention policy is not enforced. Here I am using Email Hygiene as an example. Of course the same is true for any other email server component that may cause silent deletion of emails.

What's the Problem with Deleting Junk?

As long as our retention policy is only failing to cover Junk emails, silent deletion should not be cause of concern. However do appreciate that here Email Hygiene is only an example. It is up to you to check:

• if any other software may be causing silent deletion at transport level
• if the retention policy scope is limited to the mailbox store

Even if we stick to the Email Hygiene example, are we sure only junk and malware is being deleted? No matter what software vendors say, the possibility of false classification exists. This may lead to the loss of legitimate emails. What if the lost email happens to be the one you require during litigation? Shouldn't this email also be covered by your retention policy?

Some may argue that in practice Email Hygiene applications are very accurate and the chance for something like this to happen is next to nil. The counter argument is that software is made by humans and managed by humans. The possibility of something going wrong should not be underestimated.

You have the possibility of bugs and errors from software vendors. We normally only hear of the most serious bugs concerning the big vendors because of the noise their name guarantees. The fact is that no one is immune to error.

A much more common cause for software to go wrong is incorrect configurations. For example if your software allows for manual email blacklisting, then getting a false classification is just a matter of setting the wrong entry at the blacklist. Just imagine someone erroneously blacklisting a sender when he was instead supposed to whitelist him.

In all cases the point is, why risk when the technology allowing your server to be compliant is available?

Solutions

This type of problem can be dealt with when designing the system and choosing the components that provide additional functionality to the email server.

One obvious solution is to prefer SMTP rejection over silent deletion. When it comes to email hygiene we discussed this in Anti-Spam Golden Rules – Reject if You Can. However this is only possible if the email is blocked immediately before it is accepted by the internet facing SMTP transport.

Another option is to employ solutions that provide built in archiving. Email Hygiene solutions such as IMF Tune, today allow you to archive deleted emails. Furthermore once archived emails reach the specified age limit; these may also be purged automatically.

In this manner we can extend the reach of our retention policy. The mailbox store is no longer a physical limit. If our company has a policy that all emails are retained for 60 days minimum, all we need is to provide the necessary disk space for storing 60 days of emails to our Email Hygiene solution.

Final Tips

In the last few years the IT world developed solutions to respond to the legislation requiring Organizations to correctly manage the information on their servers. Today many have absorbed the core concepts and made significant steps towards compliancy. However some are unaware that their solution is less than bullet proof. Luckily the solutions to close the gaps are readily available.