Philosophies of Spam Solutions

Written by: Ron Edison, founder and CTO, Internet Defense Technologies

While spam by the billions reaches out to users around the globe, they are scanned, blocked, deleted, sorted, filtered, rejected, quarantined, moved, dropped, replied to, and bounced by a panorama of spam solutions and email systems as varied as they are many. Accompanying the ubiquity of spam is a mixed bag of strategies and applications to deal with it.

And, while users rail at spam, they typically complain far louder about "false positives" (legitimate messages treated, blocked, etc., incorrectly by spam filtering systems) — and for good reason. If there's anything worse than getting drowned in spam, many users agree, it is missing "that all important message", which at least seemingly is often subject to collateral damage as servers wage their constant battle with spam. Here there is opportunity for an effective solution as long as it can deliver accuracy.

While spam and its effects have been the topic of endless discourse, rarely mentioned in much detail is the underlying philosophy or even exact goal of one solution or another. Certainly, one could say that — of course — the goal is to block all spam and deliver all "ham" (the colloquial for "legitimate email"), but there is more to this than meets the eye.

Enter "reputation.” More and more reputation-based systems exist today. Corporations invest in managing their "email reputation" to certify their email as "ham" and keep their servers off blacklists and their email reaching user inboxes. Still, there are few guarantees, "please add us to your safe senders list," "if you don't receive your confirmation email, please check your spam folder," and other such missives are often seen from small and large companies alike.

Much spam gets blocked, however it is often in the style of "accuracy by volume" (or inaccuracy by volume as the case may be). Spam, sometimes quite a bit, still gets through while countless user hours are wasted asking — "did you get that email?" — or worse, the opportunity is completely missed, and critical data goes undelivered. Plus, some systems block by inconclusive criteria leading to a great many legitimate messages getting blocked, with no way for users to access them or even become aware of the problem. Are messages identified as spam blocked silently? Moved to users' spam folders? Tagged? Simply deleted with no notification at all? And, is it up to the user to check a quarantine or "Spam" folder for messages thus categorized? Or are users expected to access their spam folder or "quarantine" some other way? The answers are inextricably linked to the accuracy of the system. Once more, there is opportunity for a VAR to showcase knowledge by explaining these pitfalls to a potential client making a technology decision about their security filter.

It has certainly been determined conclusively that spam is best dealt with at the mail gateway, the entrance point of mail to any given domain name or system. But again what exactly is done at that point varies widely, governed by the philosophy of its architects. In fact, a well-configured gateway can reduce the flood of mail (comprised of mostly spam) to a tiny trickle of mostly legitimate mail.

For any business system, the focus of the philosophy should actually be on the legitimate email — the ham — not the spam. This, of course, is the entire point. A spam solution that requires the user to do more work or worse, the spam solution that blocks legitimate messages, even occasionally, has defeated the original idea of such a solution, to ease and speed communications, to make email more effective and reliable.

Spam folders, message digests, tagging messages for users to sort or filter on their email clients, all these things are incomplete at best and worse than useless at worst. Indeed, at times just the idea of an effective spam solution is met with disbelief that can be difficult to overcome. If the goods can be delivered, however, then a free trial that is easily enabled can be an effective approach.
Of course, legitimate email should be delivered with utmost speed while spam is blocked, that is the goal and should be the driving philosophy. Unfortunately, many solutions will never do this since they block spam based on criteria that can be shown to block legitimate mail in many cases, and move to or place such false positives in a location rarely viewed or at times even inaccessible to the end user. There are even some systems that really don't do much spam filtering at all but use a few canned solutions and then allow certain "certified senders" to bypass the restrictions. These commonly block legitimate mail as well.

What if it was your company? Your email? You would want the same focus: Every piece of legitimate mail delivered at once with spam kept to a minimum. Email that cannot be conclusively determined one way or the other (there are always some) should be very easily accessible and indeed the user should be notified, these messages should not be shuffled away in a folder seldom checked or in a quarantine protected by credentials not remembered. The trick is to keep these "gray area" messages to an absolute minimum.

The good news is that it is possible: email can be an effective tool without wading through spam or wondering what messages have been incorrectly blocked this week, this day, this hour.

Ron Edison is founder and CTO of security vendor Internet Defense Technologies, which features as its flagship product, Total Mail Defense, a spam, email-born virus and malware solution.