"Curt Monash's publications provide unmatched insight into
technology and marketplace trends. I have read them avidly for over a decade."

--Larry Ellison, Chairman
and CEO, Oracle

"Curt Monash possesses the rare ability to distill the essence of technological issues into understandable terms. He is particularly adept at melding a firm's product positioning, corporate strategy, and valuation parameters into a concise and coherent framework upon which one can make an informed investment decision. He is a trusted resource."

Matthew P. Kaufler, CFA
Portfolio Manager
Clover Capital Management, Inc.

Note To Folks Looking for Monash University

Known-Spam-Blocking: The Best Anti-Spam Technology

Spam can be disguised in many ways, but it has one unalterable characteristic: Spam is sent to many email addresses. In our opinion, the best approach to spam-blocking is one based on this fact. Any email sent to enough different mailboxes is either spam, or a legitimate commercial email. (Most likely, it’s spam.) Conversely, if you monitor enough mailboxes, you’ll see multiple instances of any serious incident of spam.

This isn’t just theory. One of my internet service providers, Earthlink, is doing a great job of spam blocking right now. I get several hundred spams per day at my main email address. Earthlink blocks 95% of this (the last time I actually counted they had blocked 458 out of 481 spam) and sends this to a “Known Spam” folder. And I’ve never detected a single false positive.

What we particularly like about known-spam-blocking is that the technique is likely to keep working no matter what new tricks spammers come up with. And any tricks they develop that do confuse known-spam-blockers are likely to pose trouble for other spam-blocking technologies as well, whitelists possibly excepted.

There are four basic parts to a known-spam-blocker’s capabilities:

1. Harvesting the messages. If you set up dummy mailboxes, then anything sent to them is either an error or spam. Also, if you operate a lot of email accounts, anything sent to a whole lot of them is probable spam.

Both these approaches to spam harvesting are obviously easy for internet service providers and email outsourcers. Third-party anti-spam software and service providers need access to a sufficient number of mailboxes in order to compete, but that’s not a terribly high hurdle.

2. Identifying when “different” email messages are really the same. Serious spammers randomize certain aspects of their messages (hence those strings of nonsense characters right in the title of some spam) so as to defeat naīve spam blockers. An effective spam blocker has to see through the randomization and recognize patterns or signatures that are common to different instances of the same spam.

One way for the spam-blocking software to do this is to look at the call-to-action. Spam is sent with the goal of getting the recipient to click on a link, call a phone number, write to an address (email or snail), or maybe buy a stock. This action target will almost certainly be fairly consistent across many instances of the spam.

Another way is for the spam-blocker to do a rough analysis of the overall text. Some words may change; but the overall content will be pretty similar from instance to instance. This approach should continue to work well for all except the most purely graphical spam.

As for the graphical spam – well, most email that’s graphics-only is probably spam in the first place. And that -- er, that illustrates the next part.

3. Incorporate other spam-blocking techniques. No one spam-blocking technique is sufficient on its own. You can identify the vast majority of spam by tracking it across multiple mailboxes, but a sufficiently bizarre kind of email could slip through unblocked. The most obvious example I can come up with right now is the all-too-common graphics-heavy HTML email, but no doubt other examples will arise over time.

Hence, to build a really robust known-spam-blocker, you need a capability for bizarre-email-identification, or some other way of blocking those spams that would otherwise slip through the cracks. For a discussion of other spam-blocking techniques, please see a companion article.

4. Integrate nicely into your email system. Any kind of spam-blockers needs to play nicely with your email software. Right now, for example, Earthlink’s great spam-blocker only works with Earthlink’s pathetically slow webmail system, and hence I don’t really use it. Instead, I just download my mail and let my far inferior client-based spam blockers do what they can.

So, given that known-spam-blocking is the best technique, which vendor should you use? Actually, we’re still researching that, and will update this page when we have an answer. In the mean time, please click here for a companion article on other spam-blocking techniques, or here for other articles on internet technologies, or contact us if you’re interested in our technology and marketing consulting services.

For more information, please contact Curt Monash.

To reach Monash Information Services by phone, please call 978-266-1815.