October 15, 2003 - 12:00
GFI today released a white paper to highlight the latest and most effective method of blocking spam: Bayesian filtering. Describing this powerful new technology in detail, the paper outlines how network administrators can achieve a spam detection rate of over 98% through Bayesian filtering at the mail server or gateway level. The document can be viewed at http://www.gfi.com/whitepapers/why-bayesian-filtering.pdf.
Why traditional anti-spam methods are no longer enough
As GFI’s white paper explains, the techniques currently used by anti-spam software - such as blacklist checking, databases of known spam and keyword checking - are static, making it fairly easy for spammers to evade such filters simply by tweaking their message a little. These technologies are far from obsolete, but they cannot be used as effectively as needed if not combined with a new adaptive technique that remains familiar with spammers' tactics as they change over time. GFI’s white paper shows how the answer lies in Bayesian mathematics, which can be applied to the spam problem, resulting in an adaptive, ‘artificial intelligence’ technique that is much harder for spammers to circumvent.
“We believe Bayesian filtering is the way ahead in combating spam," said Nick Galea, GFI CEO. “The Bayesian approach is the best way to tackle spam once and for all, as it overcomes the problems posed by more static technologies while also being able to adapt to the particular organization that it is protecting from spam. A recent BBC report, for example, said that spam detection rates of over 99.7% can be achieved through Bayesian filtering with a very low number of false positives. This is the kind of anti-spam solution that enterprises are seeking today.”
How the Bayesian spam filter works
Bayesian filtering is based on the principle that most events are dependent and that the probability of an event occurring in the future can be inferred from previous occurrences of that event. This same technique can be used to classify spam. If a piece of text occurs often in spam but not in legitimate mail, then the next time that same text is encountered in a new email, it would be reasonable to assume that this email is probably spam.
Custom organization-based filtering
Before mail can be filtered using this method, the user must generate a tailor-made history for each word or token (such as the $ sign, IP addresses and domains, and so on) that is specific to the company being protected. A probability value is assigned to each word or token, based on calculations that take into account how often that word occurs in spam as opposed to legitimate mail. Once the word probabilities have been calculated, the filter is ready for use. GFI’s white paper provides more detailed information about this process, highlighting that this analysis is performed on the company's mail, and is therefore tailored to that particular company.
For example, if using a general anti-spam rule set, a financial institution that legitimately uses the word "mortgage" in scores of daily email messages would get many false positives. The Bayesian filter, on the other hand, takes note of the company's valid outbound mail and would recognize "mortgage" as being frequently used in legitimate messages. It therefore has a much better spam detection rate and a far lower false positive rate. Additionally, the Bayesian filter is constantly updated based on new spam and valid emails; its performance therefore improves over time and adapts to changes in spam tactics and/or changes in the kind of emails written by users within the organization.
In a nutshell, Bayesian filtering offers the following advantages in the battle against spam:
- Looks at the whole message
- Adapts itself over time
- Is sensitive/adapts to the company/user
- Multilingual and international
- Uses statistical intelligence
- Hard to trick.
Bayesian protection at mail server/gateway level
GFI MailEssentials for Exchange/SMTP offers spam protection at server level and eliminates the need to install and update anti-spam software on each desktop. GFI MailEssentials offers a fast set-up and a high spam detection rate using Bayesian analysis and other methods. GFI MailEssentials also adds key tools to the mail server such as disclaimers, mail archiving and monitoring, reporting, and more. Pricing starts from as little as US$275 for 10 users.
More information about GFI MailEssentials and a trial version are available at: http://www.gfi.com/mes/.
GFI is a leading software developer that provides a single source for network administrators to address their network security, content security and messaging needs. With award-winning technology, an aggressive pricing strategy and a strong focus on small-to-medium sized businesses, GFI is able to satisfy the need for business continuity and productivity encountered by organizations on a global scale. GFI has offices in the US, Malta, UK, Hong Kong and Australia which support more than 200,000 installations worldwide. GFI is a channel-focused company with over 10,000 partners worldwide. GFI is a Microsoft Gold Certified Partner. More information about GFI can be found at http://www.gfi.com.