GFI/SORBS – should I use them?

steve
Industry
December 9, 2010

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.
Today I’m looking at how this information should affect your choice of spam filtering technology.

Should my spam filters use GFI/SORBS data?

Simply, NO. The quality of reputation data GFI provide is too false-positive prone to rely on in production, even as part of a scoring system.

After all, a false positive is far worse than a false negative, as far as RBL (or general filtering system) usability is concerned.@delivery_kitty

And the problem isn’t just false positives.

Because it takes a long time before a spamming IP address reliably appears on the blacklist, not much spam is stopped. SORBS appears completely unsuitable to the most common way of spamming, via botnets.abuse department at xs4all, a major EU ISP

Your ISP
If you receive mail via your ISP then you’re unlikely to have problems with SORBS blocking your mail, as very few successful ISPs will use it for blocking outright. If you’re at a smaller ISP then they may well be using spam filters such as SpamAssassin, with a dependency on GFI / SORBS data sources, though.
But it’s not worth contacting your ISP unless you find out mail is being bounced or put into a junk folder due to a SORBS listing, or if you can tell by looking at the headers of email you receive that it’s being scored against SORBS.
(If you’re concerned about use of third-party reputation sources by your ISP, you could ask them to provide – or, better, publish – a list of the data sources they use, so their customers can make well-informed decisions about their filtering.)
Your Mailserver
If you run your own inbound mailserver, make sure it is not configured to use any of the SORBS blacklists for blocking email. How to do that varies depending on the server, but for commonly used linux mailservers grepping the configuration files for the string “sorbs” is probably a good place to check.
(There are some great blacklists, with very low false positive rates, to consider using instead – for IP based reputation: spamhaus zen, spamcop, cbl – and for URL reputation: spamhaus dbl, uribl, surbl)
SpamAssassin
SpamAssassin is a widely used server-side score based spam filter. Unfortunately it seems to ship with SORBS blacklists turned on “out of the box”.
I believe that adding the following to /etc/spamassassin/local.cf will disable it – I could be wrong, and would appreciate feedback from any SpamAssassin experts out there.

score RCVD_IN_SORBS_BLOCK 0
score RCVD_IN_SORBS_DUL 0
score RCVD_IN_SORBS_HTTP 0
score RCVD_IN_SORBS_MISC 0
score RCVD_IN_SORBS_SMTP 0
score RCVD_IN_SORBS_SOCKS 0
score RCVD_IN_SORBS_WEB 0
score RCVD_IN_SORBS_ZOMBIE 0

The default SpamAssassin scores are pretty low, so it doesn’t pay that much attention to SORBS – but that a spam filter as influential as SpamAssassin uses such a poor source of data at all is a bit of a problem. Hopefully the SpamAssassin developers will look at the issue for a future release.
Commercial Products
If you’re using a commercial spam filter, check where they’re getting their reputation data from. If you have an existing commercial filter that can use external blacklists, make sure it’s use of SORBS is disabled.
If you’re considering purchasing a new commercial spam filter, there are two things you need to consider. First, if the filter supports using SORBS or other GFI-derived reputation data make sure that can be disabled. Second, if you’re considering a commercial product that uses SORBS or GFI data out of the box, despite the multi-year history of false positives and other problems, think about how solid their other product engineering decisions might be.

I for one will not be considering any products from GFI. I had budgeted and received approval for $20K worth of GFI NSM next year. I will not be making that purchase after this latest episode with SORBS.Skyhawk

Outsourced Services
Outsourced spam filtering services can be very opaque about what approaches they use to decide whether or not email is spam, and will often hide their use of external reputation services.
Some of them are more open than others. Proofpoint posted SORBS DUHL DNS Block List Causing Widespread Email Deliverability Issues Once Again (note that GFI told Proofpoint the problem was fixed on Nov 30th, which we know isn’t true), but it’s rare that a SaaS provider will be that open about how a problem is caused by their reliance on a third-party service. Kudos to Proofpoint for their openness (though they should look elsewhere for reputation data).
Edit: Proofpoint have clarified that they were discussing the problems some of their customers had sending email due to false SORBS listings – not that they were using any data from GFI themselves. Sorry, guys. So if you’re looking for a filtering appliance or outsourced service that’s GFI/SORBS-free (and also quite a nice product), Proofpoint is worth a look.
If your SaaS or outsourced spam filter provider has a clear statement in their product description or contract which third-party data sources they use, then you have the information you need. If not, you should probably contact your support representative and find out whether they use SORBS or not. If they decline to make any statement on it, assume the worst.

In the case of SORBS, this is (at least) the second major misclassification issue we’ve observed in the last 90 days. Email administrators who currently rely on SORBS should be aware of these issues and take action as necessary.Proofpoint

There’s nothing wrong with an outsourced provider using reputable third-party services but if they’re relying on poor quality data sources you may find mail to you being bounced for no good reason, at any time. If that’s the situation you’d be well advised to consider looking at alternative filtering providers.

And Finally

There’s a lot more that could be said, but I’m sure you’re interested in seeing some non-GFI/SORBS content on this blog (and there’s a limit to the amount of technical and business analysis I really want to do for someone other than a paying customer).
Laura will probably revisit the subject soon, going into some more detail about the policy problems that I just touched lightly on and looking more generally about what other companies can learn. And I know several other industry bloggers are planning on discussing GFI and SORBS in the next week or two.
I’ll be gathering links and some other information, including a PDF version of this series of articles suitable for mailing out, at https://wordtothewise.com/sorbs/ over the next day or two.

GFI/SORBS considered harmful, part 2

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
Yesterday I talked about GFI responsiveness to queries and delisting requests about SORBS listings. Today I’m going to look at data accuracy.
The two issues are tightly intertwined – a blacklist that isn’t responsive to reports of false positive listings will end up with a lot of stale or inaccurate data, and a blacklist that has many false positives will likely be overwhelmed with complaints and delisting requests, and won’t be able to respond to them – leading to a spiral of dissatisfaction and inaccurate data feeding off each other.

Guide to resolving ISP issues

I often get a chuckle out of watching some people, who are normally on the blocking end of the delivery equation, struggle through their own blocking issues. A recent situation came up on a mailing list where someone who has very vehement opinions about how to approach her particular blocklist for delisting and that the lists policies are immutable. The company she works for is having some delivery issues and she’s looking for a contact to resolve the issues.
While digging through my blog posts to see if there was any help I could provide, I realized I don’t have a guide to resolving blocking issues at ISPs. Much of the troubleshooting can be done without ever contacting the ISPs or the blocklists.
Identify the issue.
There are a number of techniques that ISPs use to protect their users from malicious or problematic mail, from rate-liming incoming mail, putting mail in the bulk folder, or blocking specific IP addresses. Step one to resolving any delivery problem is to identify what is happening to the mail. In order to resolve the issue, you have to know what the issue is.
All too often, the description of a delivery problem is: My mail isn’t getting delivered. But that isn’t very clear as to what the actual problem is. Are you being temp failed? Is mail being blocked? Is mail going to the bulk folder? Is this something affecting just you or is it a widespread problem?
Troubleshoot your side.
Collect as much data about the problem as you can. Dig through logs and get copies of any rejection messages. Follow any URLs that are present in the bounce messages. Try sending a bare bones email to yourself at that ISP with just URLs, is it still blocked? What if you send from a different IP, does the same thing happen?
There is a lot of troubleshooting a sender can do without having to contact an ISP, and the information can lead to resolution that doesn’t involve having to contact the ISP. Also, many current ISP blocks are dynamic, they come up and go down without any human intervention. Those blocks that require contact to get them resolved have clear instructions in the bounce message.
Fix your stuff.
Whether it’s a reputation issue or a minor technical issue, fix the problem on your end. Just moving IP addresses or changing a URL isn’t a sustainable fix. There is a reason mail is being blocked or filtered and if you don’t fix that issue, the blocks are just going to come back. After you do fix your stuff, expect to see changes in a few days or a week. The ISP filters are generally quite responsive to sender improvements so if you’ve fixed the stuff you should see changes pretty quickly. Expect unblocking or filtering to take a little longer than the block was in place.
If you can’t figure out what the problem is, hire a consultant. Here at Word to the Wise we can often quickly identify a problem and provide a path to resolution. Sometimes the problem isn’t even the ISPs, we’ve had multiple cases where our clients were using custom software and their software wasn’t SMTP compliant and we were able to identify the problem and get their mail working again. There are a host of other independent consultants out there that can also help you identify and resolve blocking problems.
Contact the ISPs.
If there is a hard block or after fixing what you think the underlying problem is, you’ll have to contact the ISP. Many ISPs provide self service websites and contact forms to facilitate this process. Generally, though, most issues aren’t going to require contact.

Content based filtering

A spam filter looks at many things when it’s deciding whether or not to deliver a message to the recipients inbox, usually divided into two broad categories – the behaviour of the sender and the content of the message.
When we talk about sender behaviour we’ll often dive headfirst into the technical details of how that’s monitored and tracked – history of mail from the same IP address, SPF records, good reverse DNS, send rates and ramping, polite SMTP level behaviour, DKIM and domain-based reputation and so on. If all of those are OK and the mail still doesn’t get delivered then you might throw up your hands, fall back on “it’s content-based filtering” and not leave it at that.
There’s just as much detail and scope for diagnosis in content-based filtering, though, it’s just a bit more complex, so some delivery folks tend to gloss over it. If you’re sending mail that people want to receive, you’re sure you’re sending the mail technically correctly and you have a decent reputation as a sender then it’s time to look at the content.
You want your mail to look just like wanted mail from reputable, competent senders and to look different to unwanted mail, viruses, phishing emails, botnet spoor and so on. And not just to mechanical spam filters – if a postmaster looks at your email, you want it to look clean, honest and competently put together to them too.
Some of the distinctive content differences between wanted and unwanted email are due to the content as written by the sender, some of them are due to senders of unwanted email trying to hide their identity or their content, but many of them are due to the different quality software used to send each sort of mail. Mail clients used by individuals, and content composition software used by high quality ESPs tends to be well written and complies with both the email and MIME RFCs, and the unwritten best common practices for email composition. The software used by spammers, botnets, viruses and low quality ESPs tends not to do so well.
Here’s a (partial) list of some of the things to consider: