The view from a blacklist operator

We run top-level DNS servers for several blacklists including the CBL, the blacklist of infected machines that the SpamHaus XBL is based on. We don’t run the CBL blacklist itself (so we aren’t the right people to contact about a CBL listing) we just run some of the DNS servers – but that means that we do get to see how many different ways people mess up their spam filter configurations.
This is what a valid CBL query looks like:

  • “14.23.177.10.cbl.abuseat.org”

It’s just the IP address being queried (10.177.23.14) with the numbers reversed, with “.cbl.abuseat.org” added on the end. Not rocket science.
Here’s a tiny sample of some of the invalid queries:

  • “70.46.6.10.abuseat.org”
  • “202.204.219.10cbl.abuseat.org”
  • “252.94.193.10.ns1-cbl.abuseat.org”
  • “255.190.244.10 cbl.abuseat.org”
  • “166.193.222.10#cbl.abuseat.org”
  • “214.6.224.10.*@cbl.abuseat.org”
  • “212.9.185.10.http://cbl.abuseat.org”
  • “76.207.80.10.bl.abuseat.org”
  • “185.124.73.10.cbb.abuseat.org”
  • “201.54.179.10.cbl-xbl.abuseat.org”
  • “54.191.254.10.opm.abuseat.org”
  • “181.4.133.10.sbl-xbl.abuseat.org”
  • “176.33.165.10.cbl.abuseat.orgcbl.abuseat.org”
  • “101.126.133.10.cbl.abuseat.org:Mail from %IP% refused by blackhole site cbl.abuseat.org”

Those are just 15 of about 1800 different misconfigurations I have on file, just for queries to the CBL. I’ve seen similar things at other domains I host, and I’ve heard of just the same sort of thing from other people who own domains that are similar in some way to a domain used by a blacklist. It’s not unusual.
What happens when someone misconfigures a blacklist lookup in this way? Because of the way DNS based blacklists work the response to any of these invalid queries will be “no, that IP address isn’t listed”. So all these people are attempting to use the CBL to filter out spam and haven’t noticed that it’s never actually stopped any email. And all the time they’re doing this, they’re hammering my DNS servers (and many other peoples) with millions of pointless queries every day.
What can the DNS server operators do about that? Because of the way DNS works, blocking the broken queries will actually increase the amount of traffic they have to deal with by several times. Contacting all the people making the queries and pointing out the problem would be a huge task, and even when I have tracked down contact information and notified people by email I’ve never had a response and the problem has never been fixed.
So the only remaining option is to make the misconfiguration more obvious to the user – by responding to the invalid queries with “yes, that IP address is listed” and hoping that causing them to reject all the mail sent to their users will encourage them to fix their configuration. I check my nameserver statistics every so often and add “poison” entries for the more obvious misconfigurations I find. I did that for a bunch of misconfigurations manually yesterday, which will probably cause a lot of domains to reject a bunch of email they didn’t want to this morning.
There are fairly simple ways to make sure you’re querying a real blacklist – pretty much all of the legitimate blacklists include the IP address “127.0.0.2” as a test entry. You can use that to check that a blacklist is live manually – if the blacklist domain is sbl.spamhaus.org then a dns lookup for “2.0.0.127.sbl.spamhaus.org” should return an answer (typically 127.0.0.2) while a dns lookup for “1.0.0.127.sbl.spamhaus.org” should return “not found” / “NXDOMAIN”. If either of those tests fails, the blacklist is broken in some way, and you shouldn’t use it.
The choice of 127.0.0.2 for the test entry wasn’t arbitrary: 127.0.0.2 is a “local” address that’s always available on machine, though it’s usually never used for anything. But you can use it – if you open a commandline on your mailserver you can run an SMTP transaction by hand (as I discussed yesterday) from 127.0.0.2 using “telnet -b 127.0.0.2 your.hostname 25” (on Linux-ish systems, anyway – some other telnets use “-s” instead of “-b”). That way you can see whether you’re really rejecting based on a blacklists, and what error you’re giving. (It would be nice if every blacklist also had another test entry in 127.* as well as 127.0.0.2, so you could check them individually, but they don’t. Hint to blacklist operators.).
It’s very easy for spam filter authors to check those test entries once a day for each of the blacklists they were configured to use, and to disable the ones that failed. If you’re a postmaster who uses blacklists as part of your spam filter (and you probably should) you should check with the people who provide the filter whether it makes those checks – and if it doesn’t, ask them to add them. That will protect you from misconfigurations, blacklists being shut down, blacklists being abandoned and bought up by domain squatters and all sorts of other things that can cause you to lose a lot of mail.

Related Posts

The secret to fixing delivery problems

There is a persistent belief among some senders that the technical part of sending email is the most important part of delivery. They think that by tweaking things around the edges, like changing their rate limiting and refining bounce handling, their email will magically end up in the inbox.
This is a gross misunderstanding of the reasons for bulk foldering and blocking by the ISPs. Yes, technical behaviour does count and senders will find it harder to deliver mail if they are doing something grossly wrong. In my experience, though, most technical issues are not sufficient to cause major delivery problems.
On the other hand, senders can do everything technically perfect, from rate limiting to bounce handling to handling feedback loops through authentication and offer wording and still have delivery problems. Why? Sending unwanted mail trumps technical perfection. If no one wants the email mail then there will be delivery problems.
Now, I’ve certainly dealt with clients who had some minor engagement issues and the bulk of their delivery problems were technical in nature. Fix the technical problems and make some adjustments to the email and mail gets to the inbox. But with senders who are sending unwanted email the only way to fix delivery problems is to figure out what recipients want and then send mail meeting those needs.
Persistent delivery problems cannot be fixed by tweaking technical settings.

Read More

Spamtraps

There is a lot of mythology surrounding spamtraps, what they are, what they mean, how they’re used and how they get on lists.
Spamtraps are very simply unused addresses that receive spam. They come from a number of places, but the most common spamtraps can be classified in a few ways.

Read More

It doesn't matter what you say

“What should we tell the ISP?” is a frequent question from my customers. The answer is pretty simple. It doesn’t usually matter what you tell the ISP. What matters are your actions.
If a sender is having delivery problems then the solution is not to call the ISP and talk to them about why the sender’s mail should not be delivered to the bulk folder. Instead, the solution is to evaluate the email and the address acquisition process and the list hygiene process. Identify where potential problems are and then resolve those problems.
Typically, the ISPs won’t need to be contacted. The changes to the email will register and delivery will improve. In some cases, particularly when there’s been some major mistake, contacting the ISP and explaining the mistake and what steps have been taken to stop the mistake from happening in the future may help resolve the issue faster. But if nothing has changed, then there’s no reason for the ISP to expect anything to change.
It doesn’t matter what you say. It matters what you do.

Read More