DNSBLs, wildcards and domain expiration

Last week the megarbl.net domain name expired. Normally this would have no affect on anyone, but their domain registrar put in a wildcard DNS entry. Because of how DNSBLs work, this had the effect of causing every IP to be listed on the blocklist. The domain is now active and the listings due to the DNS wildcard are removed.

How does a domain expiration lead to a DNSBL listing the whole internet?

Well, it’s kinda complicated and involves some of those grubby corners where different things that shouldn’t happen all occur and weird stuff happens.

But I thought there were rules and standards about this stuff.

Well, there are standards on the Internet. The IETF publishes documents “Request for Comments” (RFCs). These aren’t rules as much as they are standards. While many folks treat RFCs as rules, that’s not really what they’re for. All the RFCs do is tell operators how to do things if they want to communicate seamlessly with other entities.  Not every RFC is a standard; some are informational, some are experimental, some are drafts being actively developed and some are current practices.

So how does a domain expiration lead to listing all IPv4 space on a blocklist?

Well, there are a few technical things we need to understand before we can see how we get from domain expiration to blocklisting.
How DNSBLs work
First we need to talk about how DNSBLs work. DNSBLs publish a list of IP addresses. To query a DNSBL for an IP address you take the IP address, reverse it, add the DNSBL name onto the end and query for that domain.
Let’s say I want to look up  37.49.224.172 on the SBL.  I flip the IP address and add sbl.spamhaus.org to create the domain name 172.224.49.37.sbl.spamhaus.org. Then I do a DNS lookup on the domain name. If the IP is listed, then Spamhaus will return an A record / IP address. If the IP is not listed, then Spamhaus will return an NXDOMAIN and no A record.
All DNSBLs are run this way, so if you get any answer to a DNS query it means the IP is listed. And, yes, this is how blocklists are queried, even when you use a web form. It’s all DNS behind the scenes.
DNS wildcard records
Now that we know how DNSBLs are queried, we need to talk about DNS and wildcarding. Wildcarding is allowed in the DNS RFCs, but it’s complicated. So complicated entire RFCs have been written to try and explain the grubby corners. I’m certainly not going to try and explain it. If you want more details check out the Google support page, IAB commentary on wildcarding or the Wikipedia article.
The short version that works for our purpose is that a wildcard entry in DNS means that there is an IP for every domain queried.  You can make up any domain name and it will return an A record.
Wildcard records and registrars
The next thing we need to know is that some registration providers take unregistered domains and publish wildcard DNS records for them. Typically this is done to collect add revenue for parked or unused domains.

OK… so how does that lead to blocking?

When a domain expires and the registrar reclaims it and publishes a wildcard DNS record, then every query to the DNSBL will come back positive. The only negative DNSBL response is “NXDOMAIN” so the presence of any A record is a listing.
Expiration_Listing

That shouldn’t happen…

Well, no one is technically doing anything wrong here, except possibly the registrar by publishing wildcard record for expired domains. Or the domain owner by forgetting to renew their domain in a timely fashion. The good news is that it happens very rarely, typically once you lose a domain due to expiration you figure out how not to do that again. And not all registrars wildcard expired domains. This is just one of those things where no one is doing anything wrong or unusual, but when they all happen together unexpected things happen.
The good news is that, as of today, the domain is back and the false listings are removed.

Can a DNSBL stop this from happening?

There are number of ways a DNSBL can stop this from happening. The two big ones are not to let the domains expire and use a registrar that won’t wildcard parked domains. Other than that, though, they can’t really.

I’m a DNSBL user, how can I know this is happening?

Well run IPv4 DNSBLs should always list 127.0.0.2 and should never list 127.0.0.1. DNSBL users should query a DNSBL daily for these two addresses. If it is listing 127.0.0.1 or it is not listing 127.0.0.2 then it should be considered non functional and dropped from your filtering scheme. (RFC5782 and RFC6471, )

Related Posts

How many blocklists do we need?

There’s been a discussion on the mailop list about the number of different blocklists out there. There are discussions about whether we need so many lists, and how difficult the different lists make it to run a small mail system (80K or so users). This discussion wandered around a little bit, but started me thinking about how we got to a place where there are hundreds of different blocklists, and why we need them.
shield
There is a lot of history of blocklists, and it’s long, complicated and involves many strong and passionate personalities. Some of that history is quite personal to me. Not only do I remember email before spam, I was one of MAPS’ first few employees, albeit not handling listings. I’ve talked with folks creating lists, I’ve argued with folks running lists. For a while I was the voice behind a blocklist’s phone number.
The need, desire and demand for different lists has come up over the years. The answer is pretty simple: there are many different types of abuse. One list cannot effectively address all abusive traffic nor have policies that minimize false positives.
Lists need different policies and different delisting criteria. The SBL lists based on volume of email to addresses that are known to have not opted in to receive mail. The PBL lists IPs where the IP owner (usually an ISP) says that the IPs are not supposed to be sending mail by their policy. URIBL and SURBL list domains, not IPs. Some lists have delisting requirements, some let listees remove themselves.
The policies of listing and delisting are not one size fits all, nor should they be.
There are two widely used lists that have significantly different delisting policies: the SBL and the CBL.
The SBL focuses on IP addresses they believe are under the control of or supporting the services of spammers. They measure this by primarily relying on spamtraps, but they also accept forwarded mail from some trusted individuals. Getting delisted from the SBL means explaining to Spamhaus what steps were taken to stop the spam from coming. It’s a manual process with humans in the loop and can require significant business process changes for listees. (We’ve helped dozens of companies resolve SBL listings over the years, contact us if you need help.)
On the other hand, the CBL is a mostly automated list. It lists ources of mail that aren’t real mail servers sending real mail, but are sending a lot of stuff. As they describe it:

Read More

SPF: The rule of ten

Some mechanisms and modifiers (collectively, “terms”) cause DNS queries at the time of evaluation, and some do not. The following terms cause DNS queries: the “include”, “a”, “mx”, “ptr”, and “exists” mechanisms, and the “redirect” modifier. SPF implementations MUST limit the total number of those terms to 10 during SPF evaluation, to avoid unreasonable load on the DNS. If this limit is exceeded, the implementation MUST return “permerror”.

Read More

PTR Records

PTR records are easy to over look and they have a significant impact on your ability to deliver mail without them.  Some ISP and mailbox providers will reject mail from IP addresses that do not have a PTR record created. PTR records are a type of DNS record that resolves an IP address to a fully qualified domain name or FQDN.  The PTR records are also called Reverse DNS records. If you are sending mail on a shared IP address, you’ll want to check to make sure the PTR record is setup, however you most likely will not be able to change it.  If you are on a dedicated IP address or using a hosting provider like Rackspace or Amazon AWS, you’ll want to create or change the PTR records to reflect your domain name.
We usually think about DNS records resolving a domain name such as www.wordtothewise.com to an IP address.  A query for www.wordtothewise.com is sent to a DNS server and the server checks for a matching record and returns the IP address of 184.105.179.167.  The A record for www is stored within the zone file for wordtothewise.com.  PTR records are not stored within your domain zonefile, they are stored in a zonefile usually managed by your service provider or network provider.
Some service providers provide an interface where you can create the PTR record yourself, others require you to submit a support request to create or change the PTR record.
If you know what IP address you are sending mail from, use our web based DNS tool to check if you have a PTR record created.
http://tools.wordtothewise.com/dns
Checking for a PTR record for 184.105.179.167 returns
167.128-25.179.105.184.in-addr.arpa 3600 PTR webprod.wordtothewise.com.
If you received Response: NXDOMAIN (There is no record of any type for x.x.x.x.in-addr.arpa), this means you’re missing the PTR record and need to create one ASAP if you are sending mail from that IP address!

Read More