DNS Flag Day

There are quite a lot of broken DNS servers out there. I’m sure that’s no surprise to you, but some of them might be yours. And you might not notice that until your domains stop working early next year.

DNS is quite an old protocol, and when it was originally specified there wasn’t really a good way to extend the protocol to add new features. That was fixed about 19 years ago when Extension Mechanisms for DNS (EDNS0) was specified, and solidly standardized in RFC 6891 in 2013. It added a backwards compatible way for a DNS client to ask “Hey! Do you support new features?” and for servers to include as part of their response “Yes! Yes I do!”.

That’s incredibly useful, and critical for extending the DNS to support new features (such as DNSSEC, or support for larger replies). And yet some authoritative DNS servers not only don’t support it, they misbehave when they’re asked if they support it. It’s been the case forever that DNS servers should just ignore (some sorts of) fields in requests if they don’t understand them. So when you send a request that includes an EDNS0 “Do you support new features?” field to a DNS server that doesn’t understand EDNS0 it should return a regular DNS response. Some (broken) nameservers don’t do that – instead they drop the request on the floor and don’t respond (or, even worse, crash). Eventually the recursive resolver will give up on the request.

(DNS servers broken in this way aren’t that rare in 2018 – just last week I had to add code to a DNS library I use so that it didn’t crash when it saw EDNS0 requests.)

Right now most recursive resolvers will see a timeout for a request that included EDNS0 and decide “Maybe it only failed because the remote server has buggy EDNS0 handling”. They’ll retry the request without EDNS0 and get an answer. This workaround means that the DNS will resolve eventually, after five or ten seconds of delay. Not good, but the web page will open or the mail will be delivered eventually.

But it’s a horrible workaround, and the developers of the most widely used recursive resolvers are done with this silliness. As of February 1st next year they’re not going to do it any more. If your DNS server is broken with respect to EDNS0 your hostnames won’t resolve via a large fraction of recursive resolvers. Your webpages won’t load, mail you send won’t have any SPF, DKIM or DMARC information or even any reverse DNS. Lots of things will break in a very visible way.

You can check whether your DNS server is broken or not, and get a bunch more technical details at dnsflagday.net.

Related Posts

TLS certificates and CAA records

Transport Layer Security (TLS) is what gives you the little padlock in your browser bar. Some people still call it SSL, but TLS has been around for 18 years –  it’s time to move on.
TLS provides two things. One is encryption of traffic as it goes across the wire, the other is a cryptographic proof that you’re talking to the domain you think you’re talking to.
The second bit is important, as if you can’t prove you’re talking to, for example, your bank you could really be talking to a malicious third party who has convinced your browser to talk to their server instead of your bank which makes the encryption of the traffic much less useful. They could even act as a man-in-the-middle and pass your traffic through to your bank, so that you wouldn’t notice anything wrong.

When your browser connects to a website over TLS it, as part of setting up the connection, fetches a “TLS certificate” from the server. That certificate includes the hostname of the server, so the browser can be sure that it’s talking to the server it thinks it is.
How does the browser know to trust the certificate, though? There’s not really a great way to do that, yet. There’s a protocol called DANE that stores information in DNS to validate the certificate, much the same as we do with DKIM. It’s a promising approach, but not widely supported.
What we have today are “Certificate Authorities” (CAs). These are companies that will confirm that you own a domain, issue you a certificate for that domain where they vouch for it’s authenticity (and usually charge you for the privilege). Anyone can set themselves up as a CA (really – it’s pretty trivial, and you can download scripts to do the hard stuff), but web browsers keep a list of “trusted” CAs, and only certificates from those authorities count. Checking my mac, I see 169 trusted root certificate authorities in the pre-installed list. Many of those root certificates “cross-sign” with other certificate authorities, so the actual number of companies who are trusted to issue TLS Certificates is much, much higher.
If any of those trusted CAs issue a certificate for your domain name to someone, they can pretend to be you, secure connection padlock and all.
Some of those trusted CAs are trustworthy, honest and competent. Others aren’t. If a CA is persistently, provably dishonest enough then they may, eventually, be removed from the list of trusted Certificate Authorities, as StartCom and WoSign were last year. More often, they don’t: Trustwave, MCS Holdings/CNNIC, ANSSI, National Informatics Center of India (who are currently operating a large spam operation, so …).
In 2011 attackers compromised a Dutch CA, DigiNotar, and issued themselves TLS Certificates for over 500 high-profile domains – Skype, Mozilla, Microsoft, Gmail, … – and used them as part of man-in-the-middle attacks to compromise hundreds of thousands of users in Iran. Coverage at the time blamed it on “DigiNotar’s shocking ineptness“.
In 2015 Symantec/Thawte issued 30,000 certificates without authorization of the domain owners, and even when they issued extended validation (“green bar”) certificates for “google.com” they weren’t removed from the trusted list.
So many CAs are incompetent, many are dishonest, and any of them can issue a certificate for your domain. Even if you choose to use a competent, reputable CA – something that’s not trivial in itself – that doesn’t stop an attacker getting certificates for your domain from somewhere else.
This is where CAA DNS records come in. They’re really simple and easy to explain, with no fancy crypto needed to set them up. If I publish this DNS record …

Read More

DNSBLs, wildcards and domain expiration

Last week the megarbl.net domain name expired. Normally this would have no affect on anyone, but their domain registrar put in a wildcard DNS entry. Because of how DNSBLs work, this had the effect of causing every IP to be listed on the blocklist. The domain is now active and the listings due to the DNS wildcard are removed.

Read More

The Internet is hard.

There are so many things that need to happen to make the Internet work. DNS entries need to be right. MXs need to be set up. Web servers need to be configured. And, let’s be honest, anyone who has ever run their own services on the Internet has flubbed a configuration.
We don’t think about it, because most of the time the configurations are handled by scripts and they do things right. But at some point someone needs to type in something and there’s a risk it will go horribly wrong. I’ve been digging into domain data for a client of mine today. I think I’m going cross-eyed over it. But I have found so many weird things that just mean someone isn’t paying attention to what they’re doing.
Like the domain that has a MX record that says:
nullmx
 
I’m pretty sure the intention of the domain owner is to publish a null MX. But they added an extraneous “0” in there and ended up publishing something really weird. Even worse, the MTA that this client is using is listing this as a “delivered” email. I’m pretty sure that mail to that domain never left the MTA.
I’ve found horribly typoed MX domains for popular spam filters. I’ve found domains that have invalid characters in them. I’ve found domains that are totally a mess.
The vast majority of us have some story or other of the time we really broke things by accident. Like the time a very large ISP deleted their MX records. Or when a different ISP changed their internal forwarding and broke SPF authentication for everyone mailing that domain. Or when another ISP accidentally blocked every IP beginning with 6.
Sometimes I’m amazed that the Internet ever works. No matter how big it gets, there are actual people writing actual code and configurations. The number of things that have to happen to get packets from A to B is pretty impressive. We rarely ever notice the breakages, the people who run things are really good at their jobs. But sometimes poking in the grotty corners reminds me how easy it is to break things. It’s sometimes a wonder things actually work.
 

Read More