Is your website up? Are you sure?

“What would you do for 25% more sales?”
It’s panicked gift-buying season, and I got mail this morning from Boutique Academia, part of their final push before Christmas.
Inbox__18_975_messages__26_unread_
They’re hoping for some Christmas sales in the next three days. They do make some lovely jewelry – ask Laura about her necklace some time – so I clicked on their mail.
Failed_to_open_page
That’s not good. I like Boutique Academia, and fixing email and dns problems is What We Do, so I took a look.
Safari isn’t quite as bad with not-exactly-truthful error messages as Internet Explorer, but I still don’t really trust it. Perhaps the problem is with the click-tracking domain in the email, rather than with boutiqueacademia.com? So I open the base page at http://boutiqueacademia.com, get redirected immediately to https://www.boutiqueacademia.com – which fails to load.
15542402_1501169219896451_6901276936993410491_n
OK, start with the basics. DNS.

~ ∙ dig www.boutiqueacaedmia.com
; <<>> DiG 9.8.3-P1 <<>> www.boutiqueacademia.com
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 19459
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.boutiqueacademia.com. IN A

That’s pretty authoritative (DNS joke … never mind). The response says “NXDOMAIN”, which is DNS-speak for “no such domain”. It means that there isn’t any DNS record for www.boutiqueacademia.com – no A record, nor any DNS record of any other type for that hostname. And it isn’t something that can be caused by a temporary network glitch – it means that the DNS server for the domain says authoritatively “there’s no DNS record”.
This is beginning to look familiar. Lets see what the authoritative DNS servers for boutiqueacademia.com have to say.

~$ dig +short boutiqueacademia.com ns
ns2.hostdns4u.com.
ns1.hostdns4u.com.
~$ dig @ns1.hostdns4u.com www.boutiqueacademia.com
; <<>> DiG 9.9.5-3ubuntu0.7-Ubuntu <<>> @ns1.hostdns4u.com www.boutiqueacademia.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4587
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; QUESTION SECTION:
;www.boutiqueacademia.com. IN A
;; ANSWER SECTION:
www.boutiqueacademia.com. 30 IN CNAME boutique-academia.myshopify.com.

There’s our problem. Boutique Academia are using hostdns4u.com to host the DNS for their domain – and hostdns4u.com are sending broken DNS responses. They’re responding with a CNAME to myshopify, the ecommerce site, but they’re also responding with the status “NXDOMAIN” – there’s no DNS record for this hostname.
That response violates the way DNS works. If it’s returning any matching DNS record – such as the CNAME – it should return a “NOERROR” status, not “NXDOMAIN”.
Where this gets interesting is when you ask what a DNS resolver will do with this invalid response. That’s not really defined, and different recursive resolvers will treat this response in different ways. Some will ignore the NXDOMAIN and return the CNAME record. Some will ignore the CNAME record and return NXDOMAIN.
That means that depending on which sort of DNS server your recipient is using everything may work perfectly, or any attempt to visit the website may give an error. If the DNS server you’re using is sloppy about NXDOMAINs you may never see the problem in your own web browser.
As far as I can tell this bug is specific to CNAME records, and exists at several DNS hosting companies.
The end result is that if you’re using an ecommerce or blog-hosting company that uses CNAMEs in your DNS zone to point to their servers (such as shopify or wordpress) and you’re  hosting your DNS with a company that has this bug then a significant fraction of the Internet will not be able to reach your website.
You’ll see their opens and clicks on your email. If you’re using pay-per-click advertising you’re paying for each customer. But they’ll never see your site.
I’m not sure what fraction of the Internet is using recursive resolvers that are strict about invalid CNAME responses, but I’d guess it’s at least 20%. If your website suffers from this issue then you could get 25% more sales by moving your DNS hosting to a company that doesn’t suffer from this issue.
(I have told Boutique Academia about this issue. Hostdns4u.com have a cPanel error page for their website, so I didn’t try to contact them.)

Related Posts

DNSBLs, wildcards and domain expiration

Last week the megarbl.net domain name expired. Normally this would have no affect on anyone, but their domain registrar put in a wildcard DNS entry. Because of how DNSBLs work, this had the effect of causing every IP to be listed on the blocklist. The domain is now active and the listings due to the DNS wildcard are removed.

Read More

HE.net DNS problems

Hurricane Electric had a significant outage of their authoritative DNS servers this morning, causing them to return valid responses with no results for all(?) queries. This will have caused delivery problems for any mail going to domains using HE.net DNS – which will include some of their colocation customers, as well as users of their free services – but also will have caused reverse DNS to fail for most servers hosted by Hurricane Electric worldwide, so if any of your mail is being sent from HE hosted machines you may have seen problems.
(We’re HE customers so we noticed. Still happy with them as a vendor.)

Read More

I can't click through if you don't exist

Recipients can’t click through if you don’t exist
A tale of misconfigured DNS wrecking someone’s campaign.
I got mail this morning from A Large Computer Supplier, asking me to fill in a survey about them. I had some feedback for them, mostly along the lines of “It’s been two decades since I bought anything other than rackmount servers from you, maybe I’m not a good advertising target for $200 consumer laptops?” so I clicked the link.
 
Failed_to_open_page
 
(I’ve replaced the real domain with survey.example.com in this post, to protect the innocent, but everything else is authentic).
That’s not good. The friendly error messages web browsers give sometimes hide the underlying problem, but that looks like a DNS problem. Did they do something stupid, like putting the wrong URL in the mail they sent?
 

Read More