Wildcards and DKIM and DMARC, oh my!

If you’re an ESP with small customers you may have looked at the recent Google / Yahoo requirements around DMARC-style alignment for authentication and panicked a bit.

Don’t impersonate Gmail From: headers. Gmail will begin using a DMARC quarantine enforcement policy, and impersonating Gmail From: headers might impact your email delivery.

…

For direct mail, the domain in the sender’s From: header must be aligned with either the SPF domain or the DKIM domain. This is required to pass DMARC alignment.

So everyone who’s using their gmail address to send bulk mail is going to have to stop doing that within the next few months if they still want their mail to be delivered.

For any ESP customer that already has, or can be convinced to buy, a domain for their web presence maybe they can be persuaded to switch to using that – though even if they can, onboarding 100,000 technically naive users in three months seems like it could be a nightmare, even if you already have excellent onboarding automation in place and plenty of spare customer support bandwidth.

But what about all the other users, who don’t have a domain already or who can’t manage to bring that domain to their ESP for authentication – they may not even have access to their DNS zone, it may all be managed by their web host?

As an ESP we could just send all those customers emails from our own domain, or a dedicated domain just for those customers (halp-i-haz-no-domain.com is available for registration; you’re welcome). But that’s obviously a terrible idea from a domain reputation issue, quite apart from our customers losing any vestige of branding.

How about subdomain-per-customer? All our users probably have a login username already, it’s probably unique and branded. So our ESP, SausageMail, can set up subdomains of wurst.com for each customer. If we default that to <username>.wurst.com we can deploy that without any cooperation from each customer, and we can offer to change the string we use there if they want to.

Tasty sausages, and for no narratively justified reason, a leek.
Tasty sausages from our butcher

What do we need for each customer subdomain? We want to use it in the RFC 5322 From: header, and we want either DKIM or SPF to align with it. That means that we’d like an MX record for it, whether it points at a mailserver that forwards it to the customer, or an autoresponder or a blackhole. It’d be nice to have DMARC p=none set up for it too.

For DMARC-style alignment there’s no need for the authentication domain to match the domain in the 5322 From: exactly – it just needs to share a common organizational domain. That’ll typically mean that the authentication domain will be the parent domain (wurst.com) or a sibling subdomain to that in the 5322 From: (e.g. authentication.wurst.com). That means that we can avoid using a per-customer domain in the return path for SPF authentication, and avoid having to publish a customer-specific DKIM public key.

That’s great, because while wildcard MX records are simple and widely supported (they’re the main thing wildcard DNS records have been used for historically) wildcard TXT records risk tickling obscure bugs (e.g. wildcard TXT records mean that if an ISP asks for a DKIM public key they’ll get that key, and also DMARC records and vice-versa).

For the DMARC policy record itself we can take advantage of the way a DMARC record set for the organizational domain (wurst.com) will apply to all subdomains by default.

So you could do this:

*.wurst.com                 MX 10 reply-handler.sausagemail.com
bounces.wurst.com           MX 10 bounce-handler.sausagemail.com
bounces.wurst.com           TXT "v=spf1 include:_spf.sausagemail.com"
a._domainkey.pool.wurst.com TXT "v=DKIM1 p=..."
b._domainkey.pool.wurst.com TXT "v=DKIM1 p=..."
_dmarc.wurst.com            TXT "v=DMARC1 p=none ..."

(This isn’t a validly formatted zone file – it’s missing trailing periods, TTLs and classes – so don’t just paste it into your nameserver.)

Because of how DNS wildcards work MX queries for bounces.wurst.com will point at your bounce handler, while MX queries for anything_else.wurst.com will point at your reply handler.

Then for every mail you send for this sort of customer you use a From: header that looks like this …

From: "Customer's Pretty Name" <userid@userid.wurst.com>

… and you DKIM sign it using d=pool.wurst.com and you use bounces.wurst.com as the return path. Every mail will be sent from a valid, customer-specific, email address. It will have valid SPF authentication, and be validly DKIM signed. Both DKIM and SPF authentication domains will align with the 5322 From: header. And it’ll have a valid DMARC p=none.

If your existing customer authentication infrastructure is based on CNAMEs rather than TXT records (and it probably is) then you can set up DNS more like this:

*.wurst.com                 MX 10 reply-handler.sausagemail.com
bounces.wurst.com           MX 10 bounce-handler.sausagemail.com
bounces.wurst.com           TXT "v=spf1 include:_spf.sausagemail.com"
ssg1._domainkey.pool.wurst.com CNAME k1._domainkey.sausagemail.com
ssg2._domainkey.pool.wurst.com CNAME k2._domainkey.sausagemail.com
_dmarc.wurst.com               TXT "v=DMARC1 p=none ..."

Is this perfect? No. You’re sharing the same authentication domain across all of these customers which means that they will, to some degree, share domain reputation as well as their IP reputation. Not entirely, as large providers are very good at distinguishing mailstreams from different customer even if they share authentication domains. In a perfect world you’d use giant generated zone files, or a “stunt” DNS server to use customer_id.wurst.com as the authentication domain. But this isn’t a perfect world, and using a shared authentication domain for these customers is probably good enough, and will allow them to comply with mailbox provider requirements – and to do so in a way that seems to follow the spirit of the requirements, not just the letter.

I wrote about the better approach, using a stunt DNS server to synthesize all the DNS records needed, here.

This isn’t that scary to do, and can probably be implemented with only slight changes to your existing infrastructure. You’ve got about three months to go.

Related Posts

A brief history of TXT Records

txt
When the Domain Name System was designed thirty years ago the concept behind it was pretty simple. It’s mostly just a distributed database that lets you map hostname / query-type pairs to values.
If you want to know the IP address of cnn.com, you look up {cnn.com, A} and get back a couple of IP addresses. If you want to know where to send mail for aol.com users, you look up {aol.com, MX} and you get a set of four hostname / preference pairs back. If you want to know the hostname for the IP address 206.190.36.45 you look up {45.36.190.206.in-addr.arpa, PTR} and get a hostname back.
There’s a well-defined meaning to each of those query types  – A is for IP addresses, MX is for mailservers, PTR is for hostnames – and that was always the intent for how DNS should work.
When DNS was first standardized, though, there was one query type that didn’t really have any semantic meaning:

Read More

What to expect in 2016

WttWColorEye_forBlogI don’t always do predictions posts, even though they’re  popular. Most years I skip them because I don’t see major changes in the email space. And, I’m not the type to just write a prediction post just to post a prediction.
This year, though, I do see changes for everyone in the email space. Most of them center on finally having to deal with the technical debt that’s been accumulating over the past few years. I see ISPs and ESPs spending a lot of development effort to cope with the ongoing evolution authentication requirements.
When people started seriously looking at how to authenticate email, the first goal was getting organizations to implement the protocols. This was a practical concession; in order for a new protocol to be used it needs to be widely implemented. Phase one of authenticating email was simply about publishing protocols and getting organizations to use them.
During phase one, the organization that authenticated a mail hasn’t been important. In fact, the SPF spec almost guarantees that the ESP domain is the authenticated domain. In DKIM, the spec says any domain could sign as long as they could publish a public key in that domain’s domainkeys record.
ESPs took full advantage of this and lowered their own development overhead by taking most of the authentication responsibility on themselves. Their domains were in the 5321.from and they published the SPF records. Domains they control were in the d= and they generated and published the DKIM keys. Mail was authenticated without ESP customers having to do much.
We’ve hit the end of phase one. Most of the major players in the email space are authenticating outbound email. Many of the major players are checking authentication on the inbound. Phase one was a success.
We’re now entering phase two, and that changes thing. In phase two, SPF and DKIM are used as the foundation for user visible authentication. Neither SPF nor DKIM were designed to be user visible protocols. To understand what they’re authenticating you have to understand SMTP and email. Even now there are days when I begin talking about one of them and have to take a step back and think hard about what is being authenticated. And I use these things every day!
DMARC is the first of these end user visible protocols built on SPF and DKIM. It uses the established and widespread authentication to validate the user visible from address. This authentication requires that the d= value or the 5321.from address belong belong to the same domain in the visible from address. While you can pick whether the alignment between the visible from and the authentication is “strict” or “relaxed” you have no choice about the alignment.
Prior to DMARC no one really paid much attention to the domain doing the authentication. Authentication was a yes or a no question. If the answer was yes, then receivers could use the authenticated domain to build a reputation. But they weren’t really checking much in the way of who was doing the authentication.
In the push to deploy authentication, ESPs assumed the responsibility for authentication deployed ESPs took the responsibility and did most of the work. For many or most customers, authentication was as simple as clicking a checkbox during deployment. Some ESPs do currently let customers authenticate the mail themselves, but there’s enough overhead in getting that deployed that they often charged extra to cover the costs.
DMARC is rapidly becoming an expectation or even a full on requirement for inbox delivery. In order to authenticate with DMARC, the authenticating domain must be in the same domain space as the visible from. If senders want to use their own domain in the visible from, DNS records have to be present in that domain space. Whether it’s a SPF TXT record or a domainkeys record the email sender customer needs to publish the correct information in DNS. Even now, if you try to authenticate with DKIM through google apps, they require you to publish DNS records.
ESPs aren’t in a situation where they can effectively manage authentication alignment for all their customers. Hosting companies are in even worse shape when it comes to letting customers authenticate email. Developers are facing the fact they need to go back and rework their authentication code. Businesses are facing the fact they need to change their processes so customers can authenticate with DMARC.
It’s not just the infrastructure providers that are facing challenges with authentication. Senders are going to discover they can no longer hand authentication off to their ESPs and not worry about it. They’re going to have to get DNS records published by their own staff.
Getting DNS updates through some big companies is sometimes more difficult than it should be. I had one client a few years ago where getting rDNS changed to something non-generic took over a month. From an IT standpoint, changing DNS should require approvals and proper channels. Marketers may find this new process challenging.
And, if organizations want to publish reject policies for their domains, then they will have to publish records for every outside provider they use. Some of those providers can’t support DMARC alignment right now.
In 2016 a lot of companies will discover their current infrastructure can’t cope with modern authentication requirements. A lot of effort, both in terms of product development and software development, will need to be spent to meet current needs. This means a lot of user visible features will be displaced while the technical debt is paid.
These changes will improve the security and safety of email for everyone. It won’t be very user visible, which will give the impression this was a slow year for email development. Don’t let that fool you, this will be a pivotal year in email.

Read More

Ask Laura: Can you help me understand no auth / no entry?

AskLaura_Heading3
Dear Laura,
I’m a little confused by the term “no auth / no entry”. Gmail and other major receivers seem to be moving towards requiring authentication before they’ll even consider delivery.
Does this just mean SPF and DKIM, or does this mean the much more stringent DMARC, as well?
Thanks,
No Shirt, No Shoes, No What Now?

Read More