Ask Laura: Do I have to publish DMARC?

AskLaura_Heading3
 

Dear Laura,
I heard recently that both Gmail and Yahoo will require DMARC authentication in early 2016 or images will be automatically blocked.
Is that correct? And if so, do you know when they will be requiring DMARC?
A DMARC-Overwhelmed Admin

Dear Overwhelmed,
There are three things going on here, all of which are related to DMARC but are very different in how it affects mail delivery.
1) What policies the freemail domains publish in DMARC records.
Currently AOL, and many of the Yahoo properties publish a p=reject. This affects small senders who may use @aol.com or @yahoo.com email addresses in their bulk mail. Gmail has announced they will change their DMARC record for @gmail.com to p=reject sometime in the next year or so. This will affect all the senders who are using @gmail.com as their From address in bulk email.
2) How domains apply the policies published in DMARC records.
In other words, what does the receiver do with an incoming mail that fails DMARC authentication AND there is an active p=reject policy. Many of the large freemail providers are not strictly following “p=reject” but they are respecting it for the most part. I have seen cases where the p=reject policy is ignored for certain IP addresses. Typically these addresses are known forwarders like ieee.org and the like. This is good for everyone as it means less email failures due to a p=reject policy, but still provides domain owners the ability to make policy statements.
3) How domains handle incoming email that does not align and/or does not have a DMARC policy.
One of the things I’ve seen recently is that some of the freemail providers are checking for a DMARC-style alignment even in the absence of any published DMARC record. I’m also seeing some indications that mail that does not align is more likely to go to the bulk folder.
Example: a client was recently doing some testing. Exact same content message sent with a from address that aligned and a from address at Gmail.

% Inbox Delivery

Receiving DomainFrom: AlignedFrom: Unaligned
AOL100%5%
Gmail100%90%
Hotmail95%0%
Yahoo100%0%

Same content, same source IP, same return path, same DKIM signature, the ONLY difference was the 5322.from address. 95% bulk at AOL, 100% bulk at Hotmail and Yahoo. Ironically, it was Gmail that had no problem with a gmail.com address being used for unaligned email.
Takeaways:
1) DMARC Alignment, even in the absence of a published DMARC record, is going to start being a factor in deliverability. This is going to cause problems for a lot of senders as many ESPs don’t have a good way to send aligned mail. This is going to cause problems at ESPs because they’re going to have to make infrastructure changes to support alignment.
2) Gmail will, sometime in the next year or so, be publishing a p=reject record. This means @gmail.com addresses will be poorly delivered from ESP and private infrastructure – much like AOL and Yahoo addresses are now. Some of this does seem to be already happening at freemail providers other than Google, based on my client’s test I shared above.
3) Gmail will, sometime in the future, start delivering unauthenticated email to the bulk folder. They’re being a bit vague about whether or not this means DMARC authenticated. I think in the short and medium term this just means either SPF or DKIM authenticated. I do expect that they’ll be looking at alignment, even if senders aren’t publishing a DMARC record and that will influence delivery. Unaligned mail may get a small negative and aligned mail may get a bigger positive during delivery scoring. For a long time, Gmail was doing “best guess SPF” – we didn’t publish a SPF record for a long time, and our mail always passed SPF at Gmail because we were sending from our MX. I expect they’re going to do something similar with DMARC – if either SPF or DKIM align and pass, then the mail will be considered to pass DMARC.
Bottom line: the days of sending unauthenticated email are over. The days of easily sending unaligned bulk email are rapidly coming to a close.

Confused about delivery in general? Trying to keep up on changing policies and terminology? Need some Email 101 basics? This is the place to ask. We can’t answer specific questions about your server configuration or look at your message structure for the column (please get in touch if you’d like our help with more technical or forensic investigations!), but we’d love to answer your questions about how email works, trends in the industry, or the joys and challenges of cohabiting with felines.
Your pal,
Laura

Foundation: A toolkit for designing responsive emails
Prepping for EEC

Related Posts

A brief history of TXT Records

txt
When the Domain Name System was designed thirty years ago the concept behind it was pretty simple. It’s mostly just a distributed database that lets you map hostname / query-type pairs to values.
If you want to know the IP address of cnn.com, you look up {cnn.com, A} and get back a couple of IP addresses. If you want to know where to send mail for aol.com users, you look up {aol.com, MX} and you get a set of four hostname / preference pairs back. If you want to know the hostname for the IP address 206.190.36.45 you look up {45.36.190.206.in-addr.arpa, PTR} and get a hostname back.
There’s a well-defined meaning to each of those query types  – A is for IP addresses, MX is for mailservers, PTR is for hostnames – and that was always the intent for how DNS should work.
When DNS was first standardized, though, there was one query type that didn’t really have any semantic meaning:

Read More

DMARC: an authentication framework

A new email industry group was announced this morning. DMARC is a group of industry participants, including large senders, large receivers and relevant intermediaries working on a framework to reduce the harm from phishing.
DMARC is working on a standard to allow senders to publish sending policies and receivers to act on those policies. Currently, senders who want receivers to not deliver unauthenticated email have to negotiate private agreements with the ISPs to make that happen. This is a way to expand the existing programs. Without a published standard, the overhead in managing individual agreements would quickly become prohibitive.
It is an anti-phishing technique built on top of current authentication processes. This is the “next step” in the process and one that most people involved in the authentication process were anticipating and planning for. I’m glad to see so many big players participating.
 

Read More

What to expect in 2016

WttWColorEye_forBlogI don’t always do predictions posts, even though they’re  popular. Most years I skip them because I don’t see major changes in the email space. And, I’m not the type to just write a prediction post just to post a prediction.
This year, though, I do see changes for everyone in the email space. Most of them center on finally having to deal with the technical debt that’s been accumulating over the past few years. I see ISPs and ESPs spending a lot of development effort to cope with the ongoing evolution authentication requirements.
When people started seriously looking at how to authenticate email, the first goal was getting organizations to implement the protocols. This was a practical concession; in order for a new protocol to be used it needs to be widely implemented. Phase one of authenticating email was simply about publishing protocols and getting organizations to use them.
During phase one, the organization that authenticated a mail hasn’t been important. In fact, the SPF spec almost guarantees that the ESP domain is the authenticated domain. In DKIM, the spec says any domain could sign as long as they could publish a public key in that domain’s domainkeys record.
ESPs took full advantage of this and lowered their own development overhead by taking most of the authentication responsibility on themselves. Their domains were in the 5321.from and they published the SPF records. Domains they control were in the d= and they generated and published the DKIM keys. Mail was authenticated without ESP customers having to do much.
We’ve hit the end of phase one. Most of the major players in the email space are authenticating outbound email. Many of the major players are checking authentication on the inbound. Phase one was a success.
We’re now entering phase two, and that changes thing. In phase two, SPF and DKIM are used as the foundation for user visible authentication. Neither SPF nor DKIM were designed to be user visible protocols. To understand what they’re authenticating you have to understand SMTP and email. Even now there are days when I begin talking about one of them and have to take a step back and think hard about what is being authenticated. And I use these things every day!
DMARC is the first of these end user visible protocols built on SPF and DKIM. It uses the established and widespread authentication to validate the user visible from address. This authentication requires that the d= value or the 5321.from address belong belong to the same domain in the visible from address. While you can pick whether the alignment between the visible from and the authentication is “strict” or “relaxed” you have no choice about the alignment.
Prior to DMARC no one really paid much attention to the domain doing the authentication. Authentication was a yes or a no question. If the answer was yes, then receivers could use the authenticated domain to build a reputation. But they weren’t really checking much in the way of who was doing the authentication.
In the push to deploy authentication, ESPs assumed the responsibility for authentication deployed ESPs took the responsibility and did most of the work. For many or most customers, authentication was as simple as clicking a checkbox during deployment. Some ESPs do currently let customers authenticate the mail themselves, but there’s enough overhead in getting that deployed that they often charged extra to cover the costs.
DMARC is rapidly becoming an expectation or even a full on requirement for inbox delivery. In order to authenticate with DMARC, the authenticating domain must be in the same domain space as the visible from. If senders want to use their own domain in the visible from, DNS records have to be present in that domain space. Whether it’s a SPF TXT record or a domainkeys record the email sender customer needs to publish the correct information in DNS. Even now, if you try to authenticate with DKIM through google apps, they require you to publish DNS records.
ESPs aren’t in a situation where they can effectively manage authentication alignment for all their customers. Hosting companies are in even worse shape when it comes to letting customers authenticate email. Developers are facing the fact they need to go back and rework their authentication code. Businesses are facing the fact they need to change their processes so customers can authenticate with DMARC.
It’s not just the infrastructure providers that are facing challenges with authentication. Senders are going to discover they can no longer hand authentication off to their ESPs and not worry about it. They’re going to have to get DNS records published by their own staff.
Getting DNS updates through some big companies is sometimes more difficult than it should be. I had one client a few years ago where getting rDNS changed to something non-generic took over a month. From an IT standpoint, changing DNS should require approvals and proper channels. Marketers may find this new process challenging.
And, if organizations want to publish reject policies for their domains, then they will have to publish records for every outside provider they use. Some of those providers can’t support DMARC alignment right now.
In 2016 a lot of companies will discover their current infrastructure can’t cope with modern authentication requirements. A lot of effort, both in terms of product development and software development, will need to be spent to meet current needs. This means a lot of user visible features will be displaced while the technical debt is paid.
These changes will improve the security and safety of email for everyone. It won’t be very user visible, which will give the impression this was a slow year for email development. Don’t let that fool you, this will be a pivotal year in email.

Read More