ESPs, Non-portable Reputation and Vendor Lock-in

I’ve seen some mentions recently of ESPs suggesting that if you use your own domain in the From: of mail you send through an ESP then that ESP can’t “do email authentication” properly unless they require you to edit your domains DNS settings. That’s not really so, but there is a kernel of truth in there.
The real situation is, unsurprisingly, a bit more complicated.
What authentication features should you look for in an ESP?

  1. Awareness of email authentication, specifically SPF and DKIM
  2. They encourage you to use your own domain for authentication, but don’t require it
  3. They suggest you add something like “include:cust-spf.exacttarget.com” to an existing SPF record, rather than anything with “ip4:” in it
  4. They suggest you add an NS record something like “espname._domainkey.yourcompany.com” to your DNS for DKIM support
  5. They ask you specifically what domain you’d like to use for authentication, rather than requiring it be the same as the domain you use in your From: field
  6. They check that any DNS changes you’ve made are correct before sending email

  7. They monitor your DNS changes, to protect you from mistakes in the future

What authentication mistakes should make you wary of an ESP?

  1. Any suggestion that your reputation as a sender isn’t important, rather that it’s solely the ESPs job to get good delivery rates
  2. Any suggestion that the ESPs reputation is completely irrelevant, and your mail will be delivered solely based on your reputation
  3. They tell you that the domain name used in your email From: has to have anything to do with authentication
  4. They tell you to add a TXT record for SPF records, rather than adding an “include:” clause to your existing record
  5. They tell you to add anything with “ip4:” in it to your DNS
  6. They tell you to add an NS record like “_domainkey.yourcompany.com”
  7. Any suggestion you use ADSP (1).
  8. Any mention of DomainKeys (2)

Reputation?
When we’re talking “authentication” we’re really talking about SPF and DKIM. Both are ways to associate an identifier with the email that’s sent that receivers can use to track the reputation of your mail (so that you can get the delivery benefits of your history of sending wanted mail) and manage whitelisting and feedback loops and all sorts of other good things. See dkimcore.org for more details on why authentication can be a good thing.
Neither SPF nor DKIM require you to use the same domain you use in your From: header as you use for authentication. This has a couple of really important implications:

  1. Your ESP can use a domain of their own for authentication. That means that you can get some of the advantages of authentication (feedback loops, non-portable reputation) without having to edit your DNS or even knowing anything about what your ESP is doing.
  2. If you use a range of different domains in your From: field (for brand-specific mailing, maybe) you can use the a single identifier for authentication across all that mail, getting all the benefits of your history of sending wanted email for BrandA and BrandB when you launch a new campaign for BrandC.

Portable vs non-portable reputation
I used the term “non-portable reputation” there without explaining it.
Non-portable reputation is reputation that is tied to your ESP – as you send wanted mail from that ESP your reputation with ISPs receiving the mail will get better and better, and you’ll get all the delivery advantages associated with that. But if you take your business to another ESP, or start sending some mail yourself, you’ll lose all that good reputation and have to start over.
Portable reputation is reputation that is tied to you, and mostly independent of your ESP. You can build up a history of sending email that people want to receive through one ESP, then you can move to a different ESP and take all that good history with you, keeping all the delivery advantages. (Or use multiple ESPs for different campaigns and send some mail from in-house and pool your good reputation across all those sources of mail.)
If you’re a reputable sender, sending mail that people want, then building a portable reputation will significantly benefit you. It does require you to configure your domains DNS records to support it, though, which can be tricky to set up and isn’t supported well by some DNS providers. So if you’re just beginning to dangle your toes in email marketing using non-portable reputation from your ESP is a great way to get going, but you should plan on switching to using portable reputation if email is going to be important to your business.
Some ESPs don’t really like the idea of portable reputation as without it their good customers are likely to take a significant delivery rate hit if they move elsewhere – and that vendor lock-in helps them keep those customers. Better ESPs are more likely to be enthusiastic about portable reputation as while it makes it easier for customers to move to competing ESPs it will also make it easy for them to move to them from their competitors. Without that disincentive for customers to move from one ESP to another it makes it easier for good ESPs to compete based on price and quality of service.
You really want to be able to get portable reputation using both SPF and DKIM. For that you’ll want to use a consistent domain for authentication, often your main corporate domain, even if you use a range of domains in your From: field. You want to be able to do this in a way that doesn’t tie you to using a single ESP for all your mail, rather that lets you use multiple ESPs simultaneously without losing your reputation. And you want it done in a way that doesn’t require you to be involved in editing DNS records any time any of those ESPs needs to make a change, otherwise the IT overhead and the risk of mistakes will be something you’ll regret.
For SPF this can best be done with a DNS record that looks something like

yourdomain.com TXT “v=spf1 mx include:cust-spf.esp1.com include:cust-spf.esp2.com include:cust-spf.esp3.com ~all”

… possibly with a few “ip4:” fields near the beginning to specify your corporate mailservers. The use of include: to pull in the SPF configuration for each of the ESPs you use allows you to have valid SPF records for each of those ESPs, and allows them to make any changes that are needed as they rearchitect their networks without needing to bother you.
For DKIM this can best be done with a set of DNS records that looks something like

yourdomain._domainkey.yourdomain.com TXT (a DKIM public key goes here, if you want to use DKIM on mail you send yourself)
esp1._domainkey.yourdomain.com NS dkim-ns1.esp1.com
esp1._domainkey.yourdomain.com NS dkim-ns2.esp1.com
esp2._domainkey.yourdomain.com NS ns1.esp2.com
esp2._domainkey.yourdomain.com NS ns2.esp2.com
esp3._domainkey.yourdomain.com NS dkim-nameserver.esp3.com

The first record is just if you’re using DKIM (or DKIM Core) for mail you’re sending yourself. The remaining records allow the ESPs you’re using to do all the magic that’s needed to sign with DKIM – generating and rotating key pairs, publishing public keys, expiring public keys and so on – without needing you to do anything further once the NS records are set up.
If an ESP suggests that instead of delegating something like “esp1._domainkey.yourdomain.com” you instead delegate “_domainkey.yourdomain.com” then run away. If you were to do that it would make it impossible for you to use DKIM with any other ESP, or even to use it yourself.
(If you are an ESP, and you want more details on how to set up DKIM in this way, take a look at this.)
(Footnotes)

  1. ADSP is an extension of DKIM that can provide some minor benefits in rare circumstances, but will cause significant delivery problems most of the time. ADSP shouldn’t come up in conversation with an ESP unless you bring it up (and you shouldn’t do that unless you’re intimately aware of the damage it will do to your delivery)
  2. DomainKeys is the forerunner to DKIM. It’s obsolete now, and while a small number of ISPs still pay attention to DomainKeys signatures when there’s no DKIM signature, and some senders sign with both, there’s really no need for anyone to use DomainKeys today.

Related Posts

Predictions for 2008

I did not have a lot of predictions for what will happen with email at the beginning of the year so I did not do a traditional beginning of the year post. Over the last 3 – 4 weeks, though, I have noticed some things that I think show where the industry is going.
Authentication. In January two announcements happened that lead me to believe most legitimate mail will be DK/DKIM signed by the end of the year. AOTA announced that approximately 50% of all email was currently authenticated. They did not separate out SPF/SenderID authentication from DK/DKIM authentication, but this still suggests email authentication is being widely adopted. AOL announced they will be checking DKIM on their inbound mail. I expect more and more email will be DKIM signed in response to this announcement.
Filtering. The end of 2007 marked a steady uptick in mail being filtered or blocked by recipient domains. I expect this trend to continue throughout 2008. Recipient domains are rolling out new technology to measure complaints, evaluate reputation and monitor unwanted email in ways that tease out the bad actors from the good. This means more bad and borderline email will be blocked. Over the short term, I expect to see more good email blocked, too, but expect this will resolve itself by Q2/Q3.
Sender Improvements. As the ISPs get better at filtering, I expect that many borderline senders will discover they cannot continue to have sloppy subscription practices and still get their mail delivered. Improved authentication and better filtering let ISPs pin-point blocks. Instead of having to block by IP or by domain, they can block only some mail from a domain, or only some mail from an IP. There are a number of senders who are sending mail that users do not want mixed with mail that recipients do want. Right now, if there is more mail that recipients want in that mix, then ISPs let the mail through. This will not continue to happen through 2008. Senders will need to send mail users actively want in order to see good delivery.
Less is more. A lot of other email bloggers have talked about this, and I will echo their predictions. Less email is more. Send relevant mail that your customers want. Target, target, target. Good mailers will not send offers to their entire database, instead they will send mail to a select portion of their database.
Feedback loops. Use of feedback loops by recipient domains will continue to grow.
Mobile email. More recipients will be receiving email on mobile devices.
Suggestions for 2008

Read More

How to disable a domain

Sometimes you might want to make it clear that a domain isn’t valid for email.
Perhaps it’s a domain or subdomain that’s just used for infrastructure, perhaps it’s a brand-specific domain you’re only using for a website. Or perhaps you’re a target for phishing and you’ve acquired some lookalike domains, either pre-emptively or after enforcement action against a phisher, and you want to make clear that the domain isn’t legitimate for email.
There are several things to check before disabling email.
1. Are you receiving email at the domain? Is anyone else?
Check the MX records for the domain, using “host -t mx example.com” from a unix commandline, or using an online DNS tool such as xnnd.com.
If they’re pointing at a mailserver you control, check to see where that mail goes. Has anything been sent there recently?
If they’re pointing at a mailserver that isn’t yours, try and find out why.
If there are no MX records, but there is an A record for the domain then mail will be delivered there instead. Check whether that machine receives email for the domain and, if so, what it does with it.
Try sending mail to postmaster@ the domain, for instance postmaster@example.com. If you don’t get a bounce within a few minutes then that mail may be being delivered somewhere.
2. Are you sending email from the domain? Is anyone else?
You’re more likely to know whether you’re sending mail using the domain, but there’s a special case that many people forget. If there’s a server that has as it’s hostname the domain you’re trying to shut down then any system software running no that server – monitoring software, security alerts, output from cron and so on – is probably using that hostname to send mail. If so, fix that before you go any further.
3. Will you need mail sent to that domain for retrieving passwords?
If there are any services that might have been set up using an email address at the domain then you might need a working email address there to retrieve lost passwords. Having to set email back up for the domain in the future to recover a password is time consuming and annoying.
The domain registration for the domain itself is a common case, but if there’s any dns or web hosting being used for the domain, check the contact information being used there.
4. How will people contact you about the domain?
Even if you’re not using the domain for email it’s quite possible that someone may need to contact you about the domain, and odds are good they’ll want to use email. Make sure that the domain registration includes valid contact information that identifies you as the owner and allows people to contact you easily.
If you’re hosting web content using the domain, make sure there’s some way to contact you listed there. If you’re not, consider putting a minimal webpage there explaining the ownership, with a link to your main corporate website.
5. Disabling email
The easiest way to disable email for a domain is to add three DNS records for the domain. In bind format, they look like:

Read More

Who can you trust?

I’ve been recently dealing with a client who is looking at implementing authentication on their domains. He’s done a lot of background research into the schemes and has a relatively firm grasp on the issue. At this point we’re working out what policies he wants to set and how to correctly implement those policies.
His questions were well informed for the most part. A few of them were completely out of left field, so I asked him for some of his references. One of those references was the EEC Email Authentication Whitepaper.
My client was doing the best he could to inform himself and relies on industry groups like the EEC to provide him with accurate information. In this case, their information was incomplete and incorrect.
We all have our perspectives and biases (yes, even me!) but there are objective facts that can be independently verified. For instance, the EEC Authentication whitepaper claimed that Yahoo requires DKIM signing for access to their whitelist program. This is incorrect, a sender does not have to sign with DKIM in order to apply for the Yahoo whitelist program. A bulk sender does have to sign with DKIM for a Y! FBL, but ISPs are given access to an IP based FBL by Yahoo. I am shocked that none of the experts that contributed to the document caught that error.
Independent verification is one reason I publish the Delivery Wiki. It’s a resource for everyone and a way to share my knowledge and thought processes. But other experts can “check my work” as it were and provide corrections if my information is outdated or faulty. All too often, senders end up blaming delivery problems on evil spirits, or using “dear” in the subject line or using too much pink in the design.
Delivery isn’t that esoteric or difficult if you have a clear understanding of the policy and technical decisions at a range of ESPs and ISPs, the history and reasoning behind those decisions, and enough experience to predict the implications when they collide.
Many senders do face delivery challenges and there is considerable demand for delivery experts to provide delivery facts. That niche has been filled by a mix of people, of all levels of experience, expertise and technical knowledge, leading to the difficult task of working out which of those “experts” are experts, and which of those “facts” are facts.

Read More