SPF Fail: too many DNS lookups

I’ve had a couple folks come to me recently for help troubleshooting SPF failures. The error messages said the SPF record was invalid, but by all checks it was valid.
Eventually, we tracked the issue down to how many include files were in the SPF record.
The SPF specification specifically limits the number of lookups that can happen during a SPF check.

SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the “include” mechanism or the “redirect” modifier. If this number is exceeded during a check, a PermError MUST be returned. The “include”, “a”, “mx”, “ptr”, and “exists” mechanisms as well as the “redirect” modifier do count against this limit. The “all”, “ip4”, and “ip6” mechanisms do not require DNS lookups and therefore do not count against this limit. Processing limits

Some senders are using include: records that then have include: records that then have yet other include: records. Looking up all these include records caused the SPF lookup limits to be reached, thus causing the SPF lookups to fail (all the records can’t be pulled). Note, a lot of SPF checks don’t actually comply with the SPF spec in this area. Because of the spotty compliance, some too long SPF records won’t fail everywhere.
How can you fix this?

  • Prune unnecessary :include files from the SPF record.
  • Use different domains for mail from different places so you don’t need one SPF record to rule them all.
  • Don’t publish SPF records, rely on DKIM for authentication. Even Hotmail is checking DKIM these days.

If you need to find out how many lookups your SPF record entails you can use the SPF checker at emailstuff.org. Enter in a domain in the box and hit go. Then click on the “DNS” tab to see the actual DNS lookups that are happening. Each red domain is a separate DNS lookup.

Secureserver.net has a lot of include: files and a lot of lookups. Because this is a checking tool, it doesn’t limit the number of queries (because that wouldn’t be useful). But in places that are implementing the spec correctly, and are refusing mail based on SPF failures, secureserver.net would experience mail failures.

Related Posts

Hotmail moves to SPF authentication

Hotmail has recently stopped using Sender ID for email authentication and switched to authenticating with SPF. The protocol differences between SenderID and SPF were subtle and most senders who were getting a pass at Hotmail were already publishing SPF records.
From an email in my inbox from September:

Read More

Weird mail problems today? Clear your DNS cache!

A number of sources are reporting this morning that there was a problem with some domains in the .com zone yesterday. These problems caused the DNS records of these domains to become corrupted. The records are now fixed. Some of the domains, however, had long TTLs. If a recursive resolver pulled the corrupted records, it could take up to 2 days for the new records to naturally age out.
Folks can fix this by flushing their DNS cache, thus forcing the recursive resolver to pull the uncorrupted records.
EDIT: Cisco has published some more information about the problem. ‘Hijacking’ of DNS Records from Network Solutions

Read More

Who can you trust?

I’ve been recently dealing with a client who is looking at implementing authentication on their domains. He’s done a lot of background research into the schemes and has a relatively firm grasp on the issue. At this point we’re working out what policies he wants to set and how to correctly implement those policies.
His questions were well informed for the most part. A few of them were completely out of left field, so I asked him for some of his references. One of those references was the EEC Email Authentication Whitepaper.
My client was doing the best he could to inform himself and relies on industry groups like the EEC to provide him with accurate information. In this case, their information was incomplete and incorrect.
We all have our perspectives and biases (yes, even me!) but there are objective facts that can be independently verified. For instance, the EEC Authentication whitepaper claimed that Yahoo requires DKIM signing for access to their whitelist program. This is incorrect, a sender does not have to sign with DKIM in order to apply for the Yahoo whitelist program. A bulk sender does have to sign with DKIM for a Y! FBL, but ISPs are given access to an IP based FBL by Yahoo. I am shocked that none of the experts that contributed to the document caught that error.
Independent verification is one reason I publish the Delivery Wiki. It’s a resource for everyone and a way to share my knowledge and thought processes. But other experts can “check my work” as it were and provide corrections if my information is outdated or faulty. All too often, senders end up blaming delivery problems on evil spirits, or using “dear” in the subject line or using too much pink in the design.
Delivery isn’t that esoteric or difficult if you have a clear understanding of the policy and technical decisions at a range of ESPs and ISPs, the history and reasoning behind those decisions, and enough experience to predict the implications when they collide.
Many senders do face delivery challenges and there is considerable demand for delivery experts to provide delivery facts. That niche has been filled by a mix of people, of all levels of experience, expertise and technical knowledge, leading to the difficult task of working out which of those “experts” are experts, and which of those “facts” are facts.

Read More