Message not compliant with the RFCs

laura
Industry
May 30, 2022

Every once in a while we’ll see a rejection from Yahoo that says RFCs 554 5.0.0 Message not accepted due to failed RFC compliance. What does that mean and what can we do about it?

It really does mean exactly what it says on the label: there’s something about the message that is not in compliance with any number of RFCs and are not going to accept the message in its current state. When trying to help a colleague diagnose the issue I came up with a list of things to check.

Troubleshooting in the email

Is there any high ASCII without quoted printable or Base64 encoding in the body or the headers?
Is there a Date header?
Is there any duplication in header fields?
Is there a bare IP address in a link somewhere?
Are the line lengths inside the message shorter than 998 characters?
Are lines correctly terminated with CR/LF?
Is the DKIM signature field correct?
Is the MIME type correct?

Troubleshooting outside the email

Is the sending domain configured for DNSSec? Did something break?
Is DKIM, SPF and DMARC correct or did a DNS update get pushed and break something?
Does the SPF domain have a MX and is it answering on port 25?
Does the From domain have a MX and is it answering on port 25?
Is there a problem with TLS negotiation?
Is there a network problem somewhere preventing DNS lookups from happening?

There are so many different technical ratholes to go down when troubleshooting technical errors, but this gives people a place to start with issue. The most common problems I see are:

Line lengths too long or incorrectly terminated. Yahoo isn’t the only receiver to refuse mail for this problem.
Header duplication. There are some headers that cannot and should not show up more than once in a message and sometimes there are duplications that happen.
Lack of a Date header. I’ve seen this in mail coming in here just recently.

Overall, most receivers are fairly liberal in what they receive and don’t reject mail simply because it’s not fully in compliance with the RFCs. Some, like Yahoo, are much pickier about the standards.

The underlying solution is the same: fix whatever it is that’s broken and resend the messages.

Edit: After this was posted a representative of Yahoo contacted me to let me know that their requirements for RFC compliance were to address some types of exploits they’ve seen in the past.

What kind of mail do filters target?

All to often we think of filters as a linear scale. There’s blocking on one end, and there’s an inbox on the other. Every email falls somewhere on that line.
Makes sense, right? Bad mail is blocked, good mail goes to the inbox. The bulk folder exists for mail that’s not bad enough to block, but isn’t good enough to go to the inbox.
Once we get to that model, we can think of filters as just different tolerances for what is bad and good. Using the same model, we can see aggressive filters block more mail and send more mail to bulk, while letting less into the inbox. There are also permissive filters that block very little mail and send most mail to the inbox.
That’s a somewhat useful model, but it doesn’t really capture the full complexity of filters. There isn’t just good mail and bad mail. Mail isn’t simply solicited or unsolicited. Filters take into account any number of factors before deciding what to do with mail.

Censorship, email and politics

Spamfiltering blocks email. This is something we all know and understand. For most people, that is everyone who doesn’t manage an email server or work in the delivery field or create spamfilters, filtering is a totally unseen process. The only time the average person notices filters is when they break. The breakage could be blocking mail they shouldn’t, or not blocking mail they should.
Yesterday, a bunch of people noticed that Yahoo was blocking mail containing references to a protest against Wall Street. This understandably upset people who were trying to use email as a communication medium. Many people decided it was Yahoo (a tool of the elites!) attempting to censor their speech and stop them from organizing a protest.
Yeah. Not so much.
Yahoo looked into it and reported that the mail had gotten caught in their spam filters. Yahoo adjusted their filters to let the mail through and all was (mostly) good.
I don’t think this is actually a sign of filters being broken. The blocked mail all contained a URL pointing to a occupywallst.com. I know there was a lot of speculation about what was being blocked, but sources tell me it was the actual domain. Not the phrase, not the text, the domain.
The domain was in a lot of mostly identical mail coming out of individual email accounts. This is a current hallmark of hijacked accounts. Spammers compromise thousands of email accounts, and send a few emails out of each of them. Each email is mostly identical and points to the same URL. Just like the protest mail.
There was also a lot of bulk mail being sent with that URL in it. I’ve been talking to friends who have access to traps, and they were seeing a lot of mail mentioning occupywallst.com in their traps. This isn’t surprising, political groups have some horrible hygiene. They are sloppy with acquisition, they trade names and addresses like kids trade cold germs, they never expire anything out. It’s just not how politics is played. And it’s not one party or another, it’s all of them. I’ve consulted with major names across the political spectrum, and none actually implement best practices.
As I have often said the secret to delivery is to not have your mail look like spam. In this case, the mail looked like spam. In fact, it looked like spam that was coming from hijacked accounts as well as spam sent by large bulk mailers. I suspect there was also a high complaint rate as people sent it to friends and family who really didn’t want to hear about the protests.
To Yahoo!’s credit, though, someone on staff was on top of things. They looked into the issue and the filter was lifted within a couple hours of the first blog post. A human intervened, overruled the algorithm and let the mail out.
I bet this is one of the few times anyone has seen that Yahoo does outbound filtering. Given it’s a politically charged situation, I can see why they assume that Yahoo is filtering because of politics and censorship. They weren’t though.
More on politics, filtering and censorship.

They’re not blocking you because they hate you
It really can be your email
More on Truthout
Another perspective on the politico article

New Spamhaus lists

Spamhaus announced today they are publishing two new BGP feeds: Extended DROP and the Botnet C&C list. These lists are intended for use inside routers in order to stop all traffic to or from listed IP addresses. This is a great way to impact botnet traffic and hopefully will have a significant impact on virus infections and botnet traffic.
In other news I’ve been hearing rumbling about changes at Yahoo. It looks like they have changed their filters and some senders are feeling lots of pain because of it. It looks like senders with low to mid range reputations are most affected and are seeing more and more of their mail hit the bulk folder. This afternoon I’m hearing that some folks are seeing delivery improvements as Yahoo tweaks the changes.