When did the reject happen?

laura
Industry
November 19, 2015

Earlier today I approved a comment from Mike on a post about problems at AOL from 2012. The part of the comment that caught my attention:

SMTP error from remote mail server after end of data:
521 5.2.1 : AOL will not accept delivery of this message.

Mike also mentioned his IP reputation is good, when he checks at AOL so he doesn’t understand why mail is being blocked.
I think the big clue is after the end of data and would look at the full content of the mail, particularly domains and URLs, to identify is triggering the block.
In the SMTP transaction there are only a few places the ISP can stop the transaction and each spot tells us different things about why the ISP is rejecting the message.

After connection

A block after connection is a block either against the IP address or against the domain in the rDNS of the IP. IPs with no rDNS or generic DNS can also be blocked here. Blocks here do happen, but many recipients will let the SMTP transaction continue.

After HELO/EHLO

A block after HELO/EHLO is often a block against the domain in the HELO/EHLO or against a particular HELO/EHLO. Malware and bots often have distinctive HELO/EHLO patterns and it’s common for those kinds of senders to be blocked at this point.

After Mail From

A block after Mail From is often directed at the domain in the bounce string. Some senders do check to make sure the domain has a MX and will block if it doesn’t. Blocks don’t happen here very often.

After RCPT To

Blocks here are not always spam related. Most of the delivery failures at this point have to do with non-existent addresses.

After DATA

Blocks after data mean the ISP has actually seen the full content of the email. If a block comes after DATA the full content of the message including the recipient and their permission status should be evaluated as part of the determination about what is triggering the block.
Using when the rejection happened is an important part of understanding why a block happened. For instance, if a block happens before DATA, you know that content isn’t relevant, because the ISP never saw the content. If a block happened before Mail From: you know it’s the IP address reputation or configuration. If a block happened after DATA you know you need to look at the whole message.

Deliverability and IP addresses

Almost 2 years ago I wrote a blog post titled The Death of IP Based Reputation. These days I’m even more sure that IP based reputation is well and truly dead for legitimate senders.
There are a lot of reasons for this continued change.

Thanks for the great session

I had a great time answering questions at the 2015 All About eMail Virtual Conference & Expo today. Thanks so much to everyone who participated and asked questions. They were great and I’m sorry we didn’t have more time.
I did get some questions on twitter (@wise_laura) afterwards. One was about an example I gave to explain how filters are complex. There have been rumors going around recently that Gmail is filtering mail with more than 3 URLs in it. Let me just say right now THIS IS NOT TRUE emails with more than 3 URLs in them are being delivered just fine to Gmail.
There is a situation involving the number (and type) of URLs that I think are a useful example of the filter complexity happening at some places, like Gmail. I started working on it, but don’t quite have time to finish it today, but will keep working on and it should go up in the next day or so.
Thanks again to everyone who joined the session. You asked some great questions and I had fun answering them.

Confusing the engineers

We went camping last weekend with a bunch of friends. Had a great time relaxing on the banks of the Tuolumne River, eating way too much and visiting.
On Saturday I was wearing a somewhat geeky t-shirt. It said 554: abort mission. (Thank you MessageSystems). At some point on Saturday every engineer came up to me, read my shirt and then looked at me and said “That’s not HTTP.”
That lead to various discussions about how their junior engineers don’t actually know SMTP at all. Why? Because the SMTP libraries just work. Apparently the HTTP libraries aren’t that great, so folks have to learn more about HTTP to troubleshoot and use them.
I’m sure there’s a joke in there somewhere: A Kindle engineer, an Android engineer and a robot engineer walk into a campsite…
It did leave me thinking, though, about how it’s not that easy to run your own mail server these days. Gone are the days when running your own server was cost effective and easy. These days, there is just too much spam coming in. Crafting filters is a skilled job. It’s not that hard to run good filters. But to run good filters takes time to do well.
There are also a lot of challenges to sending mail. One of the discussions I had at the campsite was how hard it was to configure outbound mail. The engineer was helping a friend set up a website and trying to get the website to send notifications to the friend. But without setting up authentication the mail kept silently failing.
Of course, we do run our own mail server. But it’s our job and, in many ways, it keeps us honest. We don’t run many filters meaning we see what spammers are doing and can use our own experiences to better understand what commercial filters are dealing with.
For most people, though, I really think using a service is the right solution. Find one with filters that meet your needs and just pay them to deal with the headache.