Asynchronous Bounces

There are three ways that an email can fail to be delivered:

  1. immediate rejection
  2. timeout
  3. asynchronous bounce

Rejection
A rejection is any delivery attempt where the sending smarthost can tell immediately that the mail can’t be delivered.
That will often be when the receiving machine accepts a connection but returns a “hard bounce” or “5xx” error at some point in the transaction, but can also be caused by the email address being malformed, the recipients domain not existing or the domain’s DNS being configured (accidentally or intentionally) to deny any email.
Traditionally this rejection will be recorded by the smarthost generating an asynchronous bounce email locally and sending that back to the local user who sent the message – that’s usually where the bounce message you see when you typo a recipient comes from. That’s an inefficient way of handling rejections at volume, though, so smarthost designed for high-volume email will typically report rejections some other way rather than creating asynchronous bounce.
Timeout
If something fails during an attempt to deliver a message, but the sending smarthost either can tell that it’s just a temporary problem, or it can’t tell for sure that it’s a permanent problem then the smarthost will keep a copy of that outbound message in it’s spool and schedule an attempt to resend it later.
One common cause of this is a “soft bounce” or “4xx” error at some point in the transaction, but broken DNS records, overloaded or unavailable servers, network problems or firewall level filtering can also cause delivery to fail in a potentially “temporary” way.
Some of these “temporary” problems aren’t really temporary, and the scheduled attempts to resend the mail will also fail. Eventually the smarthost will have give up on the delivery attempt and report a failure. That can take a long time – while the smarthost owner can configure how long it should keep trying it will often default to a week or so. Once the smarthost gives up it will report the failure, often by generating an asynchronous bounce email locally.
Asynchronous bounce
If the receiving mailserver accepts the message it is accepting responsibility for delivering it to the final recipient. If it finds out afterwards that it can’t deliver the message it sends a notification to the sender of the message, typically with more information about why delivery failed and a partial or complete copy of the message.
Like this:

Hi. This is the qmail-send program at yahoo.com.
I’m afraid I wasn’t able to deliver your message to the following addresses.
This is a permanent error; I’ve given up. Sorry it didn’t work out.
<cacakande@gmail.comx>:
74.125.142.27 failed after I sent the message.
Remote host said: 550-5.7.1 [98.138.207.123      12] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail,
550-5.7.1 this message has been blocked. Please visit
550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for
550 5.7.1 more information. g2si7192949igq.45 – gsmtp
— Below this line is a copy of the message.

Or this:

Delivery to the following recipient failed permanently:
nev.w.goodman@gone.bristol.ac.ukx
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain gone.bristol.ac.uk byaspmx.l.google.com. [173.194.67.27].
The error that the other server returned was:
550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient’s email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 k3si5496468wjb.158 – gsmtp
—– Original message —–

That’s an asynchronous bounce – “asynchronous” because it’s not synchronized with the delivery of the original message; it may come at any time after the delivery appeared to succeed, even weeks later.
There are quite a few operational problems with asynchronous bounces. One of the bigger ones is what’s known as backscatter. The mailserver generating the asynchronous bounce will send it to the email address in the bouncing email’s “envelope from” / “bounce address” – and when the mail being bounced is spam that email address is usually fake or an innocent third party. Several hundred backscatter bounces showed up in my inbox this morning, so it’s a bit of a sore point right now.
Because of the backscatter issue it’s considered a best practice to do as much filtering and other delivery decision making as possible while you have the sending mailserver still connected, so that you can safely reject the mail. Only as a last resort when you find out later that the message can’t be delivered should you send an asynchronous bounce.
Some people go further and suggest that any competently run mailserver should never send asynchronous bounces, and that there is never any operational need for them. I think they’re wrong, and judging from the bounces in my inbox, so do Google, Yahoo, Microsoft, Telstra, Eircom, Orange, OCN, t-online.de, Interhost.it, xs4all.nl and many, many other ISPs, companies and universities across the world…

Related Posts

AOL transmitting 4xx error for user unknown

AOL is currently returning “451 4.3.0 <invaliduser@aol.com>: Temporary lookup failure” in some cases when they really mean “550 user unknown.” This message from AOL should be treated as 5xx failure and the message should not be retried (if at all possible) and the failure should be counted as a hard bounce for list management purposes.
This is something broken at AOL’s end, and the guys with the magic fingers that keep the system running are working to fix it. Right now there doesn’t seem to be an ETA on a fix, though.
Even if you are a sender who is able to stop the retries, you may see some congestion and delays when sending to AOL for the time being. Senders who don’t get the message, or who are unable to stop their MTAs from retrying 4xx mail will continue to attempt delivery of these messages until their servers time out. This may cause congestion for everyone and a noticeable  slowdown on the AOL MTAs.
AOL blog post on the issue
HT: Annalivia

Read More

June 2014: The month in email

Each month, we like to focus on a core email feature or function and present an overview for people looking to learn more. This month, we addressed authentication with SPF.
We also talked about feedback mechanisms, and the importance for senders to participate in FBL processes.
In our ongoing discussions about spam filters, we took a look at the state of our own inboxes and lamented the challenge spam we get from Spamarrest. We also pointed out a post from Cloudmark where they reiterate much of what we’ve been saying about filters: there’s no secret sauce, just a continuing series of efforts to make sure recipients get only the mail they want and expect to receive. We also looked at a grey area in the realm of wanted and expected mail: role accounts (such as “marketing@companyname.com”) and how ESPs handle them.
As always, getting into the Gmail inbox is a big priority for our clients and other senders. We talked a bit about this here, and a bit more about the ever-changing world of filters here.
On the subject of list management, we wrote about the state of affiliate mailers and the heightened delivery challenges they face getting in the inbox. We got our usual quota of spam, and a call from a marketer who had purchased our names on a list. You can imagine how effective that was for them.
And in a not-at-all-surprising development, spammers have started to employ DMARC workarounds. We highlighted some of the Yahoo-specific issues in a post that raises more questions.
We also saw some things we quite liked in June. In the Best Practices Hall of Fame, we gave props to this privacy policy change notification and to our bank’s ATM receipts.
We also reviewed some interesting new and updated technology in the commercial MTA space, and were happy to share those findings.

Read More

July 2014: The month in email

We continue to be busy with really interesting client work. Look for some new posts and white papers to come out of this research over the next few months, but for now blogging has been a bit light while we’re working hard. In parallel with our busy times, we have also been pondering the ways in which the email world illustrates the classic bon mot  “plus ça change, plus c’est la même chose”, and we’ve been revisiting some posts from a few years ago to examine this.
We started July with a nod to a good subscription experience just as CASL, the Canadian Anti-Spam Legislation went into effect on Canada Day. While companies have another 17 months to put these provisions into practice, it’s a good reminder that periodic re-engagement with customers can be very effective in helping you maintain high-quality subscriber lists. We talked a bit more about CASL here and what protections the law intends.
In stark contrast, we posted about an organization that is doing a less-than-stellar job making sure they’re only sending wanted email. The Direct Marketing Association is a terrific resource and member organization for marketers across industries and channels, but their email marketing practices don’t always live up to their mission of “Advancing and Protecting Responsible Data-Driven Marketing”, and we explored some ways in which they might improve this.
Those of you who have been reading this blog for any time at all know that we tend to talk about wanted mail and unwanted mail rather than the more general category of spam. Marketers tend to think their mail can’t possibly be spam if it’s not offering Viagra or phishing for credit card information, but that’s not really the point — if a customer doesn’t want to read your email about new mountain bikes, even if they bought a mountain bike from you three years ago, that’s unwanted email. Here’s a post we revisited about why customers might not want your mail, and a new post about engagement.
One risk of sending unwanted email, of course, is that customers complain, and that will affect your delivery going forward. We revisited a post about feedback loops, and also talked a bit about addressing delivery problems as they come up rather than waiting for them to resolve on their own (mostly, they won’t!)
I also proposed a bit of a thought experiment around monetizing the complaint stream, and followed up with a second post. There are some good points in the comments of those posts, but mostly I think it’s an interesting solution to addressing risk and abuse at ESPs.
Finally, Steve wrote a short post about our new mail servers and how quickly spammers descended as we set those up. It’s a constant battle!

Read More