Thoughts on bounce handling

This week’s Wednesday question comes from D.

What are your thoughts on bounce handling

There’s no specific standard for bounce handling as we often talk about it. While a lot of people will mention the RFCs, the RFCs only say what to do with a single email that gets a 4xy or a 5xy during the SMTP transaction. If the message gets a 5xy during the SMTP transaction, then the email MUST NOT be retried. If the message gets a 4xy during the SMTP transaction then it can be retried.
What the RFCs don’t mention is what to do with future emails to that email address. In context, this makes sense. When the RFCs were originally written there weren’t lists of tens or hundreds of thousands of addresses. At that time, every email was sent by a person and the bounces came directly back to that person. The person could decide, based on the bounce, what to do about future emails.
Volume, however, means we have to automate bounce handling and parse hundreds of different rejection messages.
I’m somewhat agnostic on exactly how someone should bounce handle. As long as a sender is making a good faith effort to remove undeliverable addresses within a reasonable time frame, I don’t think the details are critical. Removing after 4 bounces or after 5 bounces isn’t going to make a huge difference in delivery.
I don’t advocate bounce handling for 4xx messages. If you have a compliant MTA, then it will retry mail rejected with a 4xx. If that mail cannot be delivered over your retry period (usually over 3 – 5 days) then it should “hard bounce” with a 5xx error and that will be measured as a hard bounce.
The handshake standard that those ISPs and ESPs in the room agreed to back in 2002(ish) was 3 consecutive bounces in more than 14 days meant an address should be removed. It basically gives the senders mailing daily get 14 chances to mail to a non existent addresses. Mailers that send weekly get 3 attempts.
Sometimes ISPs do have issues. Very rarely they’ll lose some sync between their MTA and their backend database and indicate addresses are invalid when they really aren’t. Giving a minimum 2 weeks and a minimum number of bounces keeps these (again, VERY RARE) happenings from removing all the addresses off a list.
I don’t think spamblocks should be necessarily used to purge addresses, only unknown user 5xy responses (usually 500 or 550).
For some of my clients with more configurable systems, I will often have them purge addresses on the first bounce *if* that is also the first time they’ve mailed that address. It’s a statistics thing: the chance of a valid address getting on a list the same day as an ISP falls over is fairly low. Therefore, being aggressive in bounce handling on the first mailing is a good idea. Also, if a company is trying to re-engage users or start mailing a segment that hasn’t been mailed recently, then they should consider removing on a single bounce.
Don’t forget there are times when mail will fail even before the start of a SMTP session. This means that you WILL NOT get a 5xy or 4xy message on many MTAs (some of the commercial ones manage this for you, most of the open source ones don’t). These failures are typically DNS failures; things like no MX, a MX of “.”, or NXDomain are all signs that address is undeliverable and should be handled as an unknown user response. MX . or NXdomain should be purged immediately.

Opting customers in to new programs
Censoring email

Related Posts

20% of email doesn't make it to the inbox

Return Path released their global delivery report for the second half of 2009. To put together the report, they look at mail delivery to the Mailbox Monitor accounts at 131 different ISPs for 600,000+ sends. In the US, 20% of the email sent by Mailbox Monitor customers to Return Path seed accounts doesn’t make it to the inbox. In fact, 16% of the email just disappears.
I’ve blogged in the past about previous Return Path deliverability studies. The recommendations and comments in those previous posts still apply. Senders must pay attention to engagement, permission, complaints and other policy issues. But none of those things really explain why email is missing.
Why is so much mail disappearing? It doesn’t match with the philosophy of the ISPs. Most ISPs do their best to deliver email that they accept and I don’t really expect that ISPs are starting to hard block so many Return Path customers in the middle of a send. The real clue came looking at the Yahoo numbers. Yahoo is one of those ISPs that does not delete mail they have accepted, but does slow down senders. Other ISPs are following Yahoo’s lead and using temporary failures as a way to regulate and limit email sent by senders with poor to inadequate reputations. They aren’t blocking the senders outright, but they are issuing lots of 4xx “come back later” messages.
What is supposed to happen when an ISP issues a 4xx message during the SMTP transaction is that email should be queued and retried. Modern bulk MTAs (MessageSystems, Port25, Strongmail) allow senders to fine tune bounce handling, and designate how many times an email is retried, even allowing no retries on a temporary failure.
What if the missing mail is a result of senders aggressively handling 4xx messages? Some of the companies I’ve consulted for delete email addresses from mailing lists after 2 or 3 4xx responses. Other companies only retry for 12 – 24 hours and then the email is treated as hard bounced.
Return Path is reporting this as a delivery failure, and the tone of discussion I’m seeing seems to be blaming ISPs for overly aggressive spamfiltering. I don’t really think it’s entirely an ISP problem, though. I think it is indicative of poor practices on the part of senders. Not just the obvious permission and engagement issues that many senders deal with, but also poor policy on handling bounces. Perhaps the policy is fine, but the implementation doesn’t reflect the stated policy. Maybe they’re relying on defaults from their MTA vendor.
In any case, this is yet another example of how senders are in control of their delivery problems. Better bounce handling for temporary failures would lower the amount of email that never makes it to the ISP. This isn’t sufficient for 100% inbox placement, but if the email is never handed off to the ISP it is impossible for that email to make it to the inbox.

Read More

Broken record…

The Return Path In the Know blog listed 4 reasons mailing those old addresses is a bad idea.
Ashley, the author, is completely right and I endorse everything she said. (Although I’d really like to hear what happened to the customer that added back all those addresses. What was the effect on that campaign and future email marketing?) As I was reading the article though, I realized how many times this has been said and how depressing it is that we have to say it again. And again. And again.
A number of folks have told me that the reason they don’t pay any attention to delivery professionals is because we don’t provide enough real data. They can show that sending mail to old addresses costs them nothing, and makes them real money.
That’s not really true, though. We do provide data, they just don’t like it so they don’t listen to it. Return Path publishes lots of numbers showing that mailing unengaged recipients lowers overall delivery. I can provide case studies and data but companies that are committed to sending as much mail as possible throw up many reasons why our data isn’t good or valid.
The biggest argument is that they want hard numbers. I do understand this. Numbers are great. Direct and clear answers are wonderful. But delivery is a squishy science. There are a lot of inputs and a lot of modifiers and sometimes we can’t get exactly one answer. The data is noisy, and difficult to replicate. One of the reasons is that filtering is a moving target. Filters are not, and cannot be, fixed. They are adaptive and are changing even between one hour and the next.
Delivery experts are about risk management. They are the parents requiring everyone in the car wear seat belts, even though the driver has never had an accident. They are the fire department enforcing fire codes, even though it’s the rainy season.
Risk management isn’t about the idea that bad things will absolutely happen but rather that it is more likely that a bad thing will happen in some cases.
In this case, it’s more likely that delivery problems will happen when mailing old addresses. And if those addresses aren’t actively contributing to revenue, it’s hard to argue that their presence on a list is more beneficial than their absence.
But I repeat myself. Again.

Read More

8 things that make your mail look like spam

In the comments of last week’s Wednesday question John B. asked

Read More