Troubleshooting delivery problems

Everyone has their own way of troubleshooting problems. I thought I would list out the steps I take when I’m trying to troubleshoot them.

image of a head with gears and ideas floating around it
  • Clarify the problem. As a consultant, folks come to me asking me to help them solve their delivery problems. My first step is to get them to clarify what symptoms they’re seeing. Something happened to make them contact me, and that’s where we start. Questions at this stage include:
    • Is mail going to bulk?
    • Is mail being rejected?
    • Where is mail rejected?
    • Are open rates steady?
    • What mail is affected?
  • Once you have the answers to the question, you have your problems statement. This is a one or two sentence summary that identifies the full issue. This is a step a lot of folks avoid doing, but a good problem statement drives the troubleshooting. Example problem statements:
    • Marketing email from our ESP goes to bulk at Gmail.
    • CRM email is seeing high numbers of temp failures at Yahoo.
    • Our domain is blocked by Filter provider.
  • Identify what are likely causes of the problem. Filters are pretty specific, so you want to change things that will actually make the filters do something different. Not just apply random best practices you’ve found on a website or blog (even this one). Common techniques, like sending to engaged users or removing bounced addresses don’t work everywhere, so you may end up weeks down the line with no improvement to show for it.
    • “Improving engagement” only works if the problem is bulk foldering at the three big mailbox providers. It does nothing anywhere else.
    • “List hygiene” only works if the problem is your list isn’t bounce handled (which is never, if you’re using a real ESP).
  • Implement specific changes to address the reason for the delivery problems. What was the problem and how do you back it out? “
    • Our sales dude decided to harvest addresses of a social networking site and add them to our newsletter list. We purged all those addresses (or all the addresses that clicked on a link that wasn’t unsubscribe) from our list.
    • There was a flaw in our data handling process and we reactivated addresses that had bounced off in the past We fixed the flaw and have removed the previously bounced addresses.
    • We changed our frequency and ended up sending too much mail, causing recipients to report the mail as spam. We backed down to our old volume and are being more selective about who gets the new, higher volume mailings.
    • Some of our sales folks have not been abiding by company directives and are using addresses they’ve ‘saved’ from previous campaigns but that should be suppressed. We’ve implemented technical steps to prevent them sending mail to these addresses again.

All of these are actual solutions clients have implemented over the years. They’re all specific and required a clear understanding of mail processes and data flow. Once we identified the problems, the solutions fell out the end, really.

Too often, though, delivery folks don’t actually ask the right question and they don’t actually take the time to identify the problem. Instead, they “implement best practices” and try and make random changes hoping something will change. Some of the time it works, often the best practice change will randomly hit on the underlying cause of the problem. But when the random best practice changes don’t work, you absolutely need to step back and start from the beginning.

Related Posts

Email filters and small sends

Have you heard about the Baader-Meinhoff effect?

The Baader-Meinhof effect, also known as frequency illusion, is the illusion in which a word, a name, or other thing that has recently come to one’s attention suddenly seems to appear with improbable frequency shortly afterwards (not to be confused with the recency illusion or selection bias). Baader–Meinhof effect at Wikipedia

There has to be an corollary for email. For instance, over the last week or so I’ve gotten an influx of questions about how to fix delivery for one to one email. Some have been from clients “Oh, while we’re at it… this happened.” Others have been from groups I’m associated with “I sent this message and it ended up in spam.”

Read More

Troubleshooting delivery is hard, but doable

Even for those of us who’ve been around for a while, and who have a lot of experience troubleshooting delivery problems things are getting harder. It used to be we could identify some thing about an email and if that thing was removed then the email would get to the inbox. Often this was a domain or a URL in the message that was triggering bulk foldering.
Filters aren’t so simple now. And we can’t just randomly send a list of URLs to a test account and discover which URL is causing the problem. Sure, one of the URLs could be the issue, but that’s typically in context with other things. It’s rare that I can identify the bad URLs sending mail through my own server these days.
There are also a lot more “hey, help” questions on some of the deliverability mailing lists. Most of these questions are sticky problems that don’t map well onto IP or domain reputation.
One of my long term clients recently had a bad mail that caused some warnings at Gmail.
We tried a couple of different things to try and isolate the problem, but never could discover what was triggering the warnings. Even more importantly, we weren’t getting the same results for identical tests done hours apart. After about 3 days, all the warnings went away and all their mail was back in the inbox.
It seemed that one mailing was really bad and resulted in a bad reputation, temporarily. But as the client fixed the problem and kept mailing their reputation recovered.
Deliverability troubleshooting is complicated and this flowchart sums up what it’s like.

Here at Word to the Wise, we get a lot of clients who have gone through the troubleshooting available through their ESPs and sometimes even other deliverability consultants. We get the tough cases that aren’t easy to figure out.
What we do is start from the beginning. First thing is to confirm that there aren’t technical problems, and generally we’ll find some minor problems that should be fixed, but aren’t enough to cause delivery problems. Then we look at the client’s data. How do they collect it? How do they maintain it? What are they doing that allows false addresses on their list?
Once we have a feel for their data processes, we move on to how do we fix those processes. What can we do to collect better, cleaner data in the future? How can we improve their processes so all their recipients tell the ISP that this is wanted mail?
The challenging part is what to do with existing data, but we work with clients individually to make sure that bad addresses are expunged and good addresses are kept.
Our solutions aren’t simple. They’re not easy. But for clients who listen to us and implement our recommendations it’s worth it. Their mail gets into the inbox and deliverability becomes a solved problem.

Read More

ESPs and deliverability

There’s an ongoing discussion, one I normally avoid, regarding how much impact an ESP has on deliverability. Overall, my opinion is that as long as you have a half way decent ESP they have no impact on deliverability. Then I started writing an email and realised that my thoughts are more complex than that.

Read More