Recent Posts

AOL accidentally hard bounces valid mail

Last night (Mar 29, 2017) between about 8pm Eastern and 9:30pm Eastern AOL suffered a technical issue. Every email sent to them received a “Recipient address rejected” reply. One example of the error message:
Mar 29 20:45:12 p2-lvmail11 lsb1-99-208-250/smtp[22251]: A88DFC2DBE9: to=<redacted@aol.com>, relay=mailin-01.mx.aol.com[64. 12.91.195]:25, delay=0.18, delays=0.01/0/0.14/0.03, dsn=5.1.1, status=bounced (host mailin-01.mx.aol.com[64.12.91. 195] said: 550 5.1.1 <redacted@aol.com>: Recipient address rejected: aol.com (in reply to RCPT TO command))
The issue was brought to AOLs attention and things were fixed rapidly after that. An AOL representative has stated that these were invalid replies and that addresses do not need to be removed from future emails.
Most of the ESPs are aware of this and are working to restore any bounced addresses to their users. At some places this requires manual intervention, so it’s taking some time to get all the addresses restored.
This is one of the reasons that our best bounce handling recommendations are not to remove an address for a single bounce – sometimes the ISPs have technical problems. Like the time a routing failure meant a major ISPs MX machines couldn’t reach their authentication servers to get the list of active users. Or the time all an ISPs MXs were removed from DNS. A lot of the internet is still managed manually, and despite extensive safeguards put in place bad things can, and do, still happen. Usually these problems are resolved quickly and mail starts flowing again.
Morning advice: Do not deactivate addresses that bounced at AOL last night.

Filters do what we tell them

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

What about the botnets?!

Botnets are a huge problem for a number of reasons. Not only are they used to send spam, they’re also used in criminal activities. One of the major challenges in dealing with botnets is finding and stopping the people who create and use them. Why? Because the internet is global and crime tends to be prosecuted within local jurisdictions.

Truth of Consequences

“If you want to use another means that violates the law, and every common definition of “spam”, then by all means, go ahead. You can enjoy fines and being added to the ROKSO database,” says a comment on my recent COI blog post. It’s both disconcerting and entirely predictable.

My post was a discussion of what to do with addresses that don’t confirm. Data tells us that there is some value in those addresses – that there are people who won’t confirm for some reason but will end up purchasing from an email. Using COI leaves some fraction of revenue on the table as it were. My post was a short risk analysis of things to think about when making decisions about continuing to mail to people who don’t confirm.
Mentioning COI often brings the only-COI-mail-is-not-spam zealots out of the woodwork, as it did in this case. In this case, we have the commenter first asserting that failure to do COI is a violation of CAN SPAM (it’s not). When this was pointed out, he started arguing with two people who have been actively fighting spam for 20 years (including running a widely used blocklist). Finally, he ends up with the comment asserting that anyone not using COI will end up on ROKSO. It’s as if he thinks that statement will fear other commenters into not having opinions. It can’t because everyone in the discussion, except possibly him, knows that it’s not true.
The worst problem with folks like the commenter is that they think asserting horrible consequences will make people cower. First off, people don’t react well to threats. Secondly, this is a hollow threat and most people reading this blog know it.
There are millions of mailing lists not using COI and have zero risk of ever getting on ROKSO. The only thing hollow threats do is make people not pay attention to what you have to say. Well, OK, and have me write a blog post about how those threats are bad because they’re completely removed from reality.
Exaggerating or lying about consequences is not just wrong, it’s stupid. “Do this or else BAD THING,” is awesome, up until someone decides they’re not going to do this and the bad thing never happens. It makes people less likely or pay any attention to you in the future. It certainly means your opinions and recommendations are not going to be listened to in the future.
I probably go too far the other direction. I can spend too much time contextualizing a recommendation. It’s one of the things I’m trying to get better about. No, client doesn’t need a 4 page discussion of the history of whatever, they just need 2 lines of what they should do. If they need the context, I can provide it later.
In order to effectively modify behavior consequences have to be real. Threats of consequences are meaningless. Any toddler knows this, and can quite accurately model when mom means it and when she’s just threatening.
Risk analysis is not about modifying behavior. It’s about analyzing a particular issue and providing necessary information so the company action understands potential consequences and the chance those risks will happen. The blog post about COI was not intended to modify anyone’s behavior. I know there are companies out there successfully maintaining two mail streams: one COI and one not. I know there are other companies out there successfully mailing only single opt-in mail. I know there are companies with complex strategies to verify identity and address ownership. And I smile every time I walk into a retail store and they ask me if my email address is still X and if I want to make any changes to it.
Lying about consequences does nothing to modify behavior. All it does is diminish the standing and audience of the liar. Be truthful about the consequences of an action or lack of action. Don’t make up threats in order to bully people into doing what you think is right. Sooner or later they’re going to realize that you don’t know what you’re talking about and start to ignore you.

Friday blogging… or lack of it

It seems the last few Friday’s I’ve been lax on posting. Some of that is just by Friday I’m frantically trying to complete all my client deliverables before the weekend. The rest of it is by Friday I’m just tired. Today had the added complication of watching the Trumpcare debate and following how (and how soon) it would affect my company if it passed.
That’s been a bit distracting, along with the other stuff I posted about yesterday. I wish everyone a great weekend.

Security, safety and the cavalry

In some ways it’s been really hard to focus on email for the last few months. There are so many more important issues in the world. Terrorism, Brexit, the US elections compromised by a foreign government, nuclear threats from multiple countries, the repeal of ACA, mass deportations and ICE raids here in the US. I find myself thinking about what to blog. Then I glance at the news and wonder if there’s any value in another blog post about deliverability.
Generally I’ve tried to keep politics and world events mostly off the blog. But sometimes events are such that I need to talk about them.
Last October I had the chance to speak at the Email Innovations Summit in London. Steve and I took the chance to spend some time doing tourist things in London – including a photo walk along the Thames.

As an American I’m always a little surprised by the security in London. I grew up a few miles outside of DC. I could talk about prohibited airspace and security measures before I was 10. London is so much more open than even the DC of my youth. The surprise there is that London has been a much bigger target and attacked more than any city in the US.
The last few times we were in London I noticed a bit more visible security. In 2013 it was armed security walking through Tube stations. Last year it was Underground trains that were one long car. They were a bit weird and visually disconcerting. The part that really made me think, though, was this was a way to stop people hiding explosives between cars and to facilitate evacuations if something happened.
Last night Steve and I were talking and I mentioned the attack in London didn’t seem like terrorism to me. And it didn’t, not really. He then pointed out that explosives and guns are difficult to come by in the UK and this was classic terrorism. Oh. Sometimes our cultural differences come out in the strangest places.
Thinking about bigger issues like this make it hard to focus on email. There’s a regularly shared joke in deliverability, “There’s no such thing as a deliverability emergency.” And there isn’t, not really. Yes, even if a whole range of IPs is listed on Spamhaus, it’s still not an emergency and there’s no fast response team to deal with it.
There are abuse issues that are higher stakes than getting to the inbox. Child abuse materials. Harassment. Privacy issues. Terror threats. Every online services company, particularly the social media companies, have to deal with these kinds of things. Many of them are dealing poorly. Others have employees who are doing their best, but lack the tools, support, and training to do it well. Many companies don’t understand why they need to police their customer base.
The reality is, though, that abuse on the net (as opposed to abuse of the net) is a huge issue that needs to be dealt with. These are not small issues. The Internet is global and there’s no internet police. Law enforcement in different jurisdictions have to work together with technology experts to address crime and harassment on the internet.
It may surprise you to hear that the people who create spam filters and try and protect your inbox are the same people who fight crime on the internet. Spam and email are a vital part of online crime, so it falls on the abuse team to work with and educate law enforcement about tracing the source of email. The people you never see in ops, and abuse and support are vital to protecting folks online.
During the closing talk at MAAWG the chair was discussing how we can protect our online spaces. He stated “There is no cavalry; no second wave. It’s us or no one.” That’s a huge thing. My friends and colleagues are the people who stand protecting users online. It feels like a huge burden, but it’s something we can do to make the world a better and safer place.

It's not fair

In the delivery space, stuff comes in cycles. We’re currently in a cycle where people are unhappy with spam filters. There are two reasons they’re unhappy: false positives and false negatives.
False positives are emails that the user doesn’t think is spam but goes into the bulk folder anyway.
Fales negatives are emails that the user does thing is spam but is delivered to the inbox.
I’ve sat on multiple calls over the course of my career, with clients and potential clients, where the question I cannot answer comes up. “Why do I still get spam?”
I have a lot of thoughts about this question and what it means for a discussion, how it should be answered and what the next steps are. But it’s important to understand that I, and most of my deliverability colleagues, hate this question. Yet we get it all the time. ISPs get it, too.
A big part of the answer is because spammers spend inordinate amounts of time and money trying to figure out how to break filters. In fact, back in 2006 the FTC fined a company almost a million dollars for using deceptive techniques to try and get into filters. One of the things this company did would be to have folks manually create emails to test filters. Once they found a piece of text that would get into the inbox, they’d spam until the filters caught up. Then, they’d start testing content again to see what would get past the filters. Repeat.
This wasn’t some fly by night company. They had beautiful offices in San Francisco with conference rooms overlooking Treasure Island. They were profitable. They were spammers. Of course, not long after the FTC fined them, they filed bankruptcy and disappeared.
Other spammers create and cultivate vast networks of IP addresses and domains to be used in snowshoeing operations. Still other spammers create criminal acts to hijack reputation of legitimate senders to make it to the inbox.
Why do you still get spam? That’s a bit like asking why people speed or run red lights. You still get spam because spammers invest a lot of money and time into sending you spam. They’re OK with only a small percentage of emails getting through filters, they’ll just make it up in volume.
Spam still exists because spammers still exist.

Spam complaints… ish

I know a lot of folks working at ESPs. For those I know well, I will usually send in reports. Sometimes they’re not spam reports per se. Often it’s just “hey, this sender shouldn’t have my address, might want to poke them.”
Sometimes it’s even more specific. A few years ago I spoke at a user conference for an ESP. I stayed at the hotel for one night, and the hotel now has my email address. Not a big deal, they’re on the coast and an easy drive from here. They’ll run specials for the locals, and I like it.
Enter in hotel B. I’ve never stayed at hotel B. I’m not sure who hotel B is. They’re also local and may be the same management. I don’t know. They sent me email to the address I’ve only given to hotel A. Not only that, the message is completely unreadable. Dark blue on brown… not exactly a great design choice.
I wasn’t going to send anything in to the ESP, but then I noticed that at the bottom of the email there is a notice that says “This email was sent to: %%emailaddr%%.” That looks suspiciously like this was an accidental send. The ESP folks there are colleagues, so I sent them an email into abuse@.

Relaying Denied

I’ve got multiple clients right now looking for insights about bounce handling. This means I’m doing a lot of thought work about bounces and what they mean and how they match up and how different ISPs manage delivery and how different ESPs manage delivery and how it all fits together. One thing I’ve been trying to do is contextualize bounces based on what the reason is.
Despite what people may thing, spam filtering isn’t the only reason an email fails to deliver. There are lots of other reasons, too. There is a whole category of network problems like routing issues, TCP failures, DNS failures and such. There are address issues where a recipient simply doesn’t exist, or is blocking a particular sender. There are spam and authentication issues. The discussion of all these issues is way longer than a blog post, and I’m working on that.
One of the interesting bounces that is so rare most people, including me, never talk about is “Relaying Denied.” This is, however, one of the easier bounces to explain.
Relaying Denied means the mail server you’re talking to does not handle mail for the domain you’re sending to.
Well, OK, but how does that happen?
There are a couple reasons you might get a “Relaying Denied” message, most of them having to do with a misconfiguration somewhere. For whatever reasons, the receiving server doesn’t handle mail for a domain.
DNS records are incorrect. These can be due to a number of things

11 Innovators in the Email Game

Today AWeber published a link to 11 innovators in email marketing. I’m honored to be one of them.
I don’t really think of myself as a marketer, I’m a delivery person. My job, really, is to help clients devise email strategies (and overall digital marketing strategies) that result in inbox delivery. When I started, there were some significant divides between email marketing and deliverability. Often what was good marketing strategy was bad deliverability strategy. That’s not as true as it once was and now good deliverability advice is good marketing advice.
Thanks, AWeber!