Filters

The Future of Deliverability

There always seems to be appetite from folks to read the tea leaves and follow up with predictions about what the future holds. I mean, how many folks in the US are obsessively refreshing polls for the last few weeks? (American’s: don’t forget to vote on Tuesday!)

Welcome to deliverability week. I want to especially thank Al for doing a lot of work behind the scenes herding this group of cats. He’s an invaluable asset to the community.

They Must Have Changed Something…

One of the most common refrains I hear from folks with delivery problems is that the filters must have changed because their mail suddenly started to go to the bulk folder. A few years ago, I posted about how even when there is no change in the sender’s behavior, reputation can slowly erode until mail suddenly goes to the Gmail bulk folder. Much of that still applies – although the comments on pixel loads (what other folks call ‘open rates’) are a bit outdated due to changes in Gmail behavior.

Is email dead?

These last few years have been something, huh? Something had to give and, in my case, that something was blogging. There were a number of reasons I stopped writing here, many of them personal, some of them more global. I will admit, I was (and still am a little) burned out as it seemed I was saying and writing the same things I’d been saying and writing for more than a decade. Taking time off has helped a little bit, as much to focus on what I really want to talk about.

Message not compliant with the RFCs

Every once in a while we’ll see a rejection from Yahoo that says RFCs 554 5.0.0 Message not accepted due to failed RFC compliance. What does that mean and what can we do about it?

Step by Step guide to fixing Gmail delivery

I regularly see folks asking how to fix their Gmail delivery. This is a perennial question (see my 2019 post and the discussions from various industry experts in the comments). Since that discussion I haven’t seen as much complaining about problems.

Misinformation on filters

I’ve seen reports that someone is asserting that utm=COVID19 in URLs results in all mail going to bulk at multiple ISPs. This is the type of thing that someone says is true and dozens of folks believe it and thus a “deliverability phact” is born. For a plethora of reasons, this doesn’t pass the sniff test. Don’t believe everything you read on the internet.

Discussion Session

More than 30 people joined our delivery discussion from last Wednesday evening (Irish time). Thanks to all who joined and participated.

Troubleshooting: part 3

As I continue to think about how people troubleshoot email delivery I keep finding other things to talk about. Today we’re going to talk about the question most folks start with when troubleshooting delivery. “Did ISP change something?”

Same MX, different filters

One of the things I do for clients is look at who is really handling mail for their subscribers. Steve’s written a nifty tool that does a MX lookup for a list of domains. Then I have a SQL script that takes the raw MX lookup and categorizes not by the domain or even the MX, but by the underlying mail filter.

Troubleshooting delivery problems

Everyone has their own way of troubleshooting problems. I thought I would list out the steps I take when I’m trying to troubleshoot them.

Mentally modelling filters

When we talk about filters, we often think there is one filter. But, in many cases there are multiple stages of filters, each examining mail in a different way.

Spam is never timely nor relevant

One of the ongoing recommendations to improve deliverability is to send email that is timely and relevant to the recipient. The idea being that if you send mail a recipient wants, they’re more likely to interact with it in a way that signals to the mailbox provider that the message is wanted. The baseline for that, at least whenever I’ve talked about timely and relevant, is that the recipient asked for mail from you in the first place.

Barracuda update

The Barracuda twitter account has been very helpful and responsive to the issue. A few hours ago they tweeted that the problem should have been fixed.

End of an era

A few weeks ago, Return Path announced they were being purchased by Validity, who also own BrightVerify. Last week, they had a round of layoffs. According to sources inside the industry, Validity is closing the New York headquarters and Indianapolis offices and layoffs involved more than 170 staff members.

Their network, their rules

Much of the equipment and wires that the internet runs on is privately owned, nor is it a public utility in the traditional sense. The owners of the property have a lot of leeway to do what they like with that property. Yes, there are standards, but the standards are about interoperability. They describe things you have to do in order to exchange traffic with other entities. They do not dictate internal policies or processes.

Email filters and small sends

Have you heard about the Baader-Meinhoff effect?

The Baader-Meinhof effect, also known as frequency illusion, is the illusion in which a word, a name, or other thing that has recently come to one’s attention suddenly seems to appear with improbable frequency shortly afterwards (not to be confused with the recency illusion or selection bias). Baader–Meinhof effect at Wikipedia

There has to be an corollary for email. For instance, over the last week or so I’ve gotten an influx of questions about how to fix delivery for one to one email. Some have been from clients “Oh, while we’re at it… this happened.” Others have been from groups I’m associated with “I sent this message and it ended up in spam.”

Gmail, machine learning, filters

I’m sure by now readers have seen the article from Gmail “Spam does not bring us joy — ridding Gmail of 100 million more spam messages with TensorFlow.” If you haven’t seen it, go read it. It’s not often companies write about their filtering philosophy and what tools they’re using to manage incoming bad mail.

Filters working as intended

One of the toughest deliverability problems to deal with is when mail is blocked or going to spam because the filters are working as intended. Often the underlying issue is a lack of permission.

Automated link checking getting more sophisticated

As the volume and severity of malicious email increases, filters are increasingly following links in emails. This is really nothing new. Barracuda and other filters have been inspecting links automatically for years. From what I’ve seen there does seem to be some level of risk analysis based on domain reputation. That makes sense, not only is following links computationally expensive, it can also delay mail receipt.

Reputation is in the eye of the beholder

A few years ago reputation was generally recognised as one thing. If a sending reputation or IP reputation was good in one place it was likely good in other places. Different entities mostly reputation using the same set of signals albeit slightly tweaked to meet their own needs. More recently there is a divergence in how reputation is measured, meaning delivery can be vastly different across entities.

It’s a new year, do you know what your filters are doing?

Yesterday the NJABL domain expired. The list was disabled back in 2013 but the domain continued to be maintained as a live domain. With the expiration, it was picked up by domain squatters and is now listing everything. Steve wrote about how and why expired blocklist domains list the world last year.

Thinking about filters

Much of the current deliverability advice focuses on a few key ideas:

All filters are not equal

Many questions about delivery problems often assume that there is one standard email filter and the rules are the same across all of them. Unfortunately, this isn’t really the case.

The inbox is a moving target

The more I look at the industry, the more convinced I am that we’re in the middle of a fundamental shift in how email is filtered. This shift will change how we handle email deliverability and what tools we have and what information we can use as senders to address challenges to getting to the inbox.

Who are mimecast?

Mimecast is a filter primarily used by businesses. They’re fairly widely used. In some of the data analysis I’ve done for clients, they’re a top 10 or top 20 filter.
Earlier today someone asked on Facebook if mimecast may be blocking emails based on the TLD. The short answer is it’s unlikely. I’ve not seen huge issues with them blocking based on TLD of the domain. They’re generally more selective than that.

The good news is mimecast is really pretty good about giving you explanations for why they’re blocking. They’ll even tell you if it’s mimecast related or if it’s a specific user / user-company block.
Some example rejection messages from a recent dive into some bounce logs.

What's up with microsoft?

A c/p from an email I sent to a mailing list.
I think we’re seeing a new normal, or are still on the pathway to a new normal. Here’s my theory.
1) Hotmail made a lot of underlying code changes, learning from 2 decades of spam filtering. They had a chance to write a new codebase and they took it.
2) The changes had some interesting effects that they couldn’t test for and didn’t expect. They spent a month or two shaking out the effects and learning how to really use the new code.
3) They spent a month or two monitoring. Just watching. How are their users reacting? How are senders reacting? How are the systems handling everything?
3a) They also snagged test data along the way and started learning how their new code base worked and what it can do.
4) As they learned more about the code base they realized they can do different and much more sophisticated filtering.
5) The differences mean that some mail that was previously OK and making it to the inbox isn’t any longer.
5a) From Microsoft’s perspective, this is a feature not a bug. Some mail that was making it to the inbox previously isn’t mail MS thinks users want in their inbox. So they’re filtering it to bulk. I’ll also step out on a limb and say that most of the recipients aren’t noticing or caring about the missing mail, so MS sees no reason to make changes to the filters.
6) Expect at least another few rounds of tweak and monitor before things settle into something that changes more gradually.
Overall, I think delivery at Microsoft really is more difficult and given some of the statements coming out of MS (and some of the pointed silence) I don’t think they’re unhappy with this.

What does mitigation really mean?

It is a regular occurrence that senders ask filters and ISPs for mitigation. But there seems to be some confusion as to what mitigation really means. I regularly hear from senders who seem to think that once they’ve asked for mitigation that they don’t have to worry about filtering or blocking at that ISP for a while. They’re surprised when a few weeks or even days after they asked for mitigation their mail is, one again, blocked or in the bulk folder.

Why is my cold email going to the spam folder?

Because that’s what the spam folder is for unsolicited email.

Want some history?

I was doing some research today for an article I’m working on. The research led me to a San Francisco Law Review article from 2001 written by David E. Sorkin. Technical and Legal Approaches to Unsolicited Electronic Mail (.pdf link). The text itself is a little outdated, although not as much as I expected. There’s quite a good discussion of various ways to control spam, most of which are still true and even relevant.

From a historical perspective, the footnotes are the real meat of the document. Professor Sorkin discusses many different cases that together establish the rights of ISPs to filter mail, some of which I wasn’t aware of. He also includes links to then-current news articles about filtering and spam. He also mentions different websites and articles written by colleagues and friends from ‘back in the day’ discussing spam on a more theoretical level.
CNET articles on spam and filtering was heavily referenced by Professor Sorkin. One describes the first Yahoo spam folder. Some things never change, such as Yahoo representatives refusing to discuss how their system works. There were other articles discussing Hotmail deploying the MAPS RBL (now a part of Trend Micro) and then adding additional filters into the mix a few weeks later.
We were all a little naive back then. We thought the volumes of email and spam were out of control. One article investigated the effectiveness of filters at Yahoo and Hotmail, and quoted a user who said the filters were working well.

Is purging always effective?

What kind of mail do filters target?

All to often we think of filters as a linear scale. There’s blocking on one end, and there’s an inbox on the other. Every email falls somewhere on that line.
Makes sense, right? Bad mail is blocked, good mail goes to the inbox. The bulk folder exists for mail that’s not bad enough to block, but isn’t good enough to go to the inbox.
Once we get to that model, we can think of filters as just different tolerances for what is bad and good. Using the same model, we can see aggressive filters block more mail and send more mail to bulk, while letting less into the inbox. There are also permissive filters that block very little mail and send most mail to the inbox.
That’s a somewhat useful model, but it doesn’t really capture the full complexity of filters. There isn’t just good mail and bad mail. Mail isn’t simply solicited or unsolicited. Filters take into account any number of factors before deciding what to do with mail.

Updating the filtering model

One thing I really like about going to conferences is they’re often one of the few times I get to sit and think about the bigger email picture. Hearing other people talk about their marketing experiences, their email experiences, and their blocking experiences usually triggers big picture style thoughts.
Earlier this week I was at Activate18, hosted by Iterable. The sessions I attended were interesting and insightful. Of course, I went to the deliverability session. While listening to the presentation, I realized my previous model of email filtering needed to be updated.

Microsoft using the List-Unsubscribe header

An interesting observation from Brian Curry about how Microsoft is using the List Unsubscribe header in their interface. The short version is that Microsoft is only supporting mailto: links. They’re ignoring any List-Unsubscribe links that are a URL.
Here are some screenshots. When the sender is using a List-Unsubscribe <http://> header, Microsoft states that there is no information on how to help the user unsubscribe, so the offer to block the sender instead. Like in these two messages.
When the List-Unsubscribe header uses a mailto: link, Microsoft uses completely different language in the popup and does let the user know any future mail will go to the junk folder.

It's not a technical problem

You can’t technical your way out of the bulk folder. I wrote that a year and a half ago, and it’s even more true today. Filters at the big webmail providers continue to evolve to meet new threats and new spamming techniques. Sending technically perfect mail won’t get your mail into the inbox. Recipients have to want the mail and interact with the mail for good delivery.

Permission and B2B spam

Two of the very first posts I wrote on the blog were about permission (part 1, part 2). Re-reading those posts is interesting. Experience has taught me that recipients are much more forgiving of implicit opt-in than that post implies.
The chance in recipient expectations doesn’t mean, however, that permission isn’t important or required. In fact, The Verge reported on a chatbot that will waste the time of spammers. Users who are fed up with spam can forward their message to Re:Scam and bots will answer the mail.
I cannot tell you how tempted I am to forward all those “Hey, just give me 10 minutes of your time…” emails I get from B2B spammers. I know, those are actually bots, but there is lovely symmetry in bots bothering one another and leaving us humans out of it.

Speaking of those annoying emails, I tweeted about one (with horrible English…) last week. I tagged the company in question and they asked for an example. After I sent it, they did nothing, and I continued to get mail. Because of course I did.
These types of messages are exactly why permission is so critical for controlling spam. Way more companies can buy my email address and add me to their spam automation software than I can opt-out of in any reasonable time frame. My inbox, particularly my business inbox, is where I do business. It’s where I talk with clients, potential clients, customers and, yes, even vendors. But every unsolicited email wastes my time.
It’s not even that the mail is simply unwanted. I get mail I don’t want regularly. Collecting white papers for my library, RSVPing to events, joining webinars all result in me getting added to companies’ mailing lists. That’s fair, I gave them an email address I’ll unsubscribe.
The B2B companies who buy my address are different. They’re spamming and they understand that. The vendors who sell the automation filters tell their customers how to avoid spam filters. Spammers are told to use different domains for the unsolicited mail and their opt-in mail to avoid blocking. The software plugs into Google and G Suite account because very few companies will block Google IPs.
I’ve had many of these companies attempt to pay me to fix their delivery problems. But, in this case there’s nothing to fix. Yes, your mail is being blocked. No, I can’t help. There is nothing I can say to a filtering company or ISP or company to make them list that block. The mail is unwanted and it’s unsolicited.
The way to get mail unblocked is to demonstrate the mail is wanted. If you can’t do that, well, the filters are working as intended.

Filters do what we tell them

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

It's not fair

In the delivery space, stuff comes in cycles. We’re currently in a cycle where people are unhappy with spam filters. There are two reasons they’re unhappy: false positives and false negatives.
False positives are emails that the user doesn’t think is spam but goes into the bulk folder anyway.
Fales negatives are emails that the user does thing is spam but is delivered to the inbox.
I’ve sat on multiple calls over the course of my career, with clients and potential clients, where the question I cannot answer comes up. “Why do I still get spam?”
I have a lot of thoughts about this question and what it means for a discussion, how it should be answered and what the next steps are. But it’s important to understand that I, and most of my deliverability colleagues, hate this question. Yet we get it all the time. ISPs get it, too.
A big part of the answer is because spammers spend inordinate amounts of time and money trying to figure out how to break filters. In fact, back in 2006 the FTC fined a company almost a million dollars for using deceptive techniques to try and get into filters. One of the things this company did would be to have folks manually create emails to test filters. Once they found a piece of text that would get into the inbox, they’d spam until the filters caught up. Then, they’d start testing content again to see what would get past the filters. Repeat.
This wasn’t some fly by night company. They had beautiful offices in San Francisco with conference rooms overlooking Treasure Island. They were profitable. They were spammers. Of course, not long after the FTC fined them, they filed bankruptcy and disappeared.
Other spammers create and cultivate vast networks of IP addresses and domains to be used in snowshoeing operations. Still other spammers create criminal acts to hijack reputation of legitimate senders to make it to the inbox.
Why do you still get spam? That’s a bit like asking why people speed or run red lights. You still get spam because spammers invest a lot of money and time into sending you spam. They’re OK with only a small percentage of emails getting through filters, they’ll just make it up in volume.
Spam still exists because spammers still exist.

It depends… no more

The two most hated words in deliverability. Many people ask general questions about deliverability and most experts, including myself, answer, “It depends.”
There are a lot of problems with this answer. The biggest problem is that it’s led to the impression that there are no real answers about deliverability. That because we can’t answer hypothetical questions we are really just making the answers up.

The reason we use “it depends” is because the minute details matter when it comes to deliverability. Wether or not something will hurt or help deliverability depends on the specific implementation. Who’s doing the sending? What is their authentication setup? What IP are they using? How were the addresses collected? What is their frequency? What MTA is used? Are they linking to outside sites? Are they linking to outside services? Where are images hosted? The relevant questions go on and on and on.
I am going to stop saying it depends when answering generic deliverability questions. Instead I will be using the phrase “details matter.” Details do matter. Details are everything. Details drive deliverability.
Details Matter
The importance of details is why many deliverability people hedge their answers. The details do matter.
I will do my best to stop answering It Depends to deliverability questions. Instead, I’ll be answering with question and pointing out the details matter.

Parasites hurt email marketing

As a small business owner I am a ripe target for many companies. They buy my address from some lead generation firm, or they scrape it off LinkedIn, and they send me a message that pretends to be personalized but isn’t really.
“I looked at your website… we have a list of email addresses to sell you.”
“We offer cold calling services… can I set up a call with you?”
“I have scheduled a meeting tomorrow so I can tell you about our product that will solve all your technical issues and is also a floor wax.”
None of these emails are anything more than spam. They’re fake personalized. There’s no permission. On a good day they’ll have an opt out link. On a normal day they might include an actual name.
These are messages coming to an email address I’ve spent years trying to protect from getting onto mailing lists. I don’t do fishbowls, I’m careful about who I give my card to, I never use it to sign up for anything. And, still, that has all been for naught.
I don’t really blame the senders, I mean I do, they’re the ones that bought my address and then invested in business automation software that sends me regular emails trying to get me to give them a phone number. Or a contact for “the right person at your business to talk to about this great offer that will change your business.”
The real blame lies with the people who pretend that B2B spam is somehow not spam. Who have pivoted their businesses from selling consumer lists to business lists because permission doesn’t matter when it comes to businesses. The real blame lies with companies who sell “marketing automation software” that plugs into their Google Apps account and hijacks their reputation to get to the inbox. The real blame lies with list cleansing companies who sell list buyers a cleansing service that only hides the evidence of spamming.
There are so many parasites in the email space. They take time, energy and resources from large and small businesses, offering them services that seem good, but really are worthless.
The biologically interesting thing about parasites, though, is that they do better if they don’t overwhelm the host system. They have to stay small. They have to stay hidden. They have to not cause too much harm, otherwise the host system will fight back.
Email fights back too. Parasites will find it harder and harder to get mail delivered in any volume as the host system adapts to them. Already if I look in my junk folder, my filters are correctly flagging these messages as spam. And my filters see a very small portion of mail. Filtering companies and the business email hosting systems have a much broader view and much better defenses.
These emails annoy me, but I know that they are a short term problem. As more and more businesses move to hosted services, like Google Apps and Office365 the permission rules are going to apply to business addresses as well as consumer addresses. The parasites selling products and services to small business owners can’t overwhelm email. The defenses will step in first.

Barracuda problems

Folks were posting earlier today noticing problems delivering to Barracuda hosted services. The good news is Barracuda has been updating their status page. As of now, the status page says things are improving.

Ask Laura: Should we allow tagged addresses?

April 2016: The Month in Email

We are finishing up another busy month at WttW. April was a little nutty with network glitches, server crashes, cat woes, and other disruptions, but hopefully that’s all behind us as we head into May. I’ll be very busy in May as well, speaking at Salesforce Connections in Atlanta and the Email Innovation Summit in Las Vegas. Please come say hello if you’re attending either of these great events.

Speaking of great events, I participated in two panels at EEC16 last month. We had a lot of great audience participation, and I met many wonderful colleagues. I wrote up some more thoughts about the conference here. I also had a nice conversation with the folks over at Podbox, and they’ve posted my interview on their site.
In the Podbox interview, as always, I talked about sending mail people want to receive. It always makes me roll my eyes a bit when I see articles with titles like “5 Simple Ways to Reach the Inbox”, so I wrote a bit about that here. In addition to sending mail people want to receive, senders need to make sure they are collecting addresses and building lists in thoughtful and sustainable ways. For more on this topic, check out my post on list brokers and purchased lists.
These same not-so-simple tricks came up again in my discussion of Gmail filters. Everyone wants a magic formula to reach the inbox, and — sorry to burst your bubble — there isn’t ever going to be one. And this is for a good reason: a healthy filter ecosystem helps protect all of us from malicious senders and criminal activity. The email channel is particularly vulnerable to fraud and theft. The constant evolution of filters is one way mail providers can help protect both senders and recipients — but it can be challenging for senders and systems administrators to keep up with this constant evolution. For example, companies sometimes even inadvertently filter their own mail!
I also wrote a bit about how B2B spam is different from B2C spam, and how marketers can better comply with CAN SPAM guidelines in order to reach the inbox. We also republished our much-missed friend and colleague J.D. Falk’s DKIM Primer, which is extremely useful information that was at a no-longer-active link.
One of my favorite posts this month was about “dueling data”, and how to interpret seemingly different findings around email engagement. We also got some good questions for my “Ask Laura” column, where we cover general topics on email delivery. This month we looked at “no auth/no entry” and the Microsoft Smartscreen filter, both of which are useful things to understand for optimizing delivery.
Finally, we are pleased to announce that we’ve joined the i2Coalition, an organization of internet infrastructure providers. They posted a nice introduction on their blog, and we look forward to working with them to help advocate and protect these important technical infrastructures.

Insight into Gmail filtering

Last week I posted a link to an article discussing how Gmail builds defenses to protect their users from malicious mail. One of the things I found very interesting in that article was the discussion about how Gmail deploys many changes at once, to prevent people from figuring out what the change was.
Let’s take a look at what Gmail said.

Thoughts on filters

One of the questions we received during the EEC16 closing keynote panel was why isn’t there a single blocklist that everyone uses and why don’t ISPs share data more. It would be so much easier for senders if every ISP handled mail the same as every other. But the world isn’t that simple, and it’s not always clear which mail stream is spam and which is good mail.

Purchased lists and ESPs: 9 months later

It was about 8 months ago I published a list of ESPs that prohibit the use of purchased lists. There have been a number of interesting responses to that post.

ESPs wanted to be added to the list
The first iteration of the list was crowdsourced from different ESP representatives. They shared the info they had with each other. With their permission, I put it together into a post and published it here. Since then, I’ve had a trickle of ESPs asking to be added to the list. I’m happy to add any ESP. The only requirement is a privacy policy (or AUP) that states no purchased lists.
People reference the list regularly
I’ve had a lot of ESP deliverability folks send thanks for writing this post. They tell me they reference it regularly when dealing with clients. It’s also been listed as “one of the best blog posts of 2015” by Pardot.
Some 2016 predictions build on the post
I’ve read multiple future predictions that talk about how the era of purchased lists is over. I don’t think they’re wrong. I think that purchased lists are going to be deliverability nightmares on an internet where users wanting a mail is a prime factor in inbox deliverability. They’re already difficult to deliver, but it’s going to get worse.

Not everyone thinks this is a good post. In fact, I just recently got an comment about how wrong I was, and… well, I’ll just share it because I don’t think my summary of it will do it any justice.

Troubleshooting delivery is hard, but doable

Even for those of us who’ve been around for a while, and who have a lot of experience troubleshooting delivery problems things are getting harder. It used to be we could identify some thing about an email and if that thing was removed then the email would get to the inbox. Often this was a domain or a URL in the message that was triggering bulk foldering.
Filters aren’t so simple now. And we can’t just randomly send a list of URLs to a test account and discover which URL is causing the problem. Sure, one of the URLs could be the issue, but that’s typically in context with other things. It’s rare that I can identify the bad URLs sending mail through my own server these days.
There are also a lot more “hey, help” questions on some of the deliverability mailing lists. Most of these questions are sticky problems that don’t map well onto IP or domain reputation.
One of my long term clients recently had a bad mail that caused some warnings at Gmail.
We tried a couple of different things to try and isolate the problem, but never could discover what was triggering the warnings. Even more importantly, we weren’t getting the same results for identical tests done hours apart. After about 3 days, all the warnings went away and all their mail was back in the inbox.
It seemed that one mailing was really bad and resulted in a bad reputation, temporarily. But as the client fixed the problem and kept mailing their reputation recovered.
Deliverability troubleshooting is complicated and this flowchart sums up what it’s like.

Here at Word to the Wise, we get a lot of clients who have gone through the troubleshooting available through their ESPs and sometimes even other deliverability consultants. We get the tough cases that aren’t easy to figure out.
What we do is start from the beginning. First thing is to confirm that there aren’t technical problems, and generally we’ll find some minor problems that should be fixed, but aren’t enough to cause delivery problems. Then we look at the client’s data. How do they collect it? How do they maintain it? What are they doing that allows false addresses on their list?
Once we have a feel for their data processes, we move on to how do we fix those processes. What can we do to collect better, cleaner data in the future? How can we improve their processes so all their recipients tell the ISP that this is wanted mail?
The challenging part is what to do with existing data, but we work with clients individually to make sure that bad addresses are expunged and good addresses are kept.
Our solutions aren’t simple. They’re not easy. But for clients who listen to us and implement our recommendations it’s worth it. Their mail gets into the inbox and deliverability becomes a solved problem.

November 2015: The month in email

As we head into the last month of the year, we look back at our November adventures. I spoke twice this month, first at Message Systems Insight in Monterey (my wrap-up post is here) and then with Ken Magill at the at the 2015 All About eMail Virtual Conference & Expo (a short follow-up here, and a longer post on filters that came out of that discussion here.). Both were fun and engaging — it’s always great to get a direct sense of what challenges are hitting people in the email world, and to help clear up myths and misconceptions about what works and doesn’t work in email marketing and delivery. I’m putting together my conference and speaking schedule for 2016 — if you know of anything interesting that should be on my radar, please add it in the comments, thanks!
In industry news, we noted a sharp uptick in CBL listings, and then posted about the explanation for the false positives. Steve wrote about an interesting new Certificate Authority (CA) called Let’s Encrypt, which looks to be a wonderful (and much-needed) alternative for certificates, and I put together some thoughts on SenderScore.
Steve and I did a few posts in parallel this month. First, Steve posted an interesting exercise in SPF debugging. Are you seeing mail from legitimate senders flagged as spam? This might be why. My investigative post was about ISP rejections, and how you can figure out where the block is occurring. In each case, you’ll get a glimpse of how we go about identifying and troubleshooting issues, even when we don’t have much to go on.
We each also wrote a bit about phishing. Steve posted a timely warning about spear phishing — malware attacks disguised as legitimate email from within your organization — and reminds all of us to be careful about attachments. With all of the more secure options for document sharing these days, it’s a lot easier to avoid the risk by maintaining a no-attachments policy in your company. And I wrote about how the Department of Defense breaking HTML links in email to help combat phishing. If your lists include military addresses (.mil), you may want to come up with a strategy for marketing to those recipients that relies less on a clickthrough call to action.
We amused ourselves a bit with a game of Deliverability Bingo, then followed up with a more serious look at the thing we hear all the time — “I’m sure they’ll unblock me if I can just explain my business model.” While an ESP abuse desk is unlikely to be swayed by this strategy, it is actually at the core of how we think about deliverability at Word to the Wise. Legitimate senders have many kinds of lists, many kinds of recipients, many kinds of marketing strategies, and many kinds of business goals. For us to help marketers craft sustainable email programs, we need to understand exactly what matters most to our clients.

When did the reject happen?

Earlier today I approved a comment from Mike on a post about problems at AOL from 2012. The part of the comment that caught my attention:

DOD breaks links in .mil clients

The Department of Defense is breaking HTML links in mail to .mil domains. This is part of the DoD’s attempt to curtail phishing.

Filter complexity

During the Q&A last week, I mentioned an example of a type of filter trying to demonstrate how complex the filters are. There was some confusion about what I was saying, so I thought I’d write a blog post explaining this.

Thanks for the great session

I had a great time answering questions at the 2015 All About eMail Virtual Conference & Expo today. Thanks so much to everyone who participated and asked questions. They were great and I’m sorry we didn’t have more time.
I did get some questions on twitter (@wise_laura) afterwards. One was about an example I gave to explain how filters are complex. There have been rumors going around recently that Gmail is filtering mail with more than 3 URLs in it. Let me just say right now THIS IS NOT TRUE emails with more than 3 URLs in them are being delivered just fine to Gmail.
There is a situation involving the number (and type) of URLs that I think are a useful example of the filter complexity happening at some places, like Gmail. I started working on it, but don’t quite have time to finish it today, but will keep working on and it should go up in the next day or so.
Thanks again to everyone who joined the session. You asked some great questions and I had fun answering them.

Do system administrators have too much power?

Yesterday, Laura brought a thread from last week to my attention, and the old-school ISP admin and mail geek in me felt the need to jump up and say something in response to Paul’s comment. My text here is all my own, and is based upon personal experience as well as those of my friends. That said, I’m not speaking on their behalf, either. 🙂
I found Paul’s use of the word ‘SysAdmin’ to be a mighty wide (and — in my experience — probably incorrect) brush to be painting with, particularly when referring to operations at ISPs with any significant number of mailboxes. My fundamental opposition to use of the term comes down to this: It’s no longer 1998.
The sort of rogue (or perhaps ‘maverick’) behavior to which you refer absolutely used to be a thing, back when a clean 56k dial-up connection was the stuff of dreams and any ISP that had gone through the trouble to figure out how to get past the 64k user limit in the UNIX password file was considered both large and technically competent. Outside of a few edge cases, I don’t know many system administrators these days who are able to (whether by policy or by access controls) — much less want to — make such unilateral deliverability decisions.
While specialization may be for insects, it’s also inevitable whenever a system grows past a certain point. When I started in the field, there were entire ISPs that were one-man shows (at least on the technical side). This simply doesn’t scale. Eventually, you start breaking things up into departments, then into services, then teams assigned to services, then parts of services assigned to teams, and back up the other side of the mountain, until you end up with a whole department whose job it is to run one component of one service.
For instance, let’s take inbound (just inbound) email. It’s not uncommon for a large ISP to have several technical teams responsible for the processing of mail being sent to their users:

Politics and Delivery

Last week I posted some deliverability advice for the DNC based on their acquisition of President Obama’s 2012 campaign database. Paul asked a question on that post that I think is worth some attention.

Pattern matching primates

Why do we see faces where there are none? Paradolia
Why do we look at random noise and see patterns? Patternicity
Why do we think we have discovered what’s causing filtering if we change one thing and email gets through?
It’s all because we’re pattern matching primates, or as Michael Shermer puts it “people believe weird things because of our evolved need to believe nonweird things.”
Our brains are amazing and complex and filter a lot of information so we don’t have to think of it. Our brains also fill in a lot of holes. We’re primed at seeing patterns, even when there’s no real pattern. Our brains can, and do, lie to us all the time. For me, some of the important part of my Ph.D. work was learning to NOT trust what I thought I saw, and rather to effectively observe and test. Testing means setting up experiments in different ways to make it easier to not draw false conclusions.
Humans are also prone to confirmation bias: where we assign more weight to things that agree with our preconceived notions.
Take the email marketer who makes a number of changes to a campaign. They change some of the recipient targeting, they add in a couple URLs, they restructure the mail to change the text to image ratio and they add the word free to the subject line. The mail gets filtered to the bulk folder and they immediately jump to the word free as the proximate cause of the filtering. They changed a lot of things but they focus on the word free.
Then they remove the word free from the subject line and all of a sudden the emails are delivering. Clearly the filter in question is blocking mail with free in the subject line.
Well, no. Not really. Filters are bigger and more complex than any of us can really understand. I remember a couple years ago, when a few of my close friends were working at AOL on their filter team. A couple times they related stories where the filters were doing things that not even the developers really understood.
That was a good 5 or 6 years ago, and filters have only gotten more complex and more autonomous. Google uses an artificial neural network as their spam filter. I don’t really believe that anything this complex just looks at free in the subject line and filters based on that.
It may be that one thing used to be responsible for filtering, but those days are long gone. Modern email filters evaluate dozens or hundreds of factors. There’s rarely one thing that causes mail to go to the bulk folder. So many variables are evaluated by filters that there’s really no way to pinpoint the EXACT thing that caused a filter to trigger. In fact, it’s usually not one thing. It could be any number of things all adding up to mean this may not be mail that should go to the inbox.
There are, of course, some filters that are one factor. Filters that listen to p=reject requests can and do discard mail that fails authentication. Virus filters will often discard mail if they detect a virus in the mail. Filters that use blocklists will discard mail simply due to a listing on the blocklist.
Those filters address the easy mail. They leave the hard decisions to the more complex filters. Most of those filters are a lot more accurate than we are at matching patterns. Us pattern matching primates want to see patterns and so we find them.

4 things spammers do legitimate marketers don't

I’ve never met a spammer that claims to be a spammer. Most that I’ve met claim to be legitimate marketers (or high volume email deployers). But there are things spammers do that I never expect to see a legitimate marketer doing.
I’ve written about these things throughout the blog (tag: TWSD), but it’s probably time to actually pull them together into a single post.

What to do when an important email bounces

Some emails are more important than others. I know, I know, all emails are important, but really, some are more important than others.
I’ve recently been decluttering by the simple expedient of enrolling in paperless statements for some of our accounts. We have a 1TB NAS, I’m not going to run out of storage space and I will have so much less paper to deal with. Plus, electronic searches are easier than digging through a file I’ve just shoved statements in for the whole year.
Some companies just let you sign up for statements online and don’t take any extra steps to verify your email address or tell you what happens if your email breaks. But at least one company has gone the extra mile to establish how they handle email bounces.
consentforpaperlessdocs
First, to sign up for paperless notifications I have to give my consent to receive docs. Even better, when I look at the important information it expressly details what happens if my email address bounces.

Thoughts on Gmail filtering

Gmail has some extremely complex filters. They’re machine learning based and measure hundreds of things about incoming mail. The filters are continually adjusting to changes and updating how they treat specific mail.
One consequence of continually adjusting machine learning filters is that filtering is not static. What passes to the inbox now, may not pass in a couple hours.
One of the other challenges with Gmail filters is that they look at all the mail mentioning a particular domain and so affiliate mail and 3rd party mail can affect delivery of corporate mail.
The good news is that continually adjusting filters adapt to positive changes as well as negative ones. In fact, I recently made a segmentation suggestion to a client and they saw a significant increase in inbox delivery at Gmail the next day.
Gmail can be a challenge for delivery, but send mail users want and mail does go to the inbox.

When spam filters fail

Spam filters aren’t perfect. They sometimes catch mail they shouldn’t, although it happens less than some people think. They sometimes fail to catch mail they should.
One of the reason filters fail to catch mail they should is because some spammers invest a lot of time and energy in figuring out how to get past the filters. This is nothing new, 8 or 9 years ago I was in negotiations with a potential client. They told me they had people who started working at 5pm eastern. Their entire job was to craft mail that would get through Hotmail’s filters that day. As soon as they found a particular message that made it to the inbox, they’d blast to their list until the filters caught up. When the filters caught up, they’d start testing again. This went on all night or until the full list was sent.
Since then I’ve heard of a lot of other filter bypass techniques. Some spammers set up thousands of probe accounts at ISPs and would go through and “not spam” their mail to fool the filters (ISPs adapted). Some spammers set up thousands of IPs and rotate through them (ISPs adapted). Some spammers register new domains for every send (ISPs adapted). Some spammers used botnets (ISPs adapted)
I’m sure, even now, there are spammers who are creating new techniques to get through filters. And the ISPs will adapt.

Language as filtering criteria

A few months ago I was working on a delivery audit for a client who sends mail in multiple languages. We discovered that the language of an email has a significant delivery impact. The same email in different languages was delivered differently, particularly at Gmail. Emails in a language I don’t normally receive email in were delivered to my bulk folder.
Other folks have commented on similar things. Some filters really do look at preferred language of the recipient and treat mail in other languages as problematic. I don’t think that’s unreasonable. I do get a lot of foreign language spam and there’s no real way to stop it. Many countries don’t require opt-out links, and so there isn’t a clear way to even unsubscribe.
Writing in the recipient’s local language is one way to minimize inappropriate blocking, even when you have permission to send mail.

AHBL Wildcards the Internet

AHBL (Abusive Host Blocking List) is a DNSBL (Domain Name Service Blacklist) that has been available since 2003 and is used by administrators to crowd-source spam sources, open proxies, and open relays. By collecting the data into a single list, an email system can check this blacklist to determine if a message should be accepted or rejected. AHBL is managed by The Summit Open Source Development Group and they have decided after 11 years they no longer wish to maintain the blacklist.
A DNSBL works like this, a mail server checks the sender’s IP address of every inbound email against a blacklist and the blacklist responses with either, yes that IP address is on the blacklist or no I did not find that IP address on the list. If an IP address is found on the list, the email administrator, based on the policies setup on their server, can take a number of actions such as rejecting the message, quarantining the message, or increasing the spam score of the email.
The administrators of AHBL have chosen to list the world as their shutdown strategy. The DNSBL now answers ‘yes’ to every query. The theory behind this strategy is that users of the list will discover that their mail is all being blocked and stop querying the list causing this. In principle, this should work. But in practice it really does not because many people querying lists are not doing it as part of a pass/fail delivery system. Many lists are queried as part of a scoring system.
Maintaining a DNSBL is a lot of work and after years of providing a valuable service, you are thanked with the difficulties with decommissioning the list. Popular DNSBLs like the AHBL list are used by thousands of administrators and it is a tough task to get them to all stop using the list. RFC6471 has a number of recommendations such as increasing the delay in how long it takes to respond to a query but this does not stop people from using the list. You could change the page responding to the site to advise people the list is no longer valid, but unlike when you surf the web and come across a 404 page, a computer does not mind checking the same 404 page over and over.
Many mailservers, particularly those only serving a small number of users, are running spam filters in fire-and-forget mode, unmaintained, unmonitored, and seldom upgraded until the hardware they are running on dies and is replaced. Unless they do proper liveness detection on the blacklists they are using (and they basically never do) they will keep querying a list forever, unless it breaks something so spectacularly that the admin notices it.
So spread the word,

Email problems are costly

Last week Zulily released their quarterly earnings. Their earnings’ report was disappointing, resulting in a drop in their stock prices. The chairman of the company told reporters on a conference call that part of the reason for the drop in earnings were due to deliverability problems “at a large ISP.”

ISP filters are good for marketers

A throwback post from 2010 Attention is a limited resource.
Marketing is all about grabbing attention. You can’t run a successful marketing program without first grabbing attention. But attention is a limited resource. There are only so many things a person can remember, focus on or interact with at any one time.
In many marketing channels there is an outside limit on the amount of attention a marketer can grab. There are only so many minutes available for marketing in a TV or radio hour and they cost real dollars. There’s only so much page space available for press. Billboards cost real money and you can’t just put a billboard up anywhere. With email marketing, there are no such costs and thus a recipient can be trivially and easily overwhelmed by marketers trying to grab their attention.
Whether its unsolicited email or just sending overly frequent solicited email, an overly full mailbox overwhelms the recipient. When this happens, they’ll start blocking mail, or hitting “this is spam” or just abandoning that email address. Faced with an overflowing inbox recipients may take drastic action in order to focus on the stuff that is really important to them.
This is a reality that many marketers don’t get. They think that they can assume that if a person purchases from their company that person wants communication from that company.

Content based filtering

Content filtering is often hard to explain to people, and I’m not sure I’ve yet come up with a good way to explain it.
A lot of people think content reputation is about specific words in the message. The traditional content explanation is that words like “Free” or too many exclamation points in the subject line are bad and will be filtered. But it’s not the words that are the issue it’s that the words are often found in spam. These days filters are a lot smarter than to just look at individual words, they look at the overall context of the message.
ISP_tolerances
Even when we’re talking content filters, the content is just a way to identify mail that might cause problems. Those problems are evaluated the same way IP reputation is measured: complaints, engagement, bad addresses. But there’s a lot more to content filtering than just the engagement piece. What else is part of content evaluation?

Ever changing filtering

One of the ongoing challenges sending email, and managing a high volume outbound mail server is dealing with the ongoing changes in filtering. Filters are not static, nor can they be. As ISPs and filtering companies identify new ways to separate out wanted email from unwanted email, spammers find new ways to make their mail look more like wanted mail.
This is one reason traps are useful to filtering companies. With traps there is no discussion about whether or not the mail was requested. No one with any connection to the email address opted in to receive mail. The mail was never requested. While it is possible for trap addresses to get on any list monitoring mail to spam traps is a way to monitor which senders don’t have good practices.
New filtering techniques are always evolving. I mentioned yesterday that Gmail was making filtering changes, and that this was causing a lot of delivery issues for senders. The other major challenge for Gmail is the personalized delivery they are doing. It’s harder and harder for senders to monitor their inbox delivery because almost every inbox is different at Gmail. I’ve seen different delivery in some of my own mailboxes at Gmail.
All of this makes email delivery an ongoing challenge.

Delivering to Gmail

Gmail is a challenge for even the best senders these days.
With the recent Gmail changes there isn’t any clear fix to getting open rates or inbox delivery back up. Some of it depends on what is causing Gmail to filter the mail. Changing subject lines, from name, from address may get mail back to the inbox in the short term, but it only works until the filters catch up.
What I am seeing, across a number of clients, is that Gmail is doing a lot of content reputation and that content reputation gets spread across senders of that content. That means you want to look at who is sending any mail on your behalf (mentioning your domain or pointing at your website) and their practices. If they have poor practices, then it can reflect badly on you and result in filtering.
From what I’ve seen, these are very deliberate filtering decisions by Google. And it’s making mail a lot harder for many, many senders. But I think it is, unfortunately, the new reality.

Spam filters and mailbox usage

It’s no secret that I run very little in the way of spam filters, and what filters I do run don’t throw away mail, they just shove it into various mailboxes.
Looking at my mailboxes currently I have 11216 unread messages in my mail.app junk folder, 10600 unread messages in my work spam assassin folder and 29401 messages in my personal spam assassin folder (mail getting more than +7 on our version of spam assassin gets filtered into these folders). I went through and marked all of my messages read back in mid-January. That’s a little over 50,000 messages in a little over 5 months or slightly more than 2700 spams a week.
But these are messages I don’t have to deal with so while they’re somewhat annoying and a bit of “wow, my addresses are everywhere” they’re not a huge deal. I have strong enough filters for wanted mail that I can special case it.

Abuse it and lose it

Last week I blogged about the changes at ISPs that make “ISP Relations” harder for many senders. But it’s not just ISPs that are making it a little more difficult to get answers to questions, some spam filtering companies are pulling back on offering support to senders.
For instance, Cloudmark sent out an email to some ESPs late last week informing them that Cloudmark was changing their sender support policies. It’s not that they’re overwhelmed with delisting requests, but rather that many ESPs are asking for specific data about why the mail was blocked. In December, Spamcop informed some ESPs that they would stop providing data to those ESPs about specific blocks and spam trap hits.
These decisions make it harder for ESPs to identify specific customers and lists causing them to get blocked. But I understand why the filtering companies have had to take such a radical step.
Support for senders by filtering companies is a side issue. Their customers are the users of the filtering service and support teams are there to help paying customers. Many of the folks at the filtering companies are good people, though, and they’re willing to help blocked senders and ESPs to figure out the problem.
For them, providing information that helps a company clean up is a win. If an ESP has a spamming customer and the information from the filtering company is helping the ESP force the customer to stop spamming that’s a win and that’s why the filtering companies started providing that data to ESPs.
Unfortunately, there are people who take advantage of the filtering companies. I have dozens of stories about how people are taking advantage of the filtering companies. I won’t share specifics, but the summary is that some people and ESPs ask for the same data over and over and over again. The filtering company rep, in an effort to be helpful and improve the overall email ecosystem, answers their questions and sends the data. In some cases, the ESP acts on the data, the mail stream improves and everyone is happy (except maybe the spammer). In other cases, though, the filtering company sees no change in the mail stream. All the filtering company person gets is yet another request for the same data they sent yesterday.
Repetition is tedious. Repetition is frustrating. Repetition is disheartening. Repetition is annoying.
What we’re seeing from both Spamcop and Cloudmark is the logical result from their reps being tired of dealing with ESPs that aren’t visibly fixing their customer spam problems. Both companies are sending some ESPs to the back of the line when it comes to handling information requests, whether or not those ESPs have actually been part of the problem previously.
The Cloudmark letter makes it clear what they’re frustrated about.

Thoughts on "ISP relations"

I’ve been thinking a lot about the field of ISP relations and what it means and what it actually is. A few years ago the answer was pretty simple. ISP relations is about knowing the right people at ISPs in order to get blocks lifted.
The fact that ISPs had staff just to deal with senders was actually a side effect of their anti-spam efforts. In many places blocking was at least partially manual, so there had to be smart, technical, talented people to handle both the blocking and unblocking. That meant there were people to handle sender requests for unblocking.
Spam filters have gotten better and more sophisticated. Thus, the ISPs don’t need smart, technical, talented and expensive people in the loop. Most ISPs have greatly scaled back their postmaster desks and rely on software to handle much of the blocking.
Another issue is that some people on the sender side rely too heavily on the ISPs for their data. This makes the ISP reps, and even some spam filtering company reps, reluctant to provide to much help to senders. I’ve had at least 3 cases in the last 6 months where a sender contacted me to tell me they had spoken with someone at an ISP or filtering company and were told they would get no more help on a particular issue. In talking with those reps it was usually because they were drowning under sender requests and had to put some limits on senders.
All of this means ISP Relations is totally different today than it was 5 years ago. It’s no longer about knowing the exact right person to contact. Rather it’s about being able to identify problems without ISP help. Instead of being able to ask someone for information, ISP Relations specialists need to know how to find data from different sources and use that data to identify blocking problems. Sure, knowing the right person does help in some cases when there’s an obscure and unusual issue. But mostly it’s about putting together any available evidence and then creating a solution.
We still call it “ISP Relations” but at a lot of ISPs there is no one to contact these days. I think the term is a little misleading, but it seems to be what we’re stuck with.

Looking for message labs help?

There’s a common bounce error from the Message Labs’ filtering appliance that goes no where.

Changes at Spamcop

Earlier this week some ESPs started asking if other ESPs have seen an uptick in Spamcop listings. The overwhelming answer (9 of 11 ESP representatives) said yes. I’ve also had clients start to ask me about Spamcop listings. All in all, there seems to be some changes at Spamcop that means more senders are showing up on the Spamcop radar.
Luckily, Spamcop provides us some insight into their data processing. If you look at the current monthly volume graph, we can see some very interesting changes in data.

Barracuda clicking all links in emails

A number of people have asked me recently if I know anything about appliances clicking all the links in emails. Some of those people have asked specifically about Barracuda, some have just asked if I knew of any filters that clicked links.
The answer is, yes, there are cases where spam filters have followed all the links in an email. One of the filters that I know has done this in the past is Barracuda. Based on discussions with the different people who are reporting this behavior, it does seem that this is happening more often. One person did mention that they were primarily seeing this with mail where the click domains were different from the From: domains.
I’m still working on getting more information from folks, and will update if I hear anything more. I’m also working on some advice for folks who get caught in this.
If you have experience with Barracuda (or other spam filters) clicking all the links in an email, drop me an email (contact)

Uploading your address book to social media

I am one of the moderators of a discussion list working on a document about getting off blocklists. If anyone not on the list attempts to post to the list I get a moderation request. One came through while I was gone.
Now, I don’t really think Jim Mills wants to be friends with a mailing list. I think he probably gave LinkedIn his email password and LinkedIn went through and scraped addresses out of his address book and sent invitations to all those addresses.
I don’t have any problem with connecting to people on social media. I do even understand that some people have no problem giving their passwords over to let social media sites plunder their address books and find connections. What I do have a problem with is social media sites that don’t do any pruning or editing of the scraped addresses before sending invitations.
In this case, the email address, like many mailing lists, has in the email address “mailman.” While it’s probably impossible to weed out every mailing list, support address and commercial sender, it doesn’t seem like it would be too difficult to run some minor word matching and filtering. It’s not even like those addresses have to be removed from invites. Instead they could be presented to the user for confirmation that these are real people and addresses.
Yes, it’s friction in the transaction and it costs money to do and do well. But those costs and friction are currently offloaded onto uninvolved third parties.

Filtering is not just about spam

A lot of filters started out just as filters against spam. But over the years they’ve morphed into more general blocks against dangerous or problematic email. There’s a lot of crime and bad behavior on the internet, much of it using email as a conduit or vector. Filtering is so much more than stopping spam now. It’s as much, or more, about stopping crime.
Email filters are essential to protect us from scammers. Sometimes I forget this, and then I read about a grandmother getting swindled by a Nigerian scammer and ending up dead.
There are real consequences to poor filtering and there is real crime facilitated by email. It’s easy to forget this as we deal with the email that gets caught in filters when they shouldn’t.
Filters are one of the first lines of defense against online crime.
Not only does filtering stop crime, but they also keep email working. An unfiltered mail stream is an ugly, unreadable, unworkable mess.

TWSD: Adapt to filters

This morning the new Yahoo! CEO posted about changes to Yahoo! mail. I logged into one of my Yahoo accounts to check and see if I had access to the new Yahoo! mail client yet. I don’t, but I did notice that spammers have adapted to the new Yahoo model of disabling filters in the mail folder. Most of the mail in my inbox has, at the very top of the message “Click not spam to enable links!”
My favorite has to be the animated gif of how to click “not spam.”
Spammers spend so much time and energy compensating for filters, hopping IP addresses, rotating through domains, and specially creating mail for different ISPs. I have to wonder, though, if they would waste less time by sending opt-in mail.

Check your unsubscribe process

When was the last time you actually tried to unsubscribe from one of your mailing lists? Have you ever even checked to see that your process works?
For whatever reason, unsubscribe processes don’t always work. Sometimes the problem is on the client end. Sometimes the problem is on the ESP end. But in either case, continuing to mail recipients who have attempted to opt-out from your mail is a recipe for disaster.
I mentioned last week about our new mortgage company that can’t process my unsubscribe. Today I contacted their ESP and pointed out I’d tried to unsub a few times, but was still getting mail. The ESP thanked me, pointed out that was not an ESP managed unsubscribe page and did a little digging. A few hours later their delivery guy told me that he saw my multiple unsubscribe attempts (June, July, 2 in August…) and they were all marked as “trashed.” But he’s going to make sure I’m not mailed any more and follow up with his customer.
Now, there are a lot of reasons this unsub process could have failed. It could be that the website doesn’t handle my tagged addresses well and this is a bank, it’s very possible security is locked down. But that means they shouldn’t have accepted my tagged address in the first place.
There are a couple things to take away from this story.

You can't technical yourself out of delivery problems

In many cases these days, many more cases than a lot of senders want to admit, delivery problems at the big ISPs are a result of sending mail recipients just don’t care about. The reason your mail is going to bulk? It’s not because you have minor problems in your headers. It’s not because you have some formatting issues. The reason is because your recipients just don’t care if the ISP delivers your mail or not.
A few years ago the bulk of my clients hired me to do technical audits for their mail. I fixed a lot of delivery problems that way. They’d send me their email and I’d run it through tools here and identify things they were doing that were likely to be causing problems. I’d give them some suggestions of things to change. Believe it or not, minor tweaks to headers and configuration actually did make a lot of difference in delivery.
Over time, though those tweaks less effective to fix delivery problems. Some of it is due to the MTA vendors, they’re a lot better at sending technically correct mail than they were before. There are also a lot more people giving good advice on the underlying structure and format of emails so senders can send technically clean email. I started seeing technically perfect emails from clients who were seeing major delivery problems.
There are a number of reasons that technical fixes don’t work like they used to. The short version, though, is that ISPs have dealt with much of the really blatant spam and they can focus more time and energy on the “grey mail”.
This makes my job a little harder. I can no longer just look at an email, maybe run it through some of our tools and provide a few suggestions that fix delivery problems. Delivery isn’t that simple any longer. Filters are really more focused on how the recipients react to mail. That means I need to know a lot more about a clients email program before I can even start to identify what might be causing the delivery issues.
I wish it were still so simple I could give minor technical tweaks that would appear to magically improve a client’s delivery. It was a lot simpler process then. But filters have evolved, and senders must evolve, too.

Working as intended

There’s a certain type of sender that thinks every ISP block or email delivered to the bulk folder is a false positive. They’re so sure that the filters aren’t actually supposed to catch their mail that they’ll spend any amount of money and do every possible thing to get their mail to the inbox.
The problem for these senders, though, is that their mail is exactly the type of mail filters are designed to catch. They’re sending mail without recipient permission. I’m not talking about the lists that get a few typos or problem addresses on them. I’m talking about senders that buy and trade mailing lists. I’m talking about senders that don’t believe they have to have permission to send mail.
This mail getting filtered is a sign that the filters are working as intended. They’re keeping the unsolicited email out.
A lot of us take for granted that all commercial mail, at least that isn’t selling fake watches or herbal viagra, is always sent with permission. But there’s an awful lot of mail out there that doesn’t even have a minor fig leaf of permission. Filters stop that mail. And senders have very little recourse when they do.

New Spamhaus lists

Spamhaus announced today they are publishing two new BGP feeds: Extended DROP and the Botnet C&C list. These lists are intended for use inside routers in order to stop all traffic to or from listed IP addresses. This is a great way to impact botnet traffic and hopefully will have a significant impact on virus infections and botnet traffic.
In other news I’ve been hearing rumbling about changes at Yahoo. It looks like they have changed their filters and some senders are feeling lots of pain because of it. It looks like senders with low to mid range reputations are most affected and are seeing more and more of their mail hit the bulk folder. This afternoon I’m hearing that some folks are seeing delivery improvements as Yahoo tweaks the changes.

Return Path on Content Filtering

Return Path have an interesting post up about content filtering. I like the model of 3 different kinds of filters, in fact it’s one I’ve been using with clients for over 18 months. Spamfiltering isn’t really about one number or one filter result, it’s a complex interaction of lots of different heuristics designed to answer the question: do recipients want this kind of mail?

More than just getting past the filters

I’ve been feeling a little philosophical lately. My thoughts are meandering a lot around the whys and the deeper issues surrounding stuff, including email. It means I’m a bit more distracted and less focused than usual. And more prone to pose questions than usual. This was part of the introspection that led me to write the motivating people post last week. I’m trying to figure out how to motivate volunteers in two different realms. And there’s always the question of how do I present a solution to clients in a way that motivates them to take my advice. Sure, I get paid either way, but I really like it when clients take my advice and see success.
There are other places this mental meandering is taking me.
I’m currently working on a project for a client. This particular client is struggling to get mail delivered to a very mobile business audience. In the target field, people change jobs regularly and email addresses can change multiple times a year. One of the things I’m working on for them is how to get email to the right people, that is the people who opted in, when their addresses change so frequently.
This is delivery consulting, but this project really brings home how much more there is to delivery than avoiding filters. Filters are the least of this client’s problem. The real problem is the mobility of their audience. As I was thinking about how to address this issue of mobility, I realized that my job as a delivery expert has gone well beyond telling people how to get their mail past filters.
My job is much more about helping people succeed at what it is that they’re trying to do with email. How can email work for you and for your target audience?
Looking at the broader picture means I’m less likely to focus on the minutia of “spam words” and subject lines and best time of day to send. Sure, there are always tweaks to make in an email. There are always things to test. There are always changes to try. But the effect of those changes is not near as great as actually sending mail that meets the needs of the audience.
Often clients come to me so overwhelmed in the details they forget the bigger picture. I help them find that picture again. My job is much more than getting through the filters, it’s about finding success for clients.

Filters and windmills

A colleague of mine was dealing with a client who is experiencing some difficulty delivering to the bulk folder. Said client spent much of a one hour phone call repeating “This is not how a free society works!!”
After the call my colleague commented, “I refuse to get ranty about filter systems.”
I know that filters, and the people who write and maintain them, are a frequent scapegoat for senders. The filters are always the problem, not anything the senders do.
Now, I’ll be the last person who will claim spam filters are perfect, they’re not. Filters sometimes do unexpected things, sometimes they do boneheaded things, sometimes they are broken.
We can’t forget, though, that filters perform a vital role in protecting users from malicious emails. Phishing emails, scams, fake products, viruses are a constant threat. Many end users don’t need to worry about this because filters are so good. But an unfiltered account can get thousands of scams and spams a day (ask me how I know).
Most of us in the delivery space can tell when a filter is working as intended and when there’s an underlying problem. And when the filter is working as intended there’s not a lot of use complaining about them. Ranting about filtering systems often delays a resolution. Senders that focus on what they can control tend to have more success reaching the inbox than those senders that focus on ranting about filtering systems.
Tilting at windmills doesn’t get the mail through.

Spamhaus rising?

Ken has a good article talking about how many ESPs have tightened their standards recently and are really hounding their customers to stop sending mail recipients don’t want and don’t like. Ken credits much of this change to Spamhaus and their new tools.

Content, trigger words and subject lines

There’s been quite a bit of traffic on twitter this afternoon about a recent blog post by Hubspot identifying trigger words senders should avoid in an email subject line. A number of email experts are assuring the world that content doesn’t matter and are arguing on twitter and in the post comments that no one will block an email because those words are in the subject line.
As usually, I think everyone else is a little bit right and a little bit wrong.
The words and phrases posted by Hubspot are pulled out of the Spamassassin rule set. Using those words or exact phrases will cause a spam score to go up, sometimes by a little (0.5 points) and sometimes by a lot (3+ points). Most spamassassin installations consider anything with more than 5 points to be spam so a 3 point score for a subject line may cause mail to be filtered.
The folks who are outraged at the blog post, though, don’t seem to have read the article very closely. Hubspot doesn’t actually say that using trigger words will get mail blocked. What they say is a lot more reasonable than that.

Censorship, email and politics

Spamfiltering blocks email. This is something we all know and understand. For most people, that is everyone who doesn’t manage an email server or work in the delivery field or create spamfilters, filtering is a totally unseen process. The only time the average person notices filters is when they break. The breakage could be blocking mail they shouldn’t, or not blocking mail they should.
Yesterday, a bunch of people noticed that Yahoo was blocking mail containing references to a protest against Wall Street. This understandably upset people who were trying to use email as a communication medium. Many people decided it was Yahoo (a tool of the elites!) attempting to censor their speech and stop them from organizing a protest.
Yeah. Not so much.
Yahoo looked into it and reported that the mail had gotten caught in their spam filters. Yahoo adjusted their filters to let the mail through and all was (mostly) good.
I don’t think this is actually a sign of filters being broken. The blocked mail all contained a URL pointing to a occupywallst.com. I know there was a lot of speculation about what was being blocked, but sources tell me it was the actual domain. Not the phrase, not the text, the domain.
The domain was in a lot of mostly identical mail coming out of individual email accounts. This is a current hallmark of hijacked accounts. Spammers compromise thousands of email accounts, and send a few emails out of each of them. Each email is mostly identical and points to the same URL. Just like the protest mail.
There was also a lot of bulk mail being sent with that URL in it. I’ve been talking to friends who have access to traps, and they were seeing a lot of mail mentioning occupywallst.com in their traps. This isn’t surprising, political groups have some horrible hygiene. They are sloppy with acquisition, they trade names and addresses like kids trade cold germs, they never expire anything out. It’s just not how politics is played. And it’s not one party or another, it’s all of them. I’ve consulted with major names across the political spectrum, and none actually implement best practices.
As I have often said the secret to delivery is to not have your mail look like spam. In this case, the mail looked like spam. In fact, it looked like spam that was coming from hijacked accounts as well as spam sent by large bulk mailers. I suspect there was also a high complaint rate as people sent it to friends and family who really didn’t want to hear about the protests.
To Yahoo!’s credit, though, someone on staff was on top of things. They looked into the issue and the filter was lifted within a couple hours of the first blog post. A human intervened, overruled the algorithm and let the mail out.
I bet this is one of the few times anyone has seen that Yahoo does outbound filtering. Given it’s a politically charged situation, I can see why they assume that Yahoo is filtering because of politics and censorship. They weren’t though.
More on politics, filtering and censorship.

They’re not blocking you because they hate you
It really can be your email
More on Truthout
Another perspective on the politico article

Email filters

What makes the best email filter? There isn’t really a single answer to that question. Different people and different organizations have different tolerances for how false positives versus false negatives. For instance, we’re quite sensitive to false positives here, so we run extremely conservative filtering and don’t block very much at the MTA level. Other people I know are very sensitive to false negatives and run more aggressive filtering and block quite a bit of mail at the MTA level.
For the major ISPs, the people who plan, approve, design and monitor the filters usually want to maximize customer happiness. They want to deliver as much real mail as possible while blocking as much bad mail. Blocking real mail and letting through bad mail both result in unhappy customers and increase the ISP’s costs, either through customer churn or through support calls. And this is a process, filters are not static. ISPs roll out new filters all the time, sometimes they are an improvement and sometimes they’re not. When they’re not, they’re pulled out of production. This works both for positive filters like Return Path and negative filters like blocklists.
Then there is mail filtering that doesn’t have to do with spam. Business filters, for instance, often block non-business mail. Permission of the recipient often isn’t even a factor. Companies don’t often go out of their way to block personal mail, but if personal mail gets blocked (say the vacation plane ticket or the amazon receipt) they don’t often unblock it. But when you think about why a business provides email, it makes perfect sense. The business provides email to further its own business goals. Some personal usage is usually OK, but if someone notices and blocks personal email then it’s unlikely the business will unblock it, even if the employee opted in.
In the case of email filters, the free market does work. Different ISPs filter mail differently. Some people love Gmail’s filters. Other people think Hotmail has the best filtering. There are different standards for filtering, and that makes email stronger and more robust. Consumers have choices in their mail provider and spamfiltering.

Spam works

I got a spam today advertising spamming services that ended with a tagline that can be paraphrased: We managed to spam you, let us spam others on your behalf!
OK, so what they actually said was:

Feedback loops

There are a lot of different perspectives on Feedback Loops (FBLs) and “this is spam” buttons across the email industry.
Some people think FBLs are the best thing since sliced bread and can’t figure out why more ISPs don’t offer them. These people use use the data to clean addresses off their lists, lower complaints and send better mail. They use the complaints as a data source to help them send mail their recipients want. Too many recipients opted out on a particular offer? Clearly there is a problem with the offer or the segmentation or something.
Other people, though, think the existence of “this is spam” buttons and FBLs is horrible. They call people who click “this is spam” terrorists or anti-commerce-net-nazis. They want to be able to dispute every click of the button. They think that too many ISPs offer this is spam buttons and too many ESPs and network providers pay way to much attention to complaints. The argue ISPs should remove these buttons and stop paying attention to what recipients think.
Sadly, I’m not actually making up the terminology in the last paragraph. There really are who think that the problem isn’t with the mail that they’re sending but that the recipients can actually express an opinion about it and the ISPs listen to those opinions. “Terrorists” and “Nazis” are the least of the things they have called people who complain about their mail.
One of the senior engineers at Cloudmark recently posted an article talking about FBLs and “this is spam” buttons. I think it’s a useful article to read as it explains what value FBLs play in helping spam filters become more accurate.

Light blogging for a while

Sorry for the lack of substantive posts, things seem to have gone completely out of control and I’m not finding a lot of extra cycles to sit down and blog. I’ll try and get some stuff up this week, but I’m also getting ready for MAAWG and the sessions I’m a part of there.
There was an interesting post by Romer over on his personal blog. If you don’t know, Romer helps maintain one of the commercial mail filters. He recently got spammed by one of his vendors and talked about how this is probably not the best idea. Al adds his own take on companies assuming permission. I’ve talked about taking permission in the past but haven’t touched on things like “spamming the guy who runs the filter.”
You’d be surprised, or maybe you wouldn’t, about how many people who run filters for large organizations get spammed regularly. You wouldn’t be surprised to find out that those people do factor in their own personal spam load when adjusting their organizational filters.

Botnets and viruses and phishing, oh my!

MessageLabs released their monthly report on email threats yesterday. Many media outlets picked up and reported that 41% of spam was from a the Rustock botnet.
Other highlights from the report include:

Gmail and the PBL

Yesterday I wrote about the underlying philosophy of spam filtering and how different places have different philosophies that drive their filtering decisions. That post was actually triggered by a blog post I read where the author was asking why Gmail was using the PBL but instead of rejecting mail from PBL listed hosts they instead accepted and bulkfoldered the mail.
The blog post ends with a question:

Why do ISPs do that?

One of the most common things I hear is “but why does the ISP do it that way?” The generic answer for that question is: because it works for them and meets their needs. Anyone designing a mail system has to implement some sort of spam filtering and will have to accept the potential for lost mail. Even the those recipients who runs no software filtering may lose mail. Their spamfilter is the delete key and sometimes they’ll delete a real mail.
Every mailserver admin, whether managing a MTA for a corporation, an ISP or themselves inevitably looks at the question of false positives and false negatives. Some are more sensitive to false negatives and would rather block real mail than have to wade through a mailbox full of spam. Others are more sensitive to false positives and would rather deal with unfiltered spam than risk losing mail.
At the ISPs, many of these decisions aren’t made by one person, but the decisions are driven by the business philosophy, requirements and technology. The different consumer ISPs have different philosophies and these show in their spamfiltering.
Gmail, for instance, has a lot of faith in their ability to sort, classify and rank text. This is, after all, what Google does. Therefore, they accept most of the email delivered to Gmail users and then sort after the fact. This fits their technology, their available resources and their business philosophy. They leave as much filtering at the enduser level as they can.
Yahoo, on the other hand, chooses to filter mail at the MTA. While their spamfoldering algorithms are good, they don’t want to waste CPU and filtering effort on mail that they think may be spam. So, they choose to block heavily at the edge, going so far as to rate limit senders that they don’t know about the mail. Endusers are protected from malicious mail and senders have the ability to retry mail until it is accepted.
The same types of entries could be written about Hotmail or AOL. They could even be written about the various spam filter vendors and blocklists. Every company has their own way of doing things and their way reflects their underlying business philosophy.

Legitimate mail in spamfilters

It can be difficult and frustrating for a sender to understand they whys and wherefores of spam filtering. Clearly the sender is not spamming, so why is their mail getting caught in spam filters?
I have a client that goes through this frustration on rare occasions. They send well crafted, fun, engaging content that their users really want. They have a solid reputation at the ISPs and their inbox stats are always above 98%. Very, very occasionally, though, they will see some filtering difficulties at Postini. It’s sad for all of us because Postini doesn’t tell us enough about what they’re doing to understand what my client is doing to trigger the filters. They get frustrated because they don’t know what’s going wrong; I get frustrated because I can’t really help them, and I’m sure their recipients are frustrated because they don’t get their wanted mail.
Why do a lot of filter vendors not communicate back to listees? Because not all senders are like my clients. Some senders send mail that recipients can take or leave. If the newsletter shows up in their inbox they may read it. If the ad gets in front of their face, they may click through. But, if the mail doesn’t show up, they don’t care. They certainly aren’t going to look for the mail in their bulk folder. Other senders send mail that users really don’t want. It is, flat out, spam.
The thing is, all these senders describe themselves as legitimate email marketers. They harvest addresses, they purchase lists, they send mail to spamtraps, and they still don’t describe themselves as spammers. Some of them have even ended up in court for violating various anti-spam laws and they still claim they’re not spammers.
Senders are competing with spammers for bandwidth and resources at the ISPs, they’re competing for postmaster attention at the ISPs and they’re competing for eyeballs in crowded inboxes.
It’s the sheer volume of spam and the crafty evilness of spammers that drives the constant change and improvement in spamfilters. It’s tough to keep up with the spamfilters because they’re trying to keep up with the spammers. And the spammers are continually looking for new ways to exploit recipients.
It can be a challenge to send relevant, engaging email while dealing with spamfilters and ISPs. But that’s what makes this job so much fun.

Personal contacts at ISPs

A lot of senders seem to think that the secret to good delivery is having personal contacts at the ISPs. That way, when there is a delivery problem you can call up your friend at the ISP and inform them that they have made a mistake. In this little sender fantasy world, the ISP rep then apologizes profusely, unblocks the sender’s mail and perform magic to prevent a block from ever happening again.
Like many fantasies, it doesn’t usually happen that way.
The big ISPs are moving more and more to automated systems that prevent individual employees from interfering. This isn’t actually anything new. I was at a party once and sharing a drink with a representative of one of the big three ISPs. We were talking about a delivery problem one of my clients was having. The rep told me that the ISP did not have any way to actually whitelist around the filter that this client was getting trapped in. The reputation based filtering systems that some ISPs are building are much more performance based and will probably result in those ISPs who can make exceptions now not being able to do so in the future.
When looking for a good delivery person, the real question senders should be asking relate to the skills of the people doing the troubleshooting not who do they know. Does the delivery person have experience troubleshooting delivery? Can they actually resolve problems without having to rely on information from the ISPs? Given the response times at many ISPs, even for personal contacts, it’s often faster to listen to your delivery person than find the ISP rep who will apologize for the mistake.

Delivery thoughts

I have spent quite a bit of time over the last few days working with new clients who are asking a lot of basic, but insightful questions about delivery problems. I have been trying to come up with a model of how spam filtering works, to help them visualize the issue and understand how the different choices they make change, either for better or worse, their delivery.
One model that I have been using is the bucket model. Spamfilters are like buckets. Everything a sender does that looks like what spammers do adds water to the bucket. Reputation is a hole in the bottom of the bucket that lets some of the water flow out. When the bucket overflows, delivery is impacted.
I think this analogy helps explain why fixing any technical issues is so important. The less water that is added to the bucket by technical problems, the less likely it is that the water from content filters will overflow the bucket.