Did the algorithm change?

laura
Industry
February 13, 2018

When faced with unexplained deliverability changes one of the first questions many folks ask is “Did the algorithm change.” In many ways this is an meaningless question. Why? Because there are two obvious answers to the question.
A1: Of course it didn’t.
A2: Of course it did.
Both answers are correct, but they’re answering different underlying questions. When we understand how two diametrically opposed answers are both correct, we understand much more about filtering.

What is an algorithm?

When we’re talking about spam filtering the algorithm is the process or rules to follow.

Basically, an algorithm is a computer program that is set up to filter spam to the bulk folder and filter wanted mail into the inbox. This algorithm doesn’t change. It can’t.
In the case of many (most?) spam filters, the filters incorporate features of machine learning.
This means the algorithm is constantly changing, learning more and more about what is spam and what is wanted mail based on user interactions.

Algorithms don’t change

Overall, the algorithms don’t change that frequently. They are fed data (lots of data) on a continual basis. They take feedback from recipients (spam / not spam buttons) and developers (new data sets of known bad and known good mail) to learn what good mail looks like and what bad mail looks like. But the underlying code doesn’t change very frequently.

Results change

Machine learning algorithms are only as good as the data they’re fed. In the case of spam filters, the input data is constantly changing. So the output results change. Sometimes an email that was not-spam one day is spam the next because the algorithm caught up with a new threat or new behavior.

Delivery is still in your control

This machine learning and reliance on end users to help tune filters may make it seem like spam filtering is completely out of the senders’ control. That there is nothing a sender can do to get into or out of the bulk folder. The good news is, the underlying algorithms are pretty simple: wanted mail goes to the inbox, unwanted mail goes to the bulk folder. As with everything, details matter. Senders who are focused on recipients usually don’t have a difficult time reaching the inbox. Companies that focus on themselves and try gimmicks find it much harder to consistently reach the inbox.

5 Simple Tricks to Reach the Inbox

I saw a post over on LinkedIn today. It was from an ESP, talking about their simple tips and tricks for getting into the inbox. The laughable bit was half the “tricks” had nothing to do with getting to the inbox, but rather were about enticing people to open the mail once it’s gotten to the inbox.
There are no “tricks” to getting to the inbox. There used to be some tricks. But the ISPs figured them out and protect against them.

Thinking about deliverability

I was chatting with folks over on one of the email slack channels today. The discussion was about an ESP not wanting to implement a particular change as it would hurt deliverability. It led me down a path of thinking about how we think of deliverability and how that informs how we approach email.
The biggest problem I see is the black and white thinking.
There’s an underlying belief in the deliverability, receiving, and filtering communities that the only way to affect sending behavior is to block (or threaten to block) mail.

This was true back in the ancient times (the late 90’s). We didn’t have sophisticated tools and fast CPUs. There weren’t a lot of ways to handle bad mail other than to block. Now the landscape is different. We have many more tools and the computing capacity to quickly sort large streams of data.
At most places these days, blocking is an escalation, not a warning shot. Many places rate limit and bulk folder questionable mail as a first strike against problem mail. Sometimes the mail is bad enough to result in a block. Other times, it’s not bad enough to block, so it disappears into the bulk folder.
There’s a corresponding belief in the sending community that if their behavior doesn’t result in blocking then they’re acting acceptably. This isn’t true either. There are a lot of things you can do (or not do) that don’t help delivery, but will actively harm delivery. Likewise, there are things you can do that don’t actively harm delivery, but will help. All of these things add up to reaching the inbox.

Data Cleansing part 2

In an effort to get a blog post out yesterday before yet another doctor’s appointment I did not do nearly enough research on the company I mentioned selling list cleansing data. As Al correctly pointed out in the comments they are currently listed on the SBL. And when I actually did the research I should have done it was clear this company has a long term history of sending unsolicited email.
Poor research and a quickly written blog post led to me endorsing a company that I absolutely shouldn’t have. And I do apologize for that.
With all that being said, Justin had a great question in the comments of yesterday’s post about data cleansing.