Updating the filtering model

One thing I really like about going to conferences is they’re often one of the few times I get to sit and think about the bigger email picture. Hearing other people talk about their marketing experiences, their email experiences, and their blocking experiences usually triggers big picture style thoughts.
Earlier this week I was at Activate18, hosted by Iterable. The sessions I attended were interesting and insightful. Of course, I went to the deliverability session. While listening to the presentation, I realized my previous model of email filtering needed to be updated.

The old model

The old model was pretty simple. It was based on the idea that ISPs ask two fundamental questions about email when it comes in.

  1. Is it safe?
  2. Is it wanted?

If the answer to both those questions is yes, the mail is delivered to the inbox.
Safe is something we don’t talk much about in the marketing space, because generally our mail is safe. But our mail has to go through the exact same filters that are set out to catch the bad guys. And sometimes we do things that trigger that set of filters unintentionally.
The wanted is where marketers can really shine. ISPs look at their user behavior to determine if mail is wanted or not. While the measurements are slightly different than what marketers use, Marketers must also look at how wanted mail is. Engagement with the email is part of it. But, you can also use other metrics you have. Do they visit your website? Are they active on your FB page? What other data do we have that says this person is engaged with your brand and won’t object to increased volume.
Spammers send unwanted mail. Spammers send mail to a lot of addresses where the address owner doesn’t log in. Spammers send mail to people who don’t want it, so they delete it immediately. If you are sending lots of mail and your recipient demographic looks like the typical spammer recipient demographic, then your mail will be treated like spam. It’s ALL about the recipient. That’s what the ISP uses to measure your mail. And if your recipients are reacting to your mail in the same way they react to spammer mail, then you’re going to face deliverability challenges.
Increasing volume is expected, but you need to be strategic with how you increase it. I’m working with one of my clients right now to help them with the final touches on the holiday marketing program. They are increasing mail, but we made some changes to their proposed program to protect them against deliverability challenges. They’re in the early stages of warmup (yes, you need to warmup for significant volume changes!) and it’s looking good.
(That’s from an email I wrote to answer a question on the Only Influencers mailing list back in 2015)

The newer model

This week, though, I realized that another question snuck into the equation. The ISPs are still asking if mail is safe. Does it have harmful content? Phishing? Viruses? Is it coming from a botnet? ISPs use IP reputation and domain/URL reputation to answer the bulk of these questions during the SMTP transaction. If an ISP determines the mail isn’t safe, they’ll reject it out of hand.
But with technology improvements and machine learning, ISPs are able to split the second question into two sub questions.

  1. Is it unsolicited?
  2. Is it wanted by the recipient?

The unsolicited piece was always part of the equation. Many of the metrics used to answer the “is it wanted” question were actually trying to determine if the mail was unsolicited. These measurements include the things we talk about: bounce rate, complaint rate, unknown user rate.  There are two reasons I think this is worth pointing out. The first is that I have often glossed over how much unsolicited mail “legitimate” senders actually send. Every company who purchases a list, who uses lead gen, who collects addresses at point of sale send unsolicited email. They may not mean to, but they do. The second reason is this is the piece of spam filtering that data hygiene companies are addressing. Everyone who cleans your list, identifies bounces, finds bad users, they’re specifically addressing this filtering. And many senders don’t understand why their mail is still going to bulk even after they’ve purchased very expensive hygiene services.
The reality is, that hygiene services make mail look less unsolicited, by removing many of the markers that tell ISPs the mail is unsolicited. But that doesn’t make the mail any more wanted by the recipient. Hence the current focus on engagement and individual delivery metrics. These are metrics that can’t be faked by the sender. Third parties can’t identify those recipients that want a particular piece of mail. And, with privacy laws like GDPR, it’s unlikely those business models will be cost effective.

Related Posts

Are seed lists still relevant?

Those of you who have seen some of my talks have seen this model of email delivery before. The concept is that there are a host of factors that contribute to the reputation of a particular email, but that at many ISPs the email reputation is only one factor in email delivery. Recipient preferences drive whether an email ends up in the bulk folder or the inbox.

The individual recipient preferences can be explicit or implicit. Users who add a sender to their address book, or block a sender, or create a specific filter for an email are stating an explicit preference. Additionally, ISPs monitor some user behavior to determine how wanted an email is. A recipient who moves an email from the bulk folder to the inbox is stating a preference. A person who hits “this-is-spam” is stating a preference. Other actions are also measured to give a user specific reputation for a mail.
Seed accounts aren’t like normal accounts. They don’t send mail ever. They only download it. They don’t ever dig anything out of the junk folder, they never hit this is spam. They are different than a user account – and ISPs can track this.
This tells us we have to take inbox monitoring tools with a grain of salt. I believe, though, they’re still valuable tools in the deliverability arsenal. The best use of these tools is monitoring for changes. If seed lists show less than 100% inbox, but response rates are good, then it’s unlikely the seed boxes are correctly reporting delivery to actual recipients. But if seed lists show 100% inbox and then change and go down, then that’s the time to start looking harder at the overall program.
The other time seed lists are useful is when troubleshooting delivery. It’s nice to be able to see if changes are making a difference in delivery. Again, the results aren’t 100% accurate but they are the best we have right now.
 

Read More

Do system administrators have too much power?

Yesterday, Laura brought a thread from last week to my attention, and the old-school ISP admin and mail geek in me felt the need to jump up and say something in response to Paul’s comment. My text here is all my own, and is based upon personal experience as well as those of my friends. That said, I’m not speaking on their behalf, either. 🙂
I found Paul’s use of the word ‘SysAdmin’ to be a mighty wide (and — in my experience — probably incorrect) brush to be painting with, particularly when referring to operations at ISPs with any significant number of mailboxes. My fundamental opposition to use of the term comes down to this: It’s no longer 1998.
The sort of rogue (or perhaps ‘maverick’) behavior to which you refer absolutely used to be a thing, back when a clean 56k dial-up connection was the stuff of dreams and any ISP that had gone through the trouble to figure out how to get past the 64k user limit in the UNIX password file was considered both large and technically competent. Outside of a few edge cases, I don’t know many system administrators these days who are able to (whether by policy or by access controls) — much less want to — make such unilateral deliverability decisions.
While specialization may be for insects, it’s also inevitable whenever a system grows past a certain point. When I started in the field, there were entire ISPs that were one-man shows (at least on the technical side). This simply doesn’t scale. Eventually, you start breaking things up into departments, then into services, then teams assigned to services, then parts of services assigned to teams, and back up the other side of the mountain, until you end up with a whole department whose job it is to run one component of one service.
For instance, let’s take inbound (just inbound) email. It’s not uncommon for a large ISP to have several technical teams responsible for the processing of mail being sent to their users:

Read More

Politics and Delivery

Last week I posted some deliverability advice for the DNC based on their acquisition of President Obama’s 2012 campaign database. Paul asked a question on that post that I think is worth some attention.

Read More