Filters do what we tell them

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

What about the botnets?!
AOL accidentally hard bounces valid mail

Related Posts

Use all the channels

One of the hardest deliverability situations to address is when all mail from a certain sender is going to the bulk folder. I’ve had numerous clients come to me to address this situation over the years. Ideally, clients come to me before all their mail is going to bulk. Then we can make some tweaks and changes to their mail program, repair the reputation and then recover other addresses. We have knobs we can twist to fix things if some people are still getting messages in their inbox. We have data to measure.
When all mail is going to bulk, though, we lose access to the knobs and the data. There are zero complaints if mail is going to bulk. There are no opens or clicks, because many ISPs disable images and links in the bulk folder. Our normal “fixing reputation” tools are taken away from us.
Senders with all their mail going to bulk are faced with a profound challenge. How can they engage customers who are unengaged and who are not seeing mail at all? How can we fix deliverability when our normal tools and metrics are unavailable?
If we can get even a small percentage of recipients to go pull mail out of bulk or spam and move it to their inbox, then we’re well on our way to repairing reputation. But how can we get them to go look for the mail in the bulk folder. Recent Litmus research suggests that a significant percentage of folks regularly check their spam folder, but this isn’t always enough to repair reputation,
The question becomes how can the senders encourage recipients to go digging through their spam folder. 
This is the point where I start quizzing clients on what other channels they use to communicate with their customers. I’ll run through the whole list: social media, snail mail, push notices through apps, SMS, website popups, Facebook ads. I work with them to identify users who are engaged with their brand and brainstorm ways to get those users to look for mail.
I’m always pleased to see large brands using these strategies. Just recently Blizzard used twitter to communicate with their users about email problems. They tweeted.
BlizzardTweet
The link takes you to the Blizzard support site. Where they give specific instructions on how to whitelist mail and what mail to whitelist.

Read More

Do system administrators have too much power?

Yesterday, Laura brought a thread from last week to my attention, and the old-school ISP admin and mail geek in me felt the need to jump up and say something in response to Paul’s comment. My text here is all my own, and is based upon personal experience as well as those of my friends. That said, I’m not speaking on their behalf, either. 🙂
I found Paul’s use of the word ‘SysAdmin’ to be a mighty wide (and — in my experience — probably incorrect) brush to be painting with, particularly when referring to operations at ISPs with any significant number of mailboxes. My fundamental opposition to use of the term comes down to this: It’s no longer 1998.
The sort of rogue (or perhaps ‘maverick’) behavior to which you refer absolutely used to be a thing, back when a clean 56k dial-up connection was the stuff of dreams and any ISP that had gone through the trouble to figure out how to get past the 64k user limit in the UNIX password file was considered both large and technically competent. Outside of a few edge cases, I don’t know many system administrators these days who are able to (whether by policy or by access controls) — much less want to — make such unilateral deliverability decisions.
While specialization may be for insects, it’s also inevitable whenever a system grows past a certain point. When I started in the field, there were entire ISPs that were one-man shows (at least on the technical side). This simply doesn’t scale. Eventually, you start breaking things up into departments, then into services, then teams assigned to services, then parts of services assigned to teams, and back up the other side of the mountain, until you end up with a whole department whose job it is to run one component of one service.
For instance, let’s take inbound (just inbound) email. It’s not uncommon for a large ISP to have several technical teams responsible for the processing of mail being sent to their users:

Read More

Parasites hurt email marketing

As a small business owner I am a ripe target for many companies. They buy my address from some lead generation firm, or they scrape it off LinkedIn, and they send me a message that pretends to be personalized but isn’t really.
“I looked at your website… we have a list of email addresses to sell you.”
“We offer cold calling services… can I set up a call with you?”
“I have scheduled a meeting tomorrow so I can tell you about our product that will solve all your technical issues and is also a floor wax.”
None of these emails are anything more than spam. They’re fake personalized. There’s no permission. On a good day they’ll have an opt out link. On a normal day they might include an actual name.
These are messages coming to an email address I’ve spent years trying to protect from getting onto mailing lists. I don’t do fishbowls, I’m careful about who I give my card to, I never use it to sign up for anything. And, still, that has all been for naught.
I don’t really blame the senders, I mean I do, they’re the ones that bought my address and then invested in business automation software that sends me regular emails trying to get me to give them a phone number. Or a contact for “the right person at your business to talk to about this great offer that will change your business.”
The real blame lies with the people who pretend that B2B spam is somehow not spam. Who have pivoted their businesses from selling consumer lists to business lists because permission doesn’t matter when it comes to businesses. The real blame lies with companies who sell “marketing automation software” that plugs into their Google Apps account and hijacks their reputation to get to the inbox. The real blame lies with list cleansing companies who sell list buyers a cleansing service that only hides the evidence of spamming.
There are so many parasites in the email space. They take time, energy and resources from large and small businesses, offering them services that seem good, but really are worthless.
The biologically interesting thing about parasites, though, is that they do better if they don’t overwhelm the host system. They have to stay small. They have to stay hidden. They have to not cause too much harm, otherwise the host system will fight back.
Email fights back too. Parasites will find it harder and harder to get mail delivered in any volume as the host system adapts to them. Already if I look in my junk folder, my filters are correctly flagging these messages as spam. And my filters see a very small portion of mail. Filtering companies and the business email hosting systems have a much broader view and much better defenses.
These emails annoy me, but I know that they are a short term problem.  As more and more businesses move to hosted services, like Google Apps and Office365 the permission rules are going to apply to business addresses as well as consumer addresses. The parasites selling products and services to small business owners can’t overwhelm email. The defenses will step in first.
 

Read More