Insight into Gmail filtering

Last week I posted a link to an article discussing how Gmail builds defenses to protect their users from malicious mail. One of the things I found very interesting in that article was the discussion about how Gmail deploys many changes at once, to prevent people from figuring out what the change was.
Let’s take a look at what Gmail said.

Make it hard for attackers to understand your defenses – Use overwhelming force and deploy many countermeasures at once
This is probably the most subtle of the lessons. Attackers constantly probe systems to find loopholes. For example, at some point one of Gmail’s spammers became very astute at finding bugs in our parsers and started to find very subtle bugs he could exploit. For example, he realized he could use the @ ambiguity (it is used in email addresses and in http links) to confuse our parsers and for a brief period of time he successfully evaded detection. This is why it is very important to make probing more difficult for attackers by rolling out multiple changes. That way they are overwhelmed by the number of things to test and can’t easily figure out what changed.
Bottom line: When rolling out change in your defenses, don’t rush (too much) and release multiple changes at once.

I cannot tell you the number of people who have approached me – in person at conferences, on twitter, through email, on LinkedIn – asking if I knew, “What gmail changed this week.” Now, at least, I have an answer. “Gmail changes a lot of things at once in order to stop people from figuring out the filters.”
I’ll be honest, I stopped trying to probe Gmail’s filters to identify ways around them a couple years ago. They are just too hard to evaluate. Sure, I can identify certain things to change that will get email into the inbox, briefly. But unless the underlying issues were fixed, the filters catch up and the mail will go back to the bulk folder. Sometimes it takes the filters days to catch up, sometimes it can take hours.
In any case, probing the filters to see what they’re doing is a very short term, limited fix to Gmail problems.
What I’ve focused on, with my clients, is getting the filters to work for them. We know that modern filters don’t treat all mail from a single company, IP or domain equally. Instead they make delivery decisions for each individual recipient of that mail. Those of you who have seen some of my talks may have seen this image before.
IndividualRecipPreferences
Things like IP reputation, domain reputation, content reputation and link reputation all contribute to the reputation of an email. If the reputation is very bad, the mail is bounced and no body receives it. But if the mail isn’t bounced, then they go through the individual recipient preferences. It is the combination of individual preferences and email reputation that determine where the mail ends up for each recipient. Different recipients may get mail differently.
This is why engagement is so important in email. Sending to people who want to receive the mail improves overall inbox delivery. If most of your recipients want your mail than chances are if you mail someone new, they’ll want your mail, too.
Gmail has a goal with their email delivery. You can make filters work for you by sending mail that users want and engage with. If you’re having problems with Gmail delivery focus on the recipients and making them happy. Don’t waste time trying to troubleshoot a filter change. Gmail isn’t going to make it easy for you.
 
 

Related Posts

Thanks for the great session

I had a great time answering questions at the 2015 All About eMail Virtual Conference & Expo today. Thanks so much to everyone who participated and asked questions. They were great and I’m sorry we didn’t have more time.
I did get some questions on twitter (@wise_laura) afterwards. One was about an example I gave to explain how filters are complex. There have been rumors going around recently that Gmail is filtering mail with more than 3 URLs in it. Let me just say right now THIS IS NOT TRUE emails with more than 3 URLs in them are being delivered just fine to Gmail.
There is a situation involving the number (and type) of URLs that I think are a useful example of the filter complexity happening at some places, like Gmail. I started working on it, but don’t quite have time to finish it today, but will keep working on and it should go up in the next day or so.
Thanks again to everyone who joined the session. You asked some great questions and I had fun answering them.
 

Read More

More from Gmail

Campaign Monitor has an interview with Gmail looking at how to get mail to the Gmail inbox. It’s a great article and I think everyone should go read it.
One of the most important things it talks about is how complex filters are.

Read More

Thoughts on Gmail filtering

Gmail has some extremely complex filters. They’re machine learning based and measure hundreds of things about incoming mail. The filters are continually adjusting to changes and updating how they treat specific mail.
One consequence of continually adjusting machine learning filters is that filtering is not static. What passes to the inbox now, may not pass in a couple hours.
One of the other challenges with Gmail filters is that they look at all the mail mentioning a particular domain and so affiliate mail and 3rd party mail can affect delivery of corporate mail.
The good news is that continually adjusting filters adapt to positive changes as well as negative ones. In fact, I recently made a segmentation suggestion to a client and they saw a significant increase in inbox delivery at Gmail the next day.
Gmail can be a challenge for delivery, but send mail users want and mail does go to the inbox.

Read More