Improving Gmail Delivery

laura
Delivery improvement
September 6, 2017

Lately I’m hearing a lot of people talk about delivery problems at Gmail. I’ve written quite a bit about Gmail (Another way Gmail is different, Gmail filtering in a nutshell, Poor delivery at Gmail but no where else, Insight into Gmail filtering) over the last year and a half or so. But those articles all focus on different parts of Gmail delivery and it’s probably time for a summary type post.

Gmail is different

There are two major reasons that Gmail filtering is different from the other webmail providers: when it was launched and who it was launched by.
Gmail entered the mail market late in the internet era when compared to other free email providers. AOL offered internet email in 1992; Yahoo Mail opened in 1994; Hotmail debuted in 1996. When these systems were in development, spam wasn’t an issue.
Spam filtering was added later, as the problem grew. Gmail didn’t launch until 2004, nearly a decade after their current competitors. Spam was already a problem by 2004, so Gmail was able to build filters in from the beginning.
The other real difference is Google’s experience and expertise in search. They built their business on being able to take lots and lots of data, categorize it and make it instantly searchable. This actually translates well to spam filtering, in that they take lots of data, categorize it and put it in appropriate mailboxes.
Those aren’t the only reasons Gmail is different. Another factor is Gmail’s attitude towards senders. The prime example is their FBL. Unlike most ISPs, Gmail doesn’t provide the full message back in its FBL. Instead, they give a count of complaints. They’re not going to help senders remove folks who complain. The flip side of this is they are leading the way in providing easier ways to unsubscribe.
The different history, expertise, and attitude of Google are the core of why Gmail delivery is so unlike others.

Metrics look great

The standard diagnostic for problem is to investigate the metrics, identify areas where they show limits and work to improve them. Along the way, email delivery improves. At Gmail, however, there’s often nothing obviously wrong with the metrics. The problem is the metrics we’re using are measuring symptoms not identifying underlying issues. Think of all the metrics we use as a fever. Just because a fever is gone (or you don’t have one) doesn’t mean you’re not sick.
Metrics are proxy measurements. The best metrics in the world aren’t going to help your delivery at Gmail if the recipients don’t want your mail.
The Recipient Has To Want Your Mail.

Why is Gmail so hard?

Because Gmail is smarter than we are.
Because Gmail looked at the things other companies did and learned what worked to decrease spam and what worked to decrease signs of spam (those are different things).
Because Gmail has years and years of experience in dealing with people who game SEO listings.
Because Gmail puts the user experience ahead of the sender experience.

Gmail filtering in a nutshell

Gmail’s approach to filtering; as described by one of the old timers. This person was dealing with network abuse back when I was still slinging DNA around as my job and just reading headers as a hobby.

Filtering by gestalt

One of those $5.00 words I learned in the lab was gestalt. We were studying fetal alcohol syndrome (FAS) and, at the time, there were no consistent measurements or numbers that would drive a diagnosis of FAS. Diagnosis was by gestalt – that is by the patient looking like someone who had FAS.
It’s a funny word to say, it’s a funny word to hear. But it’s a useful term to describe the future of spam filtering. And I think we need to get used to thinking about filtering acting on more than just the individual parts of an email.

Filtering is not just IP reputation or domain reputation. It’s about the whole message. It’s mail from this IP with this authentication containing these URLs. Earlier this year, I wrote an article about Gmail filtering. The quote demonstrates the sum of the parts, but I didn’t really call it out at the time.

Another way Gmail is different

I was answering a question on Mailop earlier today and had one of those moments of clarity. I finally managed to articulate one of the things I’ve known about Gmail, but never been able to explain. See, Gmail has never really put a lot of their filtering on the SMTP transaction and IP reputation. Other ISPs do a lot of the heavy lifting with IP filters. But not Gmail.
While I was writing the answer I realized something. Gmail was a late entrant into the email space. AOL, Hotmail, Yahoo, even the cable companies, were providing email services in the 90s. When spam started to be a problem, they started with IP based blocking. As technology got better and content filtering became viable, improvements were layered on top of IP based blocking.

Gmail didn’t enter the mailbox market until the 2000’s. When they did, they had money, lots of hardware, and internal expertise to do content filtering. They didn’t start with IP based filtering, so their base is actually content filtering. Sure, there were some times when they’d push some mail away from the MTAs, but most of their filtering was done after the SMTP transaction. The short version of this is I never really pay any attention to IP reputation when dealing with Gmail. It’s just another factor. Unless you’re blocked and if you get blocked by Gmail, wow, you really screwed up.
Gmail does, of course, do some IP based blocking. But in my experience IP filters are really only turned against really egregious spam, phishing and malicious mail. Most email marketers reading my blog won’t ever see IP filters at Gmail because their mail is not that bad.
Other companies aren’t going to throw away filters that are working, so the base of their filters are IPs. But Google never had that base to work from. Their base is content filters, with some IP rep layered on top of that.
That’s a big reason Gmail filters are different from other filters.