Gmail suddenly puts mail in the bulk folder

One of the delivery challenges that regularly comes up in various delivery discussion spaces is the “Gmail suddenly put my mail in spam.” From my perspective, there is rarely a “suddenly” about Gmail’s decision making process.

As I was answering one of these questions I had a number of thoughts. I’ll share them here on the blog so I can find them in the future.

The first thing that occurred was that I’d shifted my thinking to considering Gmail filtering, in particular, as a filter based on a mailstream, not on a message. Thus my tweet:

The other was a realization that most people don’t consider how what they’re doing might be not OK, but might not be bad enough to be filtered this time. That not OK behaviour builds up over time and eventually tips into mail being filtered. Not because of one message, but because of the dozen or 40 or 75 or hundred messages behind it.

Gmail is really good about just watching and monitoring and watching and scoring. They measure mainstreams over time, and reputation is the sum of all your sends. There may be nothing different or new about a particular send, but Gmail’s been seeing overall reputation decrease just a little bit every time you send.

To put it more succinctly: Senders see themselves “doing the same things” and not realising that every time they do this, they’re slowly eroding away at their reputation with Gmail. Once the erosion hits the tipping point

As I explained it on Facebook.

The effect (moving mail to spam) happens suddenly. The cause builds up over time. Gmail is really good about just watching and monitoring and watching and scoring. They measure mainstreams over time, they don’t really measure individual sends.

What happens is that Gmail starts moving mail sent to people who don’t engage with it to the bulk folder. Your open rates don’t change because these people aren’t opening the mail anyway. But every message delivered to the bulk folder is a ding on your reputation. Those dings build up, until your reputation hits a tipping point and all mail to some of your engaged users sometimes goes to bulk. You might see a small decrease in open rates, but nothing major. You continue mailing as you are and then “all of a sudden” your mail is going to bulk. But it’s just mail to the people who were receiving it in the inbox before, the majority of your mail was always going to bulk. 

This is why any changes you might make to mail, changing an IP or a domain name or using a slightly different format, can sometimes work for a little while. It’s basically decoupling that individual message from the broader history of sending. So you can keep doing the same things over and over again and not hit the inbox.

If, however, you don’t fix the underlying problem, one of two things will happen. Google will connect the new mailstream with the old mailstream and your reputation will fall to the old reputation in a few days. Alternatively, Google won’t connect the new mailstream with the old mailstream and you’ll have many months to slowly erode the reset reputation. In either case it’s just a matter of time before you end up back in the bulk folder again.

Gmail looks at the whole message stream when making decisions. In order to improve delivery at gmail we must also look at the whole message stream.

Related Posts

Why do ISPs do that?

One of the most common things I hear is “but why does the ISP do it that way?” The generic answer for that question is: because it works for them and meets their needs. Anyone designing a mail system has to implement some sort of spam filtering and will have to accept the potential for lost mail. Even the those recipients who runs no software filtering may lose mail. Their spamfilter is the delete key and sometimes they’ll delete a real mail.
Every mailserver admin, whether managing a MTA for a corporation, an ISP or themselves inevitably looks at the question of false positives and false negatives. Some are more sensitive to false negatives and would rather block real mail than have to wade through a mailbox full of spam. Others are more sensitive to false positives and would rather deal with unfiltered spam than risk losing mail.
At the ISPs, many of these decisions aren’t made by one person, but the decisions are driven by the business philosophy, requirements and technology. The different consumer ISPs have different philosophies and these show in their spamfiltering.
Gmail, for instance, has a lot of faith in their ability to sort, classify and rank text. This is, after all, what Google does. Therefore, they accept most of the email delivered to Gmail users and then sort after the fact. This fits their technology, their available resources and their business philosophy. They leave as much filtering at the enduser level as they can.
Yahoo, on the other hand, chooses to filter mail at the MTA. While their spamfoldering algorithms are good, they don’t want to waste CPU and filtering effort on mail that they think may be spam. So, they choose to block heavily at the edge, going so far as to rate limit senders that they don’t know about the mail. Endusers are protected from malicious mail and senders have the ability to retry mail until it is accepted.
The same types of entries could be written about Hotmail or AOL. They could even be written about the various spam filter vendors and blocklists. Every company has their own way of doing things and their way reflects their underlying business philosophy.

Read More

Inbox challenges and dull email in the tabbed inbox

Getting to the inbox is becoming a greater and greater challenge for many marketers. According to Return Path, 22% of opt in mail doesn’t make it to the inbox.
The challenge to marketers is that a lot of opt in mail isn’t important to the recipient. Sure, they’re happy enough to get it if they notice it, but if it’s not there then they don’t care. They’ll buy from an email ad, but it might not be something they’ll seek out. Recipient behaviour tells the ISPs that the mail isn’t all that important, and a lot of it is just background noise so the ISP not delivering it to the inbox doesn’t matter.
Email marketing is like the Girl Scout of the Internet. If the Girl Scout shows up at your doorstep, you’re probably going to buy those 3 boxes of thin mints. But if she doesn’t, that’s OK. If you really want the cookies, you’ll find the co-worker who is taking orders for his daughter. Or you’ll find the table outside the local coffee shop. The Girl Scout showing up on your doorstep makes it more convenient, but she’s not critical to get your fix. Of course, the bonus of the Girl Scout on the doorstep is that a lot of people who won’t go find the cookies will buy when she’s on the doorstep.
A lot of email marketing triggers purchases that recipients would make anyway. They think they might want a particular product, and when they get that coupon or discount or even just a reminder they make the purchase. The email triggers the purchase of a product the buyer intends to purchase anyway. Some email marketing trigger purchases of things the recipient didn’t know existed, but is so enticing after one email they can’t live without. Some email marketing triggers an impulse purchase. In most of these categories, if mail doesn’t show up in the inbox, the recipient really doesn’t miss it.
Many marketers, despite loud protests that all their mail is important and wanted, know this. That’s why so many marketers are having conniptions about the new Gmail tabbed inbox. They’re losing access to the impulse.
From the data I’ve seen, tabs are effecting email marketing programs. Some programs are seeing more revenue, some are seeing less. I think it really remains to be seen what the long term effects are. For many recipients the new tabbed inbox is a new way to interact with their email. Change is hard, and there is a period of adaptation whenever an interface changes. We really don’t know what the long term effect of tabs on sales will be. Sales may go back to previous levels, sales may increase over previous levels, sales may decrease from current levels or sales may stay at their current levels. The full effect isn’t going to be obvious for a while.
It does mean, though, that email marketers need to step up their game. Email marketing in the age of a tabbed inbox might be less about the impulse purchase and more about cultivation and long term branding.
 
 
 

Read More

Thoughts on Gmail and the inbox

Over the last few months more and more marketers are finding their primary delivery challenge is the Gmail inbox. I’ve been thinking about why Gmail might be such a challenge for marketers. Certainly I have gotten a lot of calls from people struggling to figure out how to get into the Gmail inbox. I’ve also seen aggressive domain based filtering from Gmail, where any mention of a particular domain results in mail going to the bulk folder.
It’s one of those things that’s a challenge, because in most of these cases there isn’t one cause for bulk foldering. Instead there’s a whole host of things that are individually very small but taken together convince Gmail that the mail doesn’t need to be in the inbox.
A pattern that I’m starting to see is that Gmail is taking a more holistic look at all the mail from a sender. If the mail is connected to an organization, all that mail is measured as part of their delivery decision making. This is hurting some ESPs and bulk senders. I’ve had multiple ESPs contact me in the last 6 months looking for help because all their customer emails are going to bulk folder.
Gmail’s filtering is extremely aggressive. From my perspective it always has been. I did get an invite for a Gmail account way back in the day. I moved a couple mailing lists over to that account to test it with some volume and discussion lists. I gave up not long after because no matter what I did I couldn’t get gmail to put all the mail from that list into the tag I had set up for it. Inevitably some mail from some certain people would end up in my spam folder.
Gmail has gotten better, now they will let you override their filters but give you a big warning that the message would have been delivered to spam otherwise.
Gmail_NotSpam
What are mailers to do? Right now I don’t have a good answer. Sending mail people want is still good advice for individual senders. But I am not sure what can be done about this ESP wide filtering that I’m starting to see. It’s possible Gmail is monitoring all the mail from a particular sender or ESP and applying a “source network” score. Networks letting customers send mail Gmail doesn’t like (such as affiliate mail or payday mail, things they mentioned specifically at M3AAWG) are having all their customers affected.
I suspect this means that ESPs seeing problems across their customer base are going to have to work harder to police their customers and remove problematic mail streams completely. Hopefully, ESPs that can get on the Gmail FBL can identify the problem customers faster before those customers tank mail for all their senders.

Read More