Gmail suddenly puts mail in the bulk folder

One of the delivery challenges that regularly comes up in various delivery discussion spaces is the “Gmail suddenly put my mail in spam.” From my perspective, there is rarely a “suddenly” about Gmail’s decision making process.

As I was answering one of these questions I had a number of thoughts. I’ll share them here on the blog so I can find them in the future.

The first thing that occurred was that I’d shifted my thinking to considering Gmail filtering, in particular, as a filter based on a mailstream, not on a message. Thus my tweet:

The other was a realization that most people don’t consider how what they’re doing might be not OK, but might not be bad enough to be filtered this time. That not OK behaviour builds up over time and eventually tips into mail being filtered. Not because of one message, but because of the dozen or 40 or 75 or hundred messages behind it.

Gmail is really good about just watching and monitoring and watching and scoring. They measure mainstreams over time, and reputation is the sum of all your sends. There may be nothing different or new about a particular send, but Gmail’s been seeing overall reputation decrease just a little bit every time you send.

To put it more succinctly: Senders see themselves “doing the same things” and not realising that every time they do this, they’re slowly eroding away at their reputation with Gmail. Once the erosion hits the tipping point

As I explained it on Facebook.

The effect (moving mail to spam) happens suddenly. The cause builds up over time. Gmail is really good about just watching and monitoring and watching and scoring. They measure mainstreams over time, they don’t really measure individual sends.

What happens is that Gmail starts moving mail sent to people who don’t engage with it to the bulk folder. Your open rates don’t change because these people aren’t opening the mail anyway. But every message delivered to the bulk folder is a ding on your reputation. Those dings build up, until your reputation hits a tipping point and all mail to some of your engaged users sometimes goes to bulk. You might see a small decrease in open rates, but nothing major. You continue mailing as you are and then “all of a sudden” your mail is going to bulk. But it’s just mail to the people who were receiving it in the inbox before, the majority of your mail was always going to bulk. 

This is why any changes you might make to mail, changing an IP or a domain name or using a slightly different format, can sometimes work for a little while. It’s basically decoupling that individual message from the broader history of sending. So you can keep doing the same things over and over again and not hit the inbox.

If, however, you don’t fix the underlying problem, one of two things will happen. Google will connect the new mailstream with the old mailstream and your reputation will fall to the old reputation in a few days. Alternatively, Google won’t connect the new mailstream with the old mailstream and you’ll have many months to slowly erode the reset reputation. In either case it’s just a matter of time before you end up back in the bulk folder again.

Gmail looks at the whole message stream when making decisions. In order to improve delivery at gmail we must also look at the whole message stream.

Related Posts

Confusing the engineers

We went camping last weekend with a bunch of friends. Had a great time relaxing on the banks of the Tuolumne River, eating way too much and visiting.
On Saturday I was wearing a somewhat geeky t-shirt. It said 554: abort mission. (Thank you MessageSystems). At some point on Saturday every engineer came up to me, read my shirt and then looked at me and said “That’s not HTTP.”
That lead to various discussions about how their junior engineers don’t actually know SMTP at all. Why? Because the SMTP libraries just work. Apparently the HTTP libraries aren’t that great, so folks have to learn more about HTTP to troubleshoot and use them.
I’m sure there’s a joke in there somewhere: A Kindle engineer, an Android engineer and a robot engineer walk into a campsite…
EmailFilters_boxes_forblogIt did leave me thinking, though, about how it’s not that easy to run your own mail server these days. Gone are the days when running your own server was cost effective and easy. These days, there is just too much spam coming in. Crafting filters is a skilled job. It’s not that hard to run good filters. But to run good filters takes time to do well.
There are also a lot of challenges to sending mail. One of the discussions I had at the campsite was how hard it was to configure outbound mail. The engineer was helping a friend set up a website and trying to get the website to send notifications to the friend. But without setting up authentication the mail kept silently failing.
Of course, we do run our own mail server. But it’s our job and, in many ways, it keeps us honest. We don’t run many filters meaning we see what spammers are doing and can use our own experiences to better understand what commercial filters are dealing with.
For most people, though, I really think using a service is the right solution. Find one with filters that meet your needs and just pay them to deal with the headache.
 

Read More

Changes at Gmail

As I’ve said before, I can usually tell when some ISP changes their filtering algorithm because I start getting tons and tons of calls about delivery problems at that ISP. This past month it’s been Gmail.
There have been two symptoms I’ve been hearing about. One is an increase in bulk folder delivery for mail that previously was reliably hitting the inbox. The other is a bit more interesting. I’ve heard of 3 different mailers, with good reputations and very clean lists, that are seeing 4xx delays on some of their mail. The only consistency I, and my colleagues at some ESPs, have identified is that the mail is “bursty.”
The senders affected by this do send out mail daily, but the daily mail is primarily order confirmations or receipts or other transactional mails. They send bi-weekly newsletters, though, exploding their volume from a few tens of thousands up to hundreds of thousands. This seems to trigger Gmail to defer mail. It does get delivered eventually. It’s frustrating to try and deal with because neither side is really doing anything wrong, but good senders are seeing delivery delays.
For the bulk foldering, Bronto has a good blog post talking about the changes and offering some solid suggestions for how to deal with them. I’m also hearing from some folks who are reliable that Gmail may be rolling back some of the bulk foldering changes based on feedback from their users.
So if you’re seeing changes at Gmail, it’s not just you.

Read More

Email marketing OF THE FUTURE!

ISPs are continually developing tools for their users. Some of the newer tools are automatic filters that help users organize the volumes of mail they’re getting. Gmail released Priority Inbox over a year ago. Hotmail announced new filters as part of Wave 5 back in October.
All of these announcements cause much consternation in the email marketing industry. Just today there was a long discussion on the Only Influencers list about the new Hotmail filtering. There was even some discussion about why the ISPs were doing this.
I think it’s pretty simple why they’re creating new tools: users are asking for them. The core of these new filters is ISPs reacting to consumer demand. They wouldn’t put the energy into development if their users didn’t want it. And many users do and will use priority inbox or the new Hotmail filtering.
Some people are concerned that marketing email will be less effective if mail is not in the inbox.

Read More