About that Junk Folder

I use a pretty standard mail filtering setup – a fairly vanilla SpamAssassin setup on the front end, combined with naive bayesian content filters in my mail client. So I don’t reject any mail, it just ends up in one of my inboxes or a junk folder. And I have a mix of normal consumer mail – facebook, twitter, lots of commercial newsletters, mail from friends and colleagues and spam. (As well as that I have a lot of high traffic industry mailing lists, but overall it’s a fairly normal mix.)
My bayesian filter gets trained mostly by me hitting “this is spam” when spam makes it to my inbox. If I’m expecting an email “immediately” – something like a mailing list COI confirmation or email as part of buying something online – I’ll check my spam filter and move the mail to my inbox in the rare case it ended up there. Other than that I let it and spamassassin chug along with no tweaking.
I’m starting a data analysis project, based on my own inboxes, and as part of that I’m using some tools to look for false positives in my junk folders, and manually fixing anything that’s misclassified. I’ve been doing this for a couple of hours now, and I’ve found some interesting things.

  1. Simple content filters work remarkably well out of the box, at least for my mail stream. Spectacularly well. There’s very little in the way of false positives. Very, very little.
  2. Of those false positives there’s nothing I’d have been bothered about. It’s generic, unexciting junk mail.
  3. Most of the systemic false positives seem to be correlated with the senders doing something bad. Heathrow Express, for instance, sent me mail every two weeks or so since I’d signed up. Then for no obvious reason they stopped sending for three months, then started sending again. Every mail they sent after that pause ended up in the junk folder, and I never missed them.
  4. I get regular newsletters from ThinkGeek. Every one of those goes to the inbox. I occasionally get mails from them about my account (“you’ve got 420 geek points left”) that are kinda transactional, but not something I expect to see – and they all end up in the junk folder. Several other senders do the same thing, and get the same result.
  5. Several companies have used tagged addresses to send me newsletters for a while, and also used them to send unsolicited facebook invites (to the tagged address, from facebook servers). The regular newsletters all go to the inbox, while the facebook invites all go to the junk folder. “Legitimate” facebook mail, meanwhile, keeps going to the inbox.
  6. Apple send me a lot of newsletters – I’m a Mac and iPhone developer, I get their consumer newsletters, transactional stuff from our local store – lots and lots of newsletters. They all made it to the inbox except for one. The one that ended up in the junk folder was a one-off about recycling, and it wasn’t up to their usual design standards – it had ugly big green “call to action” headlines in it, very different to their usual clean design.
  7. Just one sender hit the junk folder every time. The distinctive thing about their messages (apart from them not being something I missed) was that the plain text part of them was dreadful, just a bad lynx dump of the html section. Even a “No plain text for you! Go to this link!” would have been better.

I was surprised at how effective this simple content-based filtering setup had worked with little tuning other than hitting the this-is-spam button – both in it’s accuracy at removing spam while keeping a very low false positive rate, but also how well the false positives matched my judgement of “Meh. This mail isn’t interesting.”.
We spend a lot of time talking about the things you should do to make the mail relevant to the recipient – compelling content, consistent, predictable delivery schedules, clear consistent branding, use of a single consistent mail stream to communicate with a recipient rather than several different streams. Until I went through this exercise it wasn’t clear to me how much of an effect those things also have on fairly simple recipient-trained filters.
 

Related Posts

Mailing old addresses: 5 questions to ask first

James asked the question on twitter:

If you haven’t mailed an address in 5-10 yrs, would you include it in a re-engagement mail?

Read More

Setting expectations at the point of sale

In my consulting, I emphasize that senders must set recipient expectations correctly. Receiver sites spend a lot of time listening to their users and design filters to let wanted and expected mail through. Senders that treat recipients as partners in their success usually have much better email delivery than those senders that treat recipients as targets or marks.
Over the years I’ve heard just about every excuse as to why a particular client can’t set expectations well. One of the most common is that no one does it. My experience this weekend at a PetSmart indicates otherwise.
As I was checking out I showed my loyalty card to the cashier. He ran it through the machine and then started talking about the program.
Cashier: Did you give us your email address when you signed up for the program?
Me: I’m not sure, probably not. I get a lot of email already.
Cashier: Well, if you do give us an email address associated with the card every purchase will trigger coupons sent to your email address. These aren’t random, they’re based on your purchase. So if you purchase cat stuff we won’t send you coupons for horse supplies.
I have to admit, I was impressed. PetSmart has email address processes that I recommend to clients on a regular basis. No, they’re not a client so I can’t directly take credit. But whoever runs their email program knows recipients are an important part of email delivery. They’re investing time and training into making sure their floor staff communicate what the email address will be used for, what the emails will offer and how often they’ll arrive.
It’s certainly possible PetSmart has the occasional email delivery problem despite this, but I expect they’re as close to 100% inbox delivery as anyone else out there.

Read More

Recipients are the secret to good delivery

Many, many people hire me to educate them on delivery and fix their email problems. This is good, it’s what I do. And I’m quite good at helping clients see where their email program isn’t meeting expectations. I can translate tech speak into marketing. I can explain things in a way that shifts a client’s perception of what the underlying issues are. I can help them find their own way into the inbox.
But…
Most of what I do is simply think about email delivery from the point of view of a recipient and help clients better meet their recipient’s expectations. This works. This works really well. If you send mail that your recipients want your mail gets to the inbox.
Here’s the secret: ISPs and most spam filters have a design goal to deliver mail their users want. They only want to block mail their users don’t want.
Filters are not designed to block wanted mail.
Sure there are complicated situations where senders have gotten behind the 8 ball and need some help cleaning up. There are situations where filters screw up and block mail they shouldn’t (and aren’t quite designed to). Spam filters are complicated bits of code and sometimes they do things unexpectedly. All of these things do happen.
But these situations happen a lot less than most senders think. Most of the time when mail is hitting the bulk folder, or is throttled at the MTA the issue is that recipients don’t care about the mail.
Recipients aren’t engaged with a particular sender or particular brand. So ISPs react accordingly and that mail ends up slowly delivered or bulked. This upsets the senders to no end, but the recipients? The recipients often don’t care that some mail shows up in bulk or arrives Wednesday afternoon instead of Tuesday evening.
When recipients are engaged with a particular sender or brand, though? Delivery is fast and reliable. Mail is rarely delayed or bulked. When recipients want mail, they interact with it. They look in the bulk folder. They miss it when it’s not there. They complain to the ISPs when they don’t get it. The ISPs react accordingly and prioritize or “red carpet” that email.
The secret to really good delivery is to get your recipients to handle your ISP relations for you. Send mail they miss when they don’t get it, and you’ll discover most of your delivery problems go away.
 
 

Read More