Spam filters and mailbox usage

laura
Industry
June 5, 2014

It’s no secret that I run very little in the way of spam filters, and what filters I do run don’t throw away mail, they just shove it into various mailboxes.
Looking at my mailboxes currently I have 11216 unread messages in my mail.app junk folder, 10600 unread messages in my work spam assassin folder and 29401 messages in my personal spam assassin folder (mail getting more than +7 on our version of spam assassin gets filtered into these folders). I went through and marked all of my messages read back in mid-January. That’s a little over 50,000 messages in a little over 5 months or slightly more than 2700 spams a week.
But these are messages I don’t have to deal with so while they’re somewhat annoying and a bit of “wow, my addresses are everywhere” they’re not a huge deal. I have strong enough filters for wanted mail that I can special case it.

Spam in a mailbox makes it harder to find wanted mail.

I use tagged addresses whenever I sign up for something. This means I can identify and separate senders I gave addresses to from those senders who are spamming me. I can trivially filter out solicited from unsolicited email. In the same amount of time that I received 50,000+ spam, I received just over 8000 real messages. If I didn’t have an easy way to sort out spam from mail I signed up for, I’d have to manually delete 6 messages to find one message I wanted.
I don’t know how much experience you have with manually deleting spam, but I do it daily (my filters aren’t 100%) and I spam a message I shouldn’t have at least once a month.

What happens when spam and real mail mix together?

I have another mailbox, that belongs to my second non-work/school related email account. I acquired this address back in 1993 and have kept it for those 20 years. This mailbox is behind a commercial filter, but as this address was on the internet long before commercial mail or spam was a reality, it still gets a lot of spam. This mailbox is also associated with a couple discussion lists, ones I follow but don’t post on very often.
Last year I started a project to look at what happens when you unsubscribe from spam. I unsubscribed from every message coming into this mailbox for a month. Then I recorded every message that came into this mailbox for 2 months. My goal was to then count the number of messages coming into this mailbox for the next 6 months, to see what the long term consequence of unsubscribing from spam.
Because I am counting spam, I’m leaving all the spam in that mailbox as unread mail until it’s recorded. Right now there are 2709 unread messages in that mailbox. The problem is, some of that mail is actually discussion mail. But in order to find it, I have to rely on searches to find the list mail. It means I’m a lot less likely to actually look in the mailbox to find the mail I want, it’s just too painful and annoying to do it, and I don’t care that much about the discussions. If I had to do this with all my discussion lists, I’d probably declare mailbox bankruptcy and find myself a new email address. It’s that annoying.

Spam filters are a net benefit.

My point is that filters make and keep email useable. Unfiltered mailboxes quickly fill up with unsolicited mail that overwhelms any commercial or personal usage.

Legitimate mail in spamfilters

It can be difficult and frustrating for a sender to understand they whys and wherefores of spam filtering. Clearly the sender is not spamming, so why is their mail getting caught in spam filters?
I have a client that goes through this frustration on rare occasions. They send well crafted, fun, engaging content that their users really want. They have a solid reputation at the ISPs and their inbox stats are always above 98%. Very, very occasionally, though, they will see some filtering difficulties at Postini. It’s sad for all of us because Postini doesn’t tell us enough about what they’re doing to understand what my client is doing to trigger the filters. They get frustrated because they don’t know what’s going wrong; I get frustrated because I can’t really help them, and I’m sure their recipients are frustrated because they don’t get their wanted mail.
Why do a lot of filter vendors not communicate back to listees? Because not all senders are like my clients. Some senders send mail that recipients can take or leave. If the newsletter shows up in their inbox they may read it. If the ad gets in front of their face, they may click through. But, if the mail doesn’t show up, they don’t care. They certainly aren’t going to look for the mail in their bulk folder. Other senders send mail that users really don’t want. It is, flat out, spam.
The thing is, all these senders describe themselves as legitimate email marketers. They harvest addresses, they purchase lists, they send mail to spamtraps, and they still don’t describe themselves as spammers. Some of them have even ended up in court for violating various anti-spam laws and they still claim they’re not spammers.
Senders are competing with spammers for bandwidth and resources at the ISPs, they’re competing for postmaster attention at the ISPs and they’re competing for eyeballs in crowded inboxes.
It’s the sheer volume of spam and the crafty evilness of spammers that drives the constant change and improvement in spamfilters. It’s tough to keep up with the spamfilters because they’re trying to keep up with the spammers. And the spammers are continually looking for new ways to exploit recipients.
It can be a challenge to send relevant, engaging email while dealing with spamfilters and ISPs. But that’s what makes this job so much fun.

Filtering is not just about spam

A lot of filters started out just as filters against spam. But over the years they’ve morphed into more general blocks against dangerous or problematic email. There’s a lot of crime and bad behavior on the internet, much of it using email as a conduit or vector. Filtering is so much more than stopping spam now. It’s as much, or more, about stopping crime.
Email filters are essential to protect us from scammers. Sometimes I forget this, and then I read about a grandmother getting swindled by a Nigerian scammer and ending up dead.
There are real consequences to poor filtering and there is real crime facilitated by email. It’s easy to forget this as we deal with the email that gets caught in filters when they shouldn’t.
Filters are one of the first lines of defense against online crime.
Not only does filtering stop crime, but they also keep email working. An unfiltered mail stream is an ugly, unreadable, unworkable mess.

Working as intended

There’s a certain type of sender that thinks every ISP block or email delivered to the bulk folder is a false positive. They’re so sure that the filters aren’t actually supposed to catch their mail that they’ll spend any amount of money and do every possible thing to get their mail to the inbox.
The problem for these senders, though, is that their mail is exactly the type of mail filters are designed to catch. They’re sending mail without recipient permission. I’m not talking about the lists that get a few typos or problem addresses on them. I’m talking about senders that buy and trade mailing lists. I’m talking about senders that don’t believe they have to have permission to send mail.
This mail getting filtered is a sign that the filters are working as intended. They’re keeping the unsolicited email out.
A lot of us take for granted that all commercial mail, at least that isn’t selling fake watches or herbal viagra, is always sent with permission. But there’s an awful lot of mail out there that doesn’t even have a minor fig leaf of permission. Filters stop that mail. And senders have very little recourse when they do.