Email filtering: not going away.

VirusBlockI don’t do a whole lot of filtering of comments here. There are a couple people who are moderated, but generally if the comments contribute to a discussion they get to be posted. I do get the occasional angry or incoherent comment. And sometimes I get a comment that is triggers me to write an entire blog post pointing out the problems with the comment.
Today a comment from Joe King showed up for The Myth of the Low Complaint Rate.

laura, imagine situation where a postman is standing in front of your mailbox and is throwing your mail into the trash instead of your mailbox because he _thinks_ that this piece of mail is a junk.
thank God we not there, living in your bubble.
sooner or later congress will work on fixing internet mail and just like for fedex it is illegal to throw away your mail, it will be criminal offence for microsoft or yahoo to throw away your email.
and customers notice the flaw too. hotmail is a joke nobody uses it and yahoo mail numbers are on decline for years.

Let’s go with the easy bit first. Postal mail is not email. The sender of a letter pays the post office or the package delivery service to deliver that mail to recipients. In fact, the entire postal system is paid for by senders. This isn’t the case with email (or, really, any internet traffic, thus the battle over net neutrality). Sure, senders pay for their connectivity. But receivers pay for their connectivity as well. Some pay directly for their email address as part of their broadband or dialup accounts. Businesses pay for business emails in hardware or hosted solutions. Even free mail users (including Yahoo/Rocketmail users like Joe up there) “pay” for email by viewing advertising.
The postal world isn’t some sort of automatic delivery system either. There are rules, things that cannot be sent through the mail and filters to limit the amount of illegal postal mail. In fact, there are much stronger consequences for breaking postal regulations than simply having mail thrown away. The USPS has an entire division dedicated to protecting recipients from harmful mail and from scams carried out through the USPS. Certain types of mail are prohibited by the postal service and they screen and stop mail prohibited by law.
In the email world, we’re not looking at things like bombs or dangerous biologics, but there is and will always be a place for filtering. There’s too much bad mail out there. In fact, email has been a common vector in many of the large scale hacks at companies. Phishing was a primary vector for the Target hack. The RSA hack relied on phishing to get in the door and compromise their root keys. Other notable attacks using spear phishing include ICANN and Home Depot.
Sure, these are big corporations, but the same sorts of viruses and phishes are aimed at individual users on free mail systems every day. They might not have access to the data that large companies do, but their systems are still worthy of protection. Right now a lot of users are protected from most of the viruses and phishes and malware and other harmful mail due to filtering. They don’t even know they’re protected from it, because it never comes through them. But there is a LOT of bad mail out there and it should not be delivered.
What’s more, recipients want their email filtered. They want ISPs to filter out junk mail. They make aggressive use of the “this is spam” button. “Too much spam” is one of the reasons people change email addresses.
Finally, ISPs are major corporations with shareholders, bottom lines and lobbyists. Spam costs these ISPs money, real money, to handle. The ISPs have always been on the side of being able to block bad mail. They’re not going to let their Congress pass a law that requires them to deliver every incoming email message.
I don’t believe that there will be a law requiring ISPs to open the floodgates and deliver every piece of email. Even if there were, an unfiltered mailbox is a difficult thing to manage. I know, we run limited filters and I end up in my junk folder at least weekly because I got a little ^J happy and junked a message I shouldn’t have.

Related Posts

URL reputation and shorteners

A bit of  a throwback post from Steve a few years ago. The problem has gotten a little better as some shortening companies are actually disabling spammed URLs, and blocking URLs with problematic content. I still don’t recommend using a public URL shortener in email messages, though.
Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.

Read More

The 5 stages of a Spamhaus listing

Courtesy of Spencer over at Experian.
The 5 Stages of Recovery

Read More

Top Commented Blog Posts on WttW in 2014

Here are the top 6 most commented on blog topics our Industry News & Analysis blog.

Read More