A bunch of folks today mentioned they were seeing poor reputation for formerly good reputations on Google Postmaster Tools. I’m seeing a lot of screen shots that look like this one.
It looks like something is going on over there that has nothing to do with actual reputation. Could be a reporting bug, could be a filtering problem. I’m not seeing people mention delivery problems, just that the reputation monitor is showing bad reputation.
A bit of a throwback post from Steve a few years ago. The problem has gotten a little better as some shortening companies are actually disabling spammed URLs, and blocking URLs with problematic content. I still don’t recommend using a public URL shortener in email messages, though.
Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.
M3AAWG has published a new version of the Senders Best Common Practices document and the contains a lot of new information since the original publication in 2008. The new document covers how to vet ESP customers, considerations when selecting a dedicated or share IP to send mail, and includes best practices on a number of technical processes.
The Senders Best Common Practices document is targeted at deliverability teams and email marketers. Any company that is sending marketing emails, using an Email Service Provider, or provides an email enabled platform, it’s always good to go back and periodically review your system to ensure nothing was missed and to stay up-to-date on all new recommendations.
A few of the recommendations include the use of the List-Unsubscribe header, publishing a clear WHOIS for domains used for sending mail, and how to process non-delivery report messages.
The List-Unsubscribe header provides an additional way for users to opt-out of email messages. Gmail and Outlook.com both use the presence of the list-unsubscribe header to provide a one-click button to allow the user to unsubscribe from the mailing list. Often enough, if a user cannot find an opt-out link, they’re marking the message as spam. Allowing a recipient to unsubscribe easily is critical to maintaining good delivery reputation.
A WHOIS is query to determine who is the registered user or assignee of a domain name. During a session at the most recent M3AAWG meeting, it was announced that spammers throw away 19 million domains per year. When a postmaster or abuse desk receive a complaint, they’ll often query to see who owns the domain the email was sent from or who owns the domains used in the hyperlinks. If the WHOIS record is out of date or set to private, this limits the ability for the postmaster or abuse desk to reach out to the owner of the domain.
Processing non-deliver reports is critical to maintaining a high delivery reputation. Many ESPs have an acceptable-use-policy that includes a bounce rate. Mailjet recommends a bounce rate of less than 8% and Mandrill recommends less than 5%. If a system is not in place to remove the hard bounces from your mailing list, the sender’s reputation will quickly deteriorate.
The Senders Best Common Practices document can be downloaded at M3AAWG.org.
Google Apps for ISPs is being shut down. While this was a scheduled end of life, apparently some users weren’t notified (always keep the contact email address up to date at your vendor!) and other users were told that it would be discontinued in July and were surprised when their service was turned off a month earlier than they expected.
I’ve not seen any reports of mail bouncing due to this yet, but it’s likely that some consumer ISPs will be scrambling to migrate to new email providers and their inbound mail may be a mess for a while. If you see domain-wide problems at consumer domains, check to see if their MXes point at the google aspmx cluster.
It seems to be a rolling shutdown, and some ISPs have apparently had their service extended by a few days or weeks, so issues may start with some domains throughout the month.