Recent Posts

Are you ready for DMARC?

The next step in email authentication is DMARC. I wrote a Brief DMARC primer a few years ago to help clear up some of the questions about DMARC and alignment. But I didn’t talk much about where DMARC was going. Part of the reason was I didn’t know where things were going and too much was unclear to even speculate.
We’re almost 2 years down the line from the security issues that prompted Yahoo to turn on p=reject in their DMARC record. This broke a lot of common uses of email. A lot of the damage created by this has been mitigated and efforts to fix it continue. There’s even an IETF draft looking at ways to transfer authentication through mailing lists and third parties.
For 2016, DMARC alignment is going to be a major factor in deliverability for bulk email, even in the absence of a published DMARC record.

What do you think about these hot button issues?

It’s been one of those weeks where blogging is a challenge. Not because I don’t have much to say, but because I don’t have much constructive to say. Rants can be entertaining, even to write. But they’re not very helpful in terms of what do we need to change and how do we move forward.
A few different things I read or saw brought out the rants this week. Some of these are issues I don’t have answers to, and some of them are issues where I just disagree with folks, but have nothing more useful to say than, “You’re wrong.” I don’t even always have an answer to why they’re wrong, they’re just wrong.
I thought today I’d bring up the issues that made me so ranty and list the two different points of views about them and see what readers think about them. (Those of you who follow me on Facebook probably know which ones my positions are, but I’m going to try and be neutral about my specific positions.)

Troubleshooting delivery is hard, but doable

Even for those of us who’ve been around for a while, and who have a lot of experience troubleshooting delivery problems things are getting harder. It used to be we could identify some thing about an email and if that thing was removed then the email would get to the inbox. Often this was a domain or a URL in the message that was triggering bulk foldering.
Filters aren’t so simple now. And we can’t just randomly send a list of URLs to a test account and discover which URL is causing the problem. Sure, one of the URLs could be the issue, but that’s typically in context with other things. It’s rare that I can identify the bad URLs sending mail through my own server these days.
There are also a lot more “hey, help” questions on some of the deliverability mailing lists. Most of these questions are sticky problems that don’t map well onto IP or domain reputation.
One of my long term clients recently had a bad mail that caused some warnings at Gmail.
We tried a couple of different things to try and isolate the problem, but never could discover what was triggering the warnings. Even more importantly, we weren’t getting the same results for identical tests done hours apart. After about 3 days, all the warnings went away and all their mail was back in the inbox.
It seemed that one mailing was really bad and resulted in a bad reputation, temporarily. But as the client fixed the problem and kept mailing their reputation recovered.
Deliverability troubleshooting is complicated and this flowchart sums up what it’s like.

Here at Word to the Wise, we get a lot of clients who have gone through the troubleshooting available through their ESPs and sometimes even other deliverability consultants. We get the tough cases that aren’t easy to figure out.
What we do is start from the beginning. First thing is to confirm that there aren’t technical problems, and generally we’ll find some minor problems that should be fixed, but aren’t enough to cause delivery problems. Then we look at the client’s data. How do they collect it? How do they maintain it? What are they doing that allows false addresses on their list?
Once we have a feel for their data processes, we move on to how do we fix those processes. What can we do to collect better, cleaner data in the future? How can we improve their processes so all their recipients tell the ISP that this is wanted mail?
The challenging part is what to do with existing data, but we work with clients individually to make sure that bad addresses are expunged and good addresses are kept.
Our solutions aren’t simple. They’re not easy. But for clients who listen to us and implement our recommendations it’s worth it. Their mail gets into the inbox and deliverability becomes a solved problem.

BlueHornet spun off from Digital River

Earlier this week, the investment firm Marlin Equity Partners announced they purchased BlueHornet Networks from Digital River. BlueHornet has been around for quite a while. In 2004 they were acquired by Digital River and run as a wholly owned subsidiary.
Congrats to the folks working at BlueHornet.

But my purchased list is TARGETED!!!

listshoppingcart I hear this all the time. But, y’know what? It’s BS. Total BS.
In the last month, I’ve gotten “targeted” messages (that escaped my filters) from the following companies who purchased lists.

Clickthrough forensics

When you click on a link in your mail, where does it go? Are you sure?
HTTP Redirects
In most bulk mail sent the links in the mail aren’t the same as the page the recipients browser ends up at when they click on it. Instead, the link in the mail goes to a “click tracker” run by the ESP that records that that recipient clicked on this link in this email, then redirects the recipients web browser to the link the mail’s author wanted. That’s how you get the reports on how many unique users clicked through on a campaign.
In the pay-per-click business that’s often still not the final destination, and the users browser may get redirected through several brokers before ending up at the final destination. I walked through some of this a few years ago, including how to follow link redirection by hand.
HTTP Forensics
Evil spammers sometimes deploy countermeasures against that approach, though – having links that will only work once or twice, or redirects that must be followed within a certain time, or javascript within an intermediate page or any of a bunch of other evasions. For those you need something that behaves more like a web browser.
For serious forensics I might use something like wireshark to passively record all the traffic while I interact with a link from inside a sandboxed browser. That’s not terribly user-friendly to use or set up, though, and usually overkill. It’s simpler and usually good enough to use a proxy to record the web traffic from the browser. There are all sorts of web proxies, used for many different things. What they have in common is that you configure a web browser to talk to a proxy and it’ll send all requests to the proxy instead of to the actual website, allowing the proxy to make any changes it wants as it forwards the requests on and the results back.
For investigating what a browser is doing the most useful proxies are those aimed at either web developers debugging web apps or ~~crackers~~ penetration testers compromising web apps. Some examples are Fiddler (Windows), Cellist (OS X, commercial), mitmdump (OS X, linux, Windows with a little work), Charles (anything, commercial) or ZAP (anything).
I’m going to use mitmdump and Firefox. You don’t want to use your main browser for this, as the proxy will record everything you do in that browser while you have it configured – and I want to keep writing this post in Safari as I work.

November 2015: The month in email

As we head into the last month of the year, we look back at our November adventures. I spoke twice this month, first at Message Systems Insight in Monterey (my wrap-up post is here) and then with Ken Magill at the at the 2015 All About eMail Virtual Conference & Expo (a short follow-up here, and a longer post on filters that came out of that discussion here.). Both were fun and engaging — it’s always great to get a direct sense of what challenges are hitting people in the email world, and to help clear up myths and misconceptions about what works and doesn’t work in email marketing and delivery. I’m putting together my conference and speaking schedule for 2016 — if you know of anything interesting that should be on my radar, please add it in the comments, thanks!
In industry news, we noted a sharp uptick in CBL listings, and then posted about the explanation for the false positives. Steve wrote about an interesting new Certificate Authority (CA) called Let’s Encrypt, which looks to be a wonderful (and much-needed) alternative for certificates, and I put together some thoughts on SenderScore.
Steve and I did a few posts in parallel this month. First, Steve posted an interesting exercise in SPF debugging. Are you seeing mail from legitimate senders flagged as spam? This might be why. My investigative post was about ISP rejections, and how you can figure out where the block is occurring. In each case, you’ll get a glimpse of how we go about identifying and troubleshooting issues, even when we don’t have much to go on.
We each also wrote a bit about phishing. Steve posted a timely warning about spear phishing — malware attacks disguised as legitimate email from within your organization — and reminds all of us to be careful about attachments. With all of the more secure options for document sharing these days, it’s a lot easier to avoid the risk by maintaining a no-attachments policy in your company. And I wrote about how the Department of Defense breaking HTML links in email to help combat phishing. If your lists include military addresses (.mil), you may want to come up with a strategy for marketing to those recipients that relies less on a clickthrough call to action.
We amused ourselves a bit with a game of Deliverability Bingo, then followed up with a more serious look at the thing we hear all the time — “I’m sure they’ll unblock me if I can just explain my business model.” While an ESP abuse desk is unlikely to be swayed by this strategy, it is actually at the core of how we think about deliverability at Word to the Wise. Legitimate senders have many kinds of lists, many kinds of recipients, many kinds of marketing strategies, and many kinds of business goals. For us to help marketers craft sustainable email programs, we need to understand exactly what matters most to our clients.

CASL botnet take down

biohazardmail The CRTC served its first ever warrant as part of an international botnet takedown. The warrant was to take down a C&C (command and control) server for Win32/Dorkbot. International efforts to take down C&C servers take a lot of effort and work and coordination. I’ve only ever heard stories from folks involved but the scale and work that goes into these take downs is amazing.
Bots are still a problem. Even if we manage to block 99% of the botnet mail out there people are still getting infected. Those infections spread and many of the newer bots steal passwords, banking credentials and other confidential information.
This kind of crime is hard to stop, though, because the internet makes it so easy to live in one country, have a business in a third, have a shell corp in a fourth, and have victims in none of those places. Law enforcement across the globe has had to work together and develop new protocols and new processes to make these kinds of takedowns work.

Looking forward

The nice folks over at Sparkpost asked me and other email experts for some thoughts on what we think the most important issues in email will be in 2016.
I do think security is going to be a major, major change in delivery. From what I’ve seen there’s been a shift in the mindset of a lot of people. Previously a lot of folks in the email space were very accommodating to old systems and unauthenticated mail and were not quite ready to cut off senders that didn’t meet modern standards.

There were a lot of people who didn’t want to take any action that would break email. There are still a lot of people who think that breaking email is a bad thing and changes should be backwards compatible.
Then people started realizing not every change had to be backwards compatible.

There are a few reasons I think this attitude shift happened.

Buying lists costs more than just money

I’ve been talking to a lot of companies recently who are dealing with some major delivery challenges probably related to their practice of purchasing lists and then sending advertising to every address on the list. They assure me that their businesses would be non-viable if they didn’t purchase lists and it has to be that way.
Maybe that’s true, maybe it is more cost effective to purchase lists and send mail to them. I know, though, that their delivery is pretty bad. And that a lot of the addresses they buy never see their email. And that they risk losing their ESP, or they risk being SBLed, or they risk being blocked at Gmail, or they risk bulk foldering at Hotmail. There are a lot of risks to using purchased lists.
The reality is it’s only getting harder to mail to purchased lists and it’s getting more expensive to mail purchased lists. Paying for the list is a small part of the cost of using them.
Other costs incurred by companies using purchased lists include:
1) Having multiple ESPs. There are certainly legitimate reasons for companies to use different ESPs but there is a cost associated with it. Not only do they have to pay for duplicate services, but they spend a lot of employee time moving lists and recipients around to see who might have the better delivery today.
2) Multiple domains and brand new websites for every send. Landing pages are good marketing and are normal. But some ISPs track the IPs of the landing sites, and those IPs can get their own poor reputation. To get around it, senders using purchased lists often have to create new websites on new IPs for every send.
3) Complicated sending schedules. Sending schedules aren’t dictated by internal needs, they’re dictated by what ISP is blocking their IPs or domains (or even ESP) right now.
All of these costs are hidden, though. The only cost on the actual bottom line is the money they spend for the addresses themselves and that’s peanuts. Because, fundamentally, the folks selling addresses have no incentive to take any care in collecting or verifying the data. In fact, any verification they do only cuts into their profit, as buyers won’t actually pay for the verification and data hygiene and it also reduces the size of the lists they can sell.
And, no, data hygiene companies that look for traps and bounces and “bad addresses” don’t take a bad list and make it good. They just take a bad list and make it a little less bad. If the recipients don’t want the mail, all the hygiene in the world isn’t going to get that message into the inbox.
Outsourcing address collection to list selling companies is more expensive than it looks on paper. That doesn’t stop anyone from building a business around purchased lists, though.