Catching up from MAAWG SF

Had a great time a M3AAWG last week. So many familiar faces and a lot of new ones, too. I’ve got a lot of interesting stuff that I can share with readers over the next few days.
One of the things I have received permission to share is the new Office 365 IP delisting link. I botched the first time I posted it, so I’m going to try again. Office 365 IP Delisting Page. Many thanks to the Microsoft guys for getting this together for people.
While I’m talking about Microsoft, there is a bit of a problem with folks signing up with their FBL. Some people are finding that the process gets stuck and FBLs aren’t enabled. MS is aware of the issue and they are working on fixing it. As I know more I’ll share.
Unsurprisingly, authentication was a big topic of conversation, both in the hallways and in the sessions. There were some strong opinions stated. I think, though, that we’re pretty clear that we’re going to get to a more authenticated world. But we have some different opinions on how and how fast that’s going to happen.

Getting unblocked at Outlook.com
Email in 2020

Related Posts

Confusing the engineers

We went camping last weekend with a bunch of friends. Had a great time relaxing on the banks of the Tuolumne River, eating way too much and visiting.
On Saturday I was wearing a somewhat geeky t-shirt. It said 554: abort mission. (Thank you MessageSystems). At some point on Saturday every engineer came up to me, read my shirt and then looked at me and said “That’s not HTTP.”
That lead to various discussions about how their junior engineers don’t actually know SMTP at all. Why? Because the SMTP libraries just work. Apparently the HTTP libraries aren’t that great, so folks have to learn more about HTTP to troubleshoot and use them.
I’m sure there’s a joke in there somewhere: A Kindle engineer, an Android engineer and a robot engineer walk into a campsite…
EmailFilters_boxes_forblogIt did leave me thinking, though, about how it’s not that easy to run your own mail server these days. Gone are the days when running your own server was cost effective and easy. These days, there is just too much spam coming in. Crafting filters is a skilled job. It’s not that hard to run good filters. But to run good filters takes time to do well.
There are also a lot of challenges to sending mail. One of the discussions I had at the campsite was how hard it was to configure outbound mail. The engineer was helping a friend set up a website and trying to get the website to send notifications to the friend. But without setting up authentication the mail kept silently failing.
Of course, we do run our own mail server. But it’s our job and, in many ways, it keeps us honest. We don’t run many filters meaning we see what spammers are doing and can use our own experiences to better understand what commercial filters are dealing with.
For most people, though, I really think using a service is the right solution. Find one with filters that meet your needs and just pay them to deal with the headache.
 

Read More

The holiday mailing season

We’re half way through September and it seems way too early to start thinking about the holidays. But for marketers, even email marketers, planning should be starting now. This planning shouldn’t just be about content and targeting and segmentation, but should also cover deliverability.
Most retailers use email marketing to drive traffic to their websites during the holidays. Experian reported that in 2014 email was the second largest driver of traffic, behind search, to the Hitwise Retail 500. In recent years, though, some retailers have run afoul of filters during the holiday season, losing precious opportunities to reach potential buyers due to delivery problems.
Retailers should consider deliverability as a factor in their marketing strategy.
Choices about who, how, how much and when to email can and do significantly affect marketing. The good news is that smart marketers can use their understanding of filters as part of their strategic planning and avoid some of the bigger problems that have plagued retailers in the past.
In December 2012, retailers Gap and Gilt were listed on the Spamhaus Block List. Since then, other retailers have also had delivery and blocking problems during the holiday season, although none have been quite so public.
Delivery problems can have a significant impact on a retailer’s bottom line. Mark Zadon, the chairman of Zulily, blamed his company’s lower profits in Q3 2014 on changes at their unspecified email service provider. After that announcement, Zulily’s stock value dropped 15%. Zulily isn’t the only company to have email delivery problems affect business growth enough to be mentioned in SEC filings. “Various private spam blacklists have in the past reduced, and may in the future reduce, the effectiveness of our solutions and our ability to conduct our business, which may cause demand for our solutions to decline.”
Deliverability rules don’t change.
Some people argue that the increase in blocking during the holiday season is because the folks running the filters are attempting to sabotage retail marketing. The available evidence doesn’t support this conclusion. For webmail providers and consumer ISPs, the overarching rule for filters is to give users email they want and filter email users don’t want. The processes and techniques the ISPs and filter companies use don’t change during the holidays. A few years ago Return Path interviewed people at a number of providers and all agreed that the receivers don’t change during the holidays.
It is true that during the holiday season some retailers see an increase in delivery problems. These are mostly self-inflicted. The good news is that given the changes are happening at the sending end, there are things senders can do to minimize the impact of filters. It’s all in their control.
Mail volume increases for multiple reasons.
The volume of transactional email goes up because brick-and-mortar retailers collect addresses in the store and email receipts to shoppers. This often involves the shopper spelling out the address for a harried sales associate in the middle of a store blasting holiday music. Typos can, and do, happen. Even when shopping online, from the comfort of the couch, there is a risk of a mis-typed email address.
These typos hurt deliverability a few different ways. The receipt can go to the wrong person, causing a complaint and hurting the reputation of the sender. The receipt can go to a non-existent account, causing a bounce and hurting the reputation of the sender. Both of these things happen, and can hurt delivery if they happen in significant enough numbers. Of even more concern is when a receipt goes to a spamtrap. Enough trap hits or complaints and the sender risks blocking and delivery failures at one or more ISPs.
Many of the larger brick-and-mortar retailers have implemented processes to reduce the chance of bad addresses. Some ask the shopper to input their email address right into the credit card pad. Others show the address to the user on the register and have the user confirm it. These things do help lower the risk of problems and incorrect addresses. But they don’t resolve it completely. Verification services can weed out undeliverable addresses, but can’t really do anything to make sure a deliverable address is the right one.
Transactional email isn’t the only reason volume increases during the holiday season. The volume of marketing email goes up as well. Marketers increase their frequency, sometimes to ridiculous amounts. A few years ago, I was on a list for a cooking store. They increased their volume from 2x a week to 3x a day in the 3 weeks leading up to Thanksgiving. This may make perfect sense from their point of view, but some recipients just don’t want that much email.
In addition to increasing volume to current and engaged customers, retailers often look to older, unengaged lists during the holidays. This has a double negative effect. First, addresses that have gone dormant, whether they bounce or not, can drive reputation down. Second, sending to people after a long period of no email can result in increased complaint rates. Increased complaints, increased bounces, and increased email to abandoned addresses all drive reputation down.
Taken together it’s no wonder some retailers see an increase in deliverability problems during the holiday emailing season. The good news is that mailers have the ability to control and manage their deliverability, even as they manage the holiday volume.  

Read More

What to expect in 2016

WttWColorEye_forBlogI don’t always do predictions posts, even though they’re  popular. Most years I skip them because I don’t see major changes in the email space. And, I’m not the type to just write a prediction post just to post a prediction.
This year, though, I do see changes for everyone in the email space. Most of them center on finally having to deal with the technical debt that’s been accumulating over the past few years. I see ISPs and ESPs spending a lot of development effort to cope with the ongoing evolution authentication requirements.
When people started seriously looking at how to authenticate email, the first goal was getting organizations to implement the protocols. This was a practical concession; in order for a new protocol to be used it needs to be widely implemented. Phase one of authenticating email was simply about publishing protocols and getting organizations to use them.
During phase one, the organization that authenticated a mail hasn’t been important. In fact, the SPF spec almost guarantees that the ESP domain is the authenticated domain. In DKIM, the spec says any domain could sign as long as they could publish a public key in that domain’s domainkeys record.
ESPs took full advantage of this and lowered their own development overhead by taking most of the authentication responsibility on themselves. Their domains were in the 5321.from and they published the SPF records. Domains they control were in the d= and they generated and published the DKIM keys. Mail was authenticated without ESP customers having to do much.
We’ve hit the end of phase one. Most of the major players in the email space are authenticating outbound email. Many of the major players are checking authentication on the inbound. Phase one was a success.
We’re now entering phase two, and that changes thing. In phase two, SPF and DKIM are used as the foundation for user visible authentication. Neither SPF nor DKIM were designed to be user visible protocols. To understand what they’re authenticating you have to understand SMTP and email. Even now there are days when I begin talking about one of them and have to take a step back and think hard about what is being authenticated. And I use these things every day!
DMARC is the first of these end user visible protocols built on SPF and DKIM. It uses the established and widespread authentication to validate the user visible from address. This authentication requires that the d= value or the 5321.from address belong belong to the same domain in the visible from address. While you can pick whether the alignment between the visible from and the authentication is “strict” or “relaxed” you have no choice about the alignment.
Prior to DMARC no one really paid much attention to the domain doing the authentication. Authentication was a yes or a no question. If the answer was yes, then receivers could use the authenticated domain to build a reputation. But they weren’t really checking much in the way of who was doing the authentication.
In the push to deploy authentication, ESPs assumed the responsibility for authentication deployed ESPs took the responsibility and did most of the work. For many or most customers, authentication was as simple as clicking a checkbox during deployment. Some ESPs do currently let customers authenticate the mail themselves, but there’s enough overhead in getting that deployed that they often charged extra to cover the costs.
DMARC is rapidly becoming an expectation or even a full on requirement for inbox delivery. In order to authenticate with DMARC, the authenticating domain must be in the same domain space as the visible from. If senders want to use their own domain in the visible from, DNS records have to be present in that domain space. Whether it’s a SPF TXT record or a domainkeys record the email sender customer needs to publish the correct information in DNS. Even now, if you try to authenticate with DKIM through google apps, they require you to publish DNS records.
ESPs aren’t in a situation where they can effectively manage authentication alignment for all their customers. Hosting companies are in even worse shape when it comes to letting customers authenticate email. Developers are facing the fact they need to go back and rework their authentication code. Businesses are facing the fact they need to change their processes so customers can authenticate with DMARC.
It’s not just the infrastructure providers that are facing challenges with authentication. Senders are going to discover they can no longer hand authentication off to their ESPs and not worry about it. They’re going to have to get DNS records published by their own staff.
Getting DNS updates through some big companies is sometimes more difficult than it should be. I had one client a few years ago where getting rDNS changed to something non-generic took over a month. From an IT standpoint, changing DNS should require approvals and proper channels. Marketers may find this new process challenging.
And, if organizations want to publish reject policies for their domains, then they will have to publish records for every outside provider they use. Some of those providers can’t support DMARC alignment right now.
In 2016 a lot of companies will discover their current infrastructure can’t cope with modern authentication requirements. A lot of effort, both in terms of product development and software development, will need to be spent to meet current needs. This means a lot of user visible features will be displaced while the technical debt is paid.
These changes will improve the security and safety of email for everyone. It won’t be very user visible, which will give the impression this was a slow year for email development. Don’t let that fool you, this will be a pivotal year in email.

Read More