Deliverability and IP addresses

Almost 2 years ago I wrote a blog post titled The Death of IP Based Reputation. These days I’m even more sure that IP based reputation is well and truly dead for legitimate senders.
There are a lot of reasons for this continued change. Deliverability is hard when some people like the same email other people think is spam

Improved computing power

I touched on the increase in computing power in my 2013 post. The power and the complexity of filters in even greater now than then. Filters can sort through all sorts of variables faster than ever. They’re now fast enough to keep up with the high volume of email received at most incoming servers.

Planning for IPv6

In a world with 340 trillion trillion trillion IP addresses IP based reputation isn’t going to work very well. We’re not yet at the point where a lot of email is going out over IPv6, but many people are working on filtering already. When a spammer can use an IP per email, IPs are just not useable for blocking email. The smart folks who develop and maintain spam filters are planning ahead and developing technologies that don’t rely on the IP address. These technologies are being developed on current IPv4 infrastructure, and we’re seeing the effects.

Consumer demand

Years ago I was sitting in a meeting talking with a lot of very smart people about filtering. During the discussion a representative from Yahoo! mentioned that it was hard to make global decisions about email when some people really wanted it and some people really didn’t want it. Consumers want the email they want and they don’t want email they don’t want. This demand has helped drive filters away from the all or nothing approach.

Better anti-spam programs

Spam and other types of malicious emails are a global problem. Criminals are using email to gain access to individual computers and using that access to launch bigger compromises. Many of the major compromises over the past few years started with email, including Target, The Oak Ridge National LaboratoryICANN and even the RSA. Over the past few years filtering companies and organizations have worked closely with law enforcement across the world. These groups have identified and removed a number of criminal gangs from the internet, and even society. In addition to the legal work being done, many legitimate ISPs and network providers police and disconnect spammers and other email abusers. Overall, the effect is to restrict truly bad senders and criminals away from legitimate IPs. We now have bad IP neighborhoods and good IP neighborhoods. Senders in good IP neighborhoods see less IP blocking than senders in bad IP neighborhoods.

Better communication and partnerships

Filtering companies, ESPs, ISPs and large senders are working together on the spam problem. M3AAWG is part of that by simply giving diverse groups a place to talk, interact and develop relationships. These relationships were tentative at first, but continue to develop. M3AAWG is not the sole reason for the increase in cooperation. A number of individuals, on both the sending and the filtering side, took a risk and reached across the divide to work together. These relationships have changed how filters look at senders and how senders look at filters. This leads to less abuse and less need for IP based filtering.

Domain based authentication

Technologies like DKIM and SPF and DMARC have given filters and ISPs the ability to trust more data than the connecting IP address. These frameworks provide other data points that can be trusted. Trusted data can be used for reputation and that reputation can be used to filter email.
IP reputation was always a big, big club. It could affect lots of email from different senders (think shared IPs at an ESP or an ISPs outgoing servers). But most IPs don’t send all good email or all bad email. Most IPs send mixed email, and IP based reputation is really bad at dealing with that.
Newer filtering technologies mean that IP reputation isn’t as important for deliverability as it used to be. IP reputation is important for the SMTP transaction, and it will always be. There are too many “blackhat” IPs for blocking to go away completely. But once a receiver has accepted an email the IP reputation has done its part for delivery. The receiver knows that the email is coming from an IP with a minimum reputation. All the other reputation factors (domains, links, content, images) influence where that email ends up.
We don’t need the IP club any more. This is good news for good senders. Deliverability now depends more on that specific email and that specific recipient than the IP the message was sent from. This means senders really can focus more on meeting the needs of their recipients and less time worried about the health of their sending IP.

Related Posts

A series of tubes

ASeriesofTubes_thumb
The Internet and pundits had a field day with Senator Stevens, when he explained the Internet was a series of tubes.
I always interpreted his statement as coming from someone who demanded an engineer tell him why his mail was delayed. The engineer used the “tube” metaphor to explain network congestion and packets and TCP, and when the Senator tried to forward on the information he got it a little wrong. I do credit the Senator with trying to understand how the Internet works, even if he got it somewhat wrong. This knowledge, or lack there of, drove his policy positions on the issue of Net Neutrality.
In the coming years, I believe we’re going to be seeing more regulations around the net, both for individuals and for corporations. These regulations can make things better, or they can make things worse. I believe it’s extremely important that our elected officials have a working understanding of the Internet in order to make sensible policy. This understanding doesn’t have to be in their own head, they can hire smart people to answer their questions and explain the implications of policy.
Apparently I’m not the only one who thinks it is important for our elected officials to have a working knowledge of technology. Paul Schreiber put up a blog post comparing the website technology used by the current Presidential candidates. Do I really expect the candidate to be involved in decisions like what domain registrar or SSL certificate provider to use? No. But I do expect them to hire people who can create and build technology that is within current best practices.

Read More

April 2015: The Month in Email

We started the month with some conversations about best practices, both generally looking at the sort of best practices people follow (or don’t) as well as some specific practices we wanted to look at in more depth. Three for this month:

Read More

Role accounts, ESPs and commercial email

There was a discussion today on a marketing list about role accounts and marketing lists. Some ESPs block mail to role accounts, and the discussion was about why and if this is a good practice. In order to answer that question, we really need to understand role accounts a little more.

Read More