Troubleshooting: part 3

As I continue to think about how people troubleshoot email delivery I keep finding other things to talk about. Today we’re going to talk about the question most folks start with when troubleshooting delivery. “Did ISP change something?”

image of a head with gears and ideas floating around it

At least once a week I check some delivery or email fora and some form of the question is sitting there.

“Did X change something? We haven’t done anything different and our delivery went way down overnight.”

Did Y change their filters? Our delivery is tanking and all our authentication is fine.”

Anyone hear of a change at Z? We have been having increasing difficulty reaching the inbox and we don’t understand why. Looking for suggestions.

In reality, the answer to this question Does Not Matter and asking it is only going to delay actually resolving your delivery issue.

When filters change

The reality is, filters are continually changing. ISPs and filtering companies are always tuning filters. These changes are roughly in 3 categories.

  • Ongoing tweaking and improvement to provide a better experience for their users
  • Changes done to address a emergent threat (Yahoo deploying p=reject is one example of this)
  • Specific changes to catch a type of spam they had previously been unable to effectively identify and filter.

Filters are not static. They are continually adjusting based on a number of things. We can always assume the answer to the question is yes. Something changed. Now what?

There are basically 3 situations here.

  • The filters did something unexpected and caught mail it wasn’t intended to catch, causing recipients to complain to the ISP.
  • The filter change was intentional but caught more mail than was intended, causing recipients to complain to the ISP.
  • The filter change was intentional and caught exactly the mail that was intended and the recipients didn’t care enough to notice that mail was missing.

In the first two cases, the ISP is going to fix things. They’re going to listen to their users and adjust the filters. In the first case, I expect to see changes and rollback within 24 – 48 hours. In the second, I expect to see changes in 24 – 96 hours.

The third case is the interesting one. Does anyone care about mail they don’t care about going to the bulk folder? Folks sending mail, even opt-in mail, that the users don’t complain about when it’s missing is the definition of grey mail. Filter maintainers listen to their users. If users complain they’ll change things, if users don’t complain they’ll assume the filters are working as intended.

The answer to the question did the filters changed tells you nothing. Of course the filters changed. Either they’re doing something that the maintainers don’t intend, which means they’ll be fixed or they’re catching mail they’re intended to catch.

Instead of asking if the filters changed, flip the question. Why are my users not interested enough in my mail to notice it when it’s gone? Start your troubleshooting from that perspective.

Related Posts

Feedback loops

There are a lot of different perspectives on Feedback Loops (FBLs) and “this is spam” buttons across the email industry.
Some people think FBLs are the best thing since sliced bread and can’t figure out why more ISPs don’t offer them. These people use use the data to clean addresses off their lists, lower complaints and send better mail. They use the complaints as a data source to help them send mail their recipients want. Too many recipients opted out on a particular offer? Clearly there is a problem with the offer or the segmentation or something.
Other people, though, think the existence of “this is spam” buttons and FBLs is horrible.  They call people who click “this is spam” terrorists or anti-commerce-net-nazis. They want to be able to dispute every click of the button. They think that too many ISPs offer this is spam buttons and too many ESPs and network providers pay way to much attention to complaints. The argue ISPs should remove these buttons and stop paying attention to what recipients think.
Sadly, I’m not actually making up the terminology in the last paragraph. There really are who think that the problem isn’t with the mail that they’re sending but that the recipients can actually express an opinion about it and the ISPs listen to those opinions. “Terrorists” and “Nazis” are the least of the things they have called people who complain about their mail.
One of the senior engineers at Cloudmark recently posted an article talking about FBLs and “this is spam” buttons. I think it’s a useful article to read as it explains what value FBLs play in helping spam filters become more accurate.

Read More

Why do ISPs do that?

One of the most common things I hear is “but why does the ISP do it that way?” The generic answer for that question is: because it works for them and meets their needs. Anyone designing a mail system has to implement some sort of spam filtering and will have to accept the potential for lost mail. Even the those recipients who runs no software filtering may lose mail. Their spamfilter is the delete key and sometimes they’ll delete a real mail.
Every mailserver admin, whether managing a MTA for a corporation, an ISP or themselves inevitably looks at the question of false positives and false negatives. Some are more sensitive to false negatives and would rather block real mail than have to wade through a mailbox full of spam. Others are more sensitive to false positives and would rather deal with unfiltered spam than risk losing mail.
At the ISPs, many of these decisions aren’t made by one person, but the decisions are driven by the business philosophy, requirements and technology. The different consumer ISPs have different philosophies and these show in their spamfiltering.
Gmail, for instance, has a lot of faith in their ability to sort, classify and rank text. This is, after all, what Google does. Therefore, they accept most of the email delivered to Gmail users and then sort after the fact. This fits their technology, their available resources and their business philosophy. They leave as much filtering at the enduser level as they can.
Yahoo, on the other hand, chooses to filter mail at the MTA. While their spamfoldering algorithms are good, they don’t want to waste CPU and filtering effort on mail that they think may be spam. So, they choose to block heavily at the edge, going so far as to rate limit senders that they don’t know about the mail. Endusers are protected from malicious mail and senders have the ability to retry mail until it is accepted.
The same types of entries could be written about Hotmail or AOL. They could even be written about the various spam filter vendors and blocklists. Every company has their own way of doing things and their way reflects their underlying business philosophy.

Read More

Content, trigger words and subject lines

There’s been quite a bit of traffic on twitter this afternoon about a recent blog post by Hubspot identifying trigger words senders should avoid in an email subject line. A number of email experts are assuring the world that content doesn’t matter and are arguing on twitter and in the post comments that no one will block an email because those words are in the subject line.
As usually, I think everyone else is a little bit right and a little bit wrong.
The words and phrases posted by Hubspot are pulled out of the Spamassassin rule set. Using those words or exact phrases will cause a spam score to go up, sometimes by a little (0.5 points) and sometimes by a lot (3+ points). Most spamassassin installations consider anything with more than 5 points to be spam so a 3 point score for a subject line may cause mail to be filtered.
The folks who are outraged at the blog post, though, don’t seem to have read the article very closely. Hubspot doesn’t actually say that using trigger words will get mail blocked. What they say is a lot more reasonable than that.

Read More