Following the SMTP rules

An old blog post from 2013, that’s still relevant today.
“Blocked for Bot-like Behavior”
An ESP asked about this error message from Hotmail and what to do about it.
“Bot-like” behaviour usually means the sending server is doing something that bots also do. It’s not always that they’re spamming, often it’s a technical issue. But the technical problems make the sending server look like a bot, so the ISP is not taking any chances and they’re going to stop accepting mail from that server.
If you’re an ESP what should you look for when tracking down what the problem is?
First make sure your server isn’t infected with anything and that you’re not running an open relay or proxy. Second, make sure your customers aren’t compromised or have had their accounts hijacked.
Then start looking at your configuration.
HELO/EHLO values

  • are you using a consistent one for each IP or are the values changing?
  • are you using a FQDN (fully qualified domain name, i.e., a.example.com) for the HELO?
  • are you using a bare IP for the HELO/EHLO?
  • does that FQDN match the rDNS of the IP?

IP issues

  • does your IP have rDNS?
  • does that rDNS in any way look like it might be dynamic?
  • how many IPs are you using to send the same message?

Header issues

  • are you using a consistent hostname in the envelope From?
  • is that hostname similar to the hostname in the IP address?
  • are you sending a MessageID?
  • are you authenticating?
  • is that authentication correct?

Content

  • are you sending small amounts of the same content over different IP addresses?
  • is the content you’re sending being sent by other entities?
  • are any of the URLs you’re linking to infected with anything?
  • are any of the URLs you’re linking to serving ads that might be spreading viruses?

Sending issues

  • are you closing connections promptly or are you holding them open?
  • are you opening connections from different IPs and sending the same content at the same time?

The major bot-like behaviours are sending small numbers of messages from many different IP addresses, and using bad HELO/EHLO values. Even small senders using shared pools can trigger this filter at Hotmail. Try not to split small volumes of mail over multiple IPs whenever possible and particularly when you are getting this error message.
 

Related Posts

Confusing the engineers

We went camping last weekend with a bunch of friends. Had a great time relaxing on the banks of the Tuolumne River, eating way too much and visiting.
On Saturday I was wearing a somewhat geeky t-shirt. It said 554: abort mission. (Thank you MessageSystems). At some point on Saturday every engineer came up to me, read my shirt and then looked at me and said “That’s not HTTP.”
That lead to various discussions about how their junior engineers don’t actually know SMTP at all. Why? Because the SMTP libraries just work. Apparently the HTTP libraries aren’t that great, so folks have to learn more about HTTP to troubleshoot and use them.
I’m sure there’s a joke in there somewhere: A Kindle engineer, an Android engineer and a robot engineer walk into a campsite…
EmailFilters_boxes_forblogIt did leave me thinking, though, about how it’s not that easy to run your own mail server these days. Gone are the days when running your own server was cost effective and easy. These days, there is just too much spam coming in. Crafting filters is a skilled job. It’s not that hard to run good filters. But to run good filters takes time to do well.
There are also a lot of challenges to sending mail. One of the discussions I had at the campsite was how hard it was to configure outbound mail. The engineer was helping a friend set up a website and trying to get the website to send notifications to the friend. But without setting up authentication the mail kept silently failing.
Of course, we do run our own mail server. But it’s our job and, in many ways, it keeps us honest. We don’t run many filters meaning we see what spammers are doing and can use our own experiences to better understand what commercial filters are dealing with.
For most people, though, I really think using a service is the right solution. Find one with filters that meet your needs and just pay them to deal with the headache.
 

Read More

The best time to send email

This subject comes up over and over again. Many senders are convinced clock_at_sign that there is a best time to send email. Countless research hours have been dedicated to finding that best time to send email. Numerous blog posts discuss what the best time to send email is.
From my perspective, there are better places for senders to spend time than figuring out what the exact right time is.But, senders still ask when the best time to send mail is.
There are a lot of reasons I can come up with as to why there’s no best time to send email. But the really big one is that when you send a mail has no impact on when it gets delivered.  There are multiple steps between hitting the send button and the mail being delivered to the inbox totally outside the control of the individual sender.
Email is designed as “store-and-forward.” This means there are potential delays at multiple steps inside the process.
Sending queues are called queues for a reason. Emails are sent out individually, particularly when an ESP uses VERP as part of its sending. There is actually a time overhead for making a connection to a recipient server and sending the email.
Receivers have queues, too. They can only accept so many incoming connections at a time. They have limited resources to accept all the mail their users want.
Receivers may delay mail between accepting it at the MX and delivering it to the inbox. This isn’t ideal and it’s not usual, but it can happen.
Recipients using IMAP accounts may not check mail regularly. They may only collect mail a few times a day.
These are only a few of the reasons that send time doesn’t necessarily equate with delivery time. Of course, 99% of the time email is mostly instantaneous. The internet is robust enough that a message sent is delivered seconds later. I see it happen all the time, when colleagues and I send email during calls. But, when mail fails, it sometimes fails spectacularly. Back in the dark ages (of the early 90s) I had an email that took almost a year to get to the recipients. Best I can tell, it got stuck somewhere in the depths of a machine in the middle of the university mail system. Eventually that system fell over and someone noticed and rebooted it (maybe it was walled up somewhere?).  The reboot shook my message out of where ever it was stuck.
 

Read More

When did the reject happen?

conversation_for_blogEarlier today I approved a comment from Mike on a post about problems at AOL from 2012. The part of the comment that caught my attention:

Read More