Recent Posts

What … is your name?

For some reason otherwise legitimate ESPs have over the years picked up a habit of obfuscating who they are.
I don’t mean those cases where they use a customers subdomain for their infrastructure or bounce address. If the customer is Harper Collins then mail “from” @bounce.e.harpercollins.com sent from a server claiming to be mail3871.e.harpercollins.com isn’t unreasonable. (Though something in the headers that identified the ESP would be nice).
No, I mean random garbage domains created by an ESP to avoid using their real domains in the mail they send and in their network infrastructure. This isn’t exactly snowshoe behaviour. They’re not really hiding anything terribly effectively from someone determined to identify them – the domains are registered with real contact information, and the IP addresses the mail is sent from are mostly SWIPped accurately – but they do prevent a casual observer from identifying the sender.
Silverpop has registered over 9,000 domains in .com that are just “mkt” followed by some random digits that they use for infrastructure hostnames, bounce addresses and click-tracking links. Apart from anything else, it’s a terrible waste of domain name space to use links.mkt1572.com where they could just as well use links1572.silverpop.com or links.mkt1572.silverpop.com.
For what they’re paying just for domain name registration and management they could probably hire multiple full time employees.
And Marketo has registered over 17,000 domains in .com that are just “mkto-” followed by what looks like a location code.
(I’m not picking on Marketo and Silverpop in particular – several other notable ESPs do the exact same thing – they’re just relevant to the end of the story).
Using garbage domains like this makes you look more like a snowshoe spammer at first glance than a legitimate ESP.
It also makes it much harder for a human glancing at your headers to correctly identify a responsible party …
… which is probably why abuse@marketo are rather tired of receiving misdirected complaints about spam sent by Silverpop from machines called something like mkt1572.com.

Meltdown & Spectre, Oh My

If you follow any infosec sources you’ve probably already heard a lot about Meltdown and Spectre, Kaiser and KPTI. If not, you’ve probably seen headlines like Major flaw in millions of Intel chips revealed or Intel sells off for a second day as massive security exploit shakes the stock.

What is it?
These are all about a cluster of related security issues that exploit features shared by almost all modern, high performance processors. The technical details of how they work are fascinating if you have a background in CPU architecture but the impact is pretty simple: they allow programs to read from memory that they’re not supposed to be able to read.
That might mean that a program running as a normal user can read kernel memory, allowing a malicious program to steal passwords, authentication cookies or even the entire state of the kernels random number generator, potentially allowing it to compromise encryption.
Or it might mean a program running on a virtual machine being able to escape from the sandbox the virtual machine’s hypervisor keeps it in and reading memory of other virtual machines that are running on the same hardware. A malicious user could sign up for a cloud service, such as Amazon EC2 Google Code Engine or Microsoft Azure, repeatedly create temporary virtual machines and grovel through all the other virtual machines running on the same hardware to steal, login credentials or TLS private keys.
Or it might mean a malicious piece of javascript running in a browser from a hostile website or a malicious banner ad being able to steal secrets and credentials not just from your web browser, but from any other software running on your laptop.
It’s pretty bad.
Meltdown and Spectre
One variant has been given the snappy name Meltdown. It (mostly) affects Intel CPUs, and is trivial to exploit reliably by unskilled skript kiddies. It can be mitigated at the operating system level, and all major operating system vendors are doing so, but that mitigation will have significant impact on performance – perhaps 20% slower for common workloads.
The other variant has been named Spectre. It’s more subtle, relying on measuring how long it takes to run carefully crafted code. Whether the code is fast or slow tells the malicious actor whether a particular bit of forbidden memory is zero or one, allowing them to step through reading everything they want. This is likely to be harder to exploit reliably, but is also going to be much harder to mitigate reliably in software (I’ve seen some speculation that it might be impossible to mitigate – I’m pretty sure that’s not true, but it is going to be difficult to do so reliably and will probably have significant performance impact). It affects pretty much everything, including AMD processors (despite what their PR flacks would like you to believe).
What should you do

As a typical end user you should apply your security patches as normal to mitigate Meltdown. macOS was patched on December 6th, the Windows kernel has mitigation in place. The latest release candidate of the Linux kernel has mitigation patches in place, which’ll presumably trickle out to various distributions over the next few days.
You should also update your browser. One nasty vector Spectre can use is timing attacks from malicious javascript. Chrome and Firefox have partial mitigation in their mainline development, and Microsoft have announced fixes for IE11 and Edge.
Keep updating your ‘phones. At least some of the ARM chips in iPhone and Android are vulnerable, and the more constrained ‘phone environment may make targeted attacks more likely.
If you’re using any virtual machines or cloud hosted services then your provider has probably already done rolling reboots so they can patch their hypervisors to mitigate Meltdown. You’ll still need to update your kernel yourself, to protect against attacks within your machine, even though your provider has patched their hypervisors.
Performance (and Email)
The operating system level mitigation for Meltdown works by having the CPU throw away a bunch of information every time the thread of execution goes from the kernel back to the application. Most common applications will switch between kernel code and application code a lot so this has a significant performance impact.
Initial tests with PostgreSQL show slowdowns as bad as 23%, but more realistic workloads look to be maybe 5-15% slower, depending on the workload and the hardware features available.
I wondered whether there’d be much impact on network service performance, so I set up a test network with a couple of mailservers running latest release candidates of the Linux kernel. I sent mail from one to the other, using postfix, smtp-source and smtp-sink – smtp-source and -sink are test tools distributed with postfix that make it easy to send mail or to receive and discard mail.
I wasn’t really expecting to find any performance impact for something that was likely network limited, but ran some tests anyway, slinging a few million emails from one machine to the other and turning mitigation on and off on the sender and receiver. There wasn’t any performance impact that I could measure – if it’s there it was well below the noise floor.
So you’ll probably see slight performance degradation for some things, especially disk-heavy workloads, but nothing to worry too much about.

Google makes connections

One of the client projects I’m working on includes doing a lot of research on MXs, including some classification work. Part of the work involves identifying the company running the MX. Many of the times this is obvious; mail.protection.outlook.com is office365, for instance.

There are other cases where the connection between the MX and the host company is not as obvious. That’s where google comes into play. Take the domain canit.ca, it’s a MX for quite a few domains in this data set. Step one is to visit the website, but there’s no website there. Step 2 is drop the domain into google, who tells me it’s Roaring Penguin software.
In some cases, though, the domain wasn’t as obvious as the Roaring Penguin link. In those cases, Google would present me with seemingly irrelevant hosting pages. It didn’t make sense until I started digging through hosting documentation. Inevitably, whenever Google gave me results that didn’t make sense, they were right. The links were often buried in knowledge base pages telling users how to configure their setup and mentioning the domain I was searching for.
The interesting piece was that often it was the top level domain, not the support pages, that Google presented to me. I had to go find the actual pages. Based on that bit of research, it appears that Google has a comprehensive map of what domains are related to each other.
This is something we see in their handling of email as well. Gmail regularly makes connections between domains that senders don’t expect. I’ve been speaking for a while about how Gmail does this, based on observation of filtering behavior. Working through multiple searches looking at domain names was the first time I saw evidence of the connections I suspected. Gmail is able to connect seemingly disparate hostnames and relate them to one another.
For senders, it means that using different domains in an attempt to isolate different mainstreams doesn’t work. Gmail understands that domainA in acquisition mail is also the same as domainB in opt-in mail is the same as domainC in transactional mail. Companies can develop a reputation at Google which affects all email, not just a particular mail stream. This makes it harder for senders to compartmentalize their sends and requires compliance throughout the organization.
Acquisition programs do hurt all mail programs, at least at Gmail.

Happy 2018

This is the time of year when everyone starts posting their predictions for the coming year. Despite over a decade of blogging and close to 2500 blog posts, I have’t consistently written prediction articles here. Many years I don’t see big changes on the horizon, so there’s not a lot to comment on. Incremental changes are status quo, nothing earth shattering there. But I’ve been thinking about what might be on the horizon in 2018 and how that will affect email marketing.

Authentication is about Identity, not Virtue

I just got some mail claiming to be from “Bank of America <secure@bofasecure.com>”.
It passes SPF:

Organizational Domain

We often want to know whether two hostnames are controlled by the same person, or not.

November 2017: The Month in Email

We’re in the thick of the busiest time of the year for email. It’s been so busy, in fact, that we’ve seen some slowdowns and delivery issues across the email universe. It may be worth thinking about alternate strategies for end of year promotions beyond Black Friday and Cyber Monday.
I was delighted to chat with Julia Angwin for her ProPublica piece on subscription bombing and abuse prevention. Her piece is a good introduction to the topic, and very much worth reading.
ICYMI, I did a rough analysis of the data from our survey on Google Postmaster Tools. Stay tuned for more insights when I have a moment to explore this further.

[#INFOGRAPHIC] Email marketing trends 2018

It’s always an honor to be asked to provide quotes and thoughts with experts in the field. Sometimes the day to day gives me tunnel vision, but things like this give me the opportunity to think more globally. Hands down, though, the best part is seeing the final product and hearing what other folks have to say.
Go check out the full infographic.

About that DMARC "exploit"

A security researcher has identified a rendering flaw that allows for “perfect” phishing emails. From his website:

Deliverability is critical for marketing

It is increasingly clear that successful email marketing programs measure and emphasize deliverability. No longer is deliverability the crisis management team called when everything breaks. They’re part and parcel of an effective email marketing team.
Today I watched a bit of the EIS livestream where acquisition marketers were discussing their processes. Everyone of them talked about things that are critical for deliverability as core to their business.