Recent Posts

CASL Private Right of Action Delayed

Today the Canadian Government announced they were suspending the provision that allows individuals to sue marketers for violations of CASL.
Under these provisions, individual Canadian consumers had a private right of action. Any Canadian could sue any company that sent mail violating the law. This part of the law upset many senders and marketers. I’m sure many are relieved at this delay in enforcement.

This delay has no effect on the other major CASL provision with a July 1, 2017 deadline.
On July 1 a 3 year waiver on implied consent collected prior to CASL will end. What does that mean? Implied consent is just what it sounds like. Under certain conditions, senders can assume they have legal consent to mail the recipient. These conditions are spelled out in Section 10(9) of the law. Implied consent expires after 2 years. However, companies were granted a 3 year waiver on this provision for email addresses collected prior to July 1, 2014.
The waiver allowed senders to continue mailing addresses with implied consent even after the 2 year expiration. This was to allow companies time to convert implied consent into express consent as to not lose recipients. There are about 3 weeks left for senders to get explicit permission to continue mailing addresses collected prior to July 1, 2014.
Additionally, as of July 1, 2017 CASL requires a parliamentary committee to review the law and its operation over the last 3 years.

Many senders are thrilled with the indefinite suspension of the PRA. It was, I think, one of the parts of the law that worried people the most. Allowing any citizen to sue someone who sent them mail they thought violated CASL? That concept struck fear into the hearts of many a legitimate marketer. I was never quite so sure it was going to be as bad as some thought.
A few years ago I had the opportunity to sit in a conference session with an individual from the Canadian government. They explained that there were significant barriers to individuals suing senders. Plaintiffs must file in provincial courts, not local ones. Second, defendants couldn’t be under investigation by the CRTC and a PRA at the same time. The presenter implied that CRTC had priority over any joint defendant. Finally, the plaintiff must prove actual damages. This is difficult for defendants that use a freemail provider like Gmail. There aren’t really damages in that case.
The overall gist of the session was that PRA in Canada was not that simple. Individuals wanting to sue had some bigger hoops to jump through than just filing something in small claims court. Nevertheless, I’m sure that many senders are relieved to hear the PRA is indefinitely suspended.

Women. Technology. Moving Forward.

A little over a year ago, Kristin Bond posted an article (reprinted here) looking at the diversity of speakers at marketing conferences. As with many articles pointing out gender issues in technology there was quite a bit of discussion about it on a related mailing list. Some of the comments were supportive and open to the idea that gender diversity is an overall good. Some of the comments, while well meaning, indicated the commenters didn’t understand some of the more systemic issues that result in conferences with speaker lists that consist primarily of white men.
Kristin, I, Jen Capstraw and April Mullen started talking privately about the issue. What I discovered during those conversations is that I wasn’t alone in how I felt about some spaces. Being a woman in tech I expect to feel left out in many places. When I go to a conference, or I participate in an online space or I meet up with colleagues in social situations, I expect that someone will say something sexist. As a woman I regularly feel like an outsider. What I didn’t realize is other women in those same spaces felt the same way. By not saying something I was missing an opportunity to find a supportive atmosphere with other women who also thought spaces were unfriendly or toxic to women.
But we didn’t just complain; we decided to take action. What would happen if we created a space to help conferences find women speakers? What would happen if we set up a framework for women to find mentors? What did we have to lose by trying? Thus, Women of Email™ was formed.

Disappearing domains

On May 31, British broadband provider EE discontinued service for a number of email domains: Orange.net, Orangehome.co.uk, Wanadoo.co.uk, Freeserve.co.uk, Fsbusiness.co.uk, Fslife.co.uk, Fsmail.net, Fsworld.co.uk, and Fsnet.co.uk.
These domains were acquired by EE as part of multiple mergers and acquisitions. On their help page, EE explains that the proliferation of free email services with advanced functionality has led to a decrease in email usage at these domains.
Yesterday, Terra.co.br announced they were discontinuing email to a number of their free domains as of June 30, 2017: terra.com, terra.com.ar, mi.terra.cl, terra.com.co, terra.com.mx, terra.com.pe, terra.com.ve, and terra.com.ec.

I’m not surprised to see these domains going away and I think we’ll see more of it going forward. The reasons are pretty simple. Mail is not an easy service to run. Mail doesn’t bring in a lot of money. Dedicated mailbox providers do a great job and the addresses from them are portable.

Appending in a nutshell

A few months ago a colleague sent me, and every other person on his overly large LinkedIn list, an email looking for some help hiring. It starts off with “Greetings LinkedI Connections” and ends with… an unsubscribe link.

Purchased lists aren't always purchased

Spamhaus has listed a number of domains belonging to French politicians recently. In their blog post about it, they mention that the listings are directly related to address lists provided to candidates by the French government.

Are they using DKIM?

It’s easy to tell if a domain is using SPF – look up the TXT record for the domain and see if any of them begin with “v=spf1”. If one does, they’re using SPF. If none do, they’re not. (If more than one does? They’re publishing invalid SPF.)
AOL are publishing SPF. Geocities aren’t.
For DKIM it’s harder, as a DKIM key isn’t published at a well-known place in DNS. Instead, each signed email includes a “selector” and you look up a record by combining that selector with the fixed string “._domainkey.” and the domain.
If you have DKIM-signed mail from them then you can find the selector (s=) in the DKIM-Signature header and look up the key. For example, Amazon are using a selector of “taugkdi5ljtmsua4uibbmo5mda3r2q3v”, so I can look up TXT records for “taugkdi5ljtmsua4uibbmo5mda3r2q3v._domainkey.amazon.com“, see that there’s a TXT record returned and know there’s a DKIM key.
That’s a particularly obscure selector, probably one they’re using to track DKIM lookups to the user the mail was sent to, but even if a company is using a selector like “jun2016” you’re unlikely to be able to guess it.
But there’s a detail in the DNS spec that says that if a hostname exists, meaning it’s in DNS, then all the hostnames “above” it in the DNS tree also exist (even if there are no DNS records for them). So if anything,_domainkey.example.com exists in DNS, so does _domainkey.example.com. And, conversely, if _domainkey.example.com doesn’t exist, no subdomain of it exists either.
What does it mean for a hostname to exist in DNS? That’s defined by the two most common responses you get to a DNS query.
One is “NOERROR” – it means that the hostname you asked about exists, even if there are no resource records returned for the particular record type you asked about.
The other is “NXDOMAIN” – it means that the hostname you asked about doesn’t exist, for any record type.
So if you look up _domainkey.aol.com you’ll see a “NOERROR” response, and know that AOL have published DKIM public keys and so are probably using DKIM.
(This is where Steve tries to find a domain that isn’t publishing DKIM keys … Ah! Al’s blog!)
If you look up _domainkey.spamresource.com you’ll see an “NXDOMAIN” response, so you know Al isn’t publishing any DKIM public keys, so isn’t sending any DKIM signed mail using that domain.
This isn’t 100% reliable, unfortunately. Some nameservers will (wrongly) return an NXDOMAIN even if there are subdomains, so you might sometimes get an NXDOMAIN even for a domain that is publishing DKIM. shrug
Sometimes you’ll see an actual TXT record in response – e.g. Yahoo or EBay – that’s detritus left over from the days of DomainKeys, a DomainKeys policy record, and it means nothing today.

Random thoughts on spammers

I recently received a 419 spam that had a message at the top of the email.

Yup, a 419 spammer is trying to convince me there are millions of dollars waiting for me, but he won’t pay his software vendor 29.99 to comply with a license.
This is only the most recent in a long line of examples of spammers being cheap and attempting to steal services.
Back when I was working abuse almost every ISP had a story about a spammer who refused to pay their bill. Or spammers who were so high maintenance they cost the company money.
The company I worked for had a spammer that was on our system for far too long. Eventually they were cut off for non-payment and their hardware was confiscated. Still, the spammer came in and managed to remove the hardware before the building guards were alerted. It was disappointing, but at least they weren’t spamming off our network any longer.
Even now, ESPs share stories of customers who come in, spam and never pay their bill. Works for the spammer, they can get a few weeks of spamming in without having to pay for the service. They spew their stuff and leave a giant mess for the ESP to clean up. Next week, they’re on to the next ESP.
The real problem with this is that with enough ESPs and enough sends you can clean a list. This list can then be sold, or moved to a credible ESP without any of the tell tale signs of a purchased list. It’s so common it even has a name: waterfalling. It’s profitable, though, and there are enough small ESPs out there with little compliance experience that it can work.
I regularly get questions from folks who’ve worked themselves into a hole about swapping IPs or domains in order to get out of the hole. My answer is always the same. Changing identity might work in the short term, but it won’t work longer term. I also tell them that spammers have been trying to avoid filters for a lot longer than they have. Spammers are good at it, and still get caught in filters. Better to spend time trying to fix the underlying problem – typically users aren’t engaged with your mail – then trying to obfuscate who is sending the message to avoid filters.
Focus on sending good email that users want, rather than trying to avoid filters. That’s the key to getting into the inbox.

Protocol-relative URLs in email

When you link to an external resource – an image, a javascript file, some css style – from a web page you do so with a URL, usually something like “https://example.com/blahblah.css” or “http://example.com/blahblah.css”.
The world is beginning to go all https, all the time, but until recently good practice was to make a web page available via both http and https.
The problem is that if you try and load a resource from an http URL from a page that was loaded via https it’ll complain about it, and not load the resource.
And if your users web browser is loading the http version of your page because it’s Internet Explorer 6 and doesn’t speak modern SSL then it’ll be unable to load anything over https, including any of your resources.
So whether you choose https or http protocol for loading your page resources it’s going to break for someone.
One common trick to avoid the problem is to use a protocol-relative URL. That looks like “//example.com/blahblah.css”, and it’ll load the resource over the same protocol that the page was loaded over.
While we can safely use “https://…” everywhere now, “//…” URLs are still a common idiom for things like loading things like CSS libraries from public content-distribution networks as well as your own resources.
I was reading long-but-excellent writeup about Stack Overflow’s migration to TLS (hey, I read this sort of stuff for fun) and they point out something I hadn’t considered – mail clients don’t really have any sensible way to use protocol-relative URLs. The mail client loaded the “page” from a mailbox and so has no base document protocol to work from (even if there’s a <base> element in the content it’s not likely to affect a mail client). If the user is reading it via webmail or a mail client that’s using an embedded web browser to render HTML it might work, sometimes, but it’s not going to reliably load that resource in general.
So, if you’re copy-pasting content from your web collateral to reuse in an email, make sure you’re not loading anything external – including images – via a “//…” style URL. Rewrite them to use “https://…”.

ARC: Authenticated Received Chain

On Friday I talked a little about DMARC being a negative assertion rather than an authentication method, and also about how and when it could be deployed without causing problems. Today, how DMARC went wrong and a partial fix for it that is coming down the standards pipeline.
What breaks?

DMARC (with p=reject) risks causing problems any time mail with the protected domain in the From: field is either sent from a mailserver that is not under the control of the protected domain, or forwarded by a mailserver not under the control of the protected domain (and modified, however trivially, as it’s forwarded). “Problems” meaning the email is silently discarded.
This table summarizes some of the mail forwarding situations and what they break – but only from the original sender’s perspective. (If forwarding mail from a users mailbox on provider A to their mailbox on provider-Y breaks because of a DMARC policy on provider-A that’s the user’s problem, or maybe provider-A or provider-Y, but not the original sender’s.)

The philosophy of DMARC

We know that legitimate email sent with valid SPF and a DKIM signature often breaks in transit.
SPF will fail any time mail is forwarded – via a mailing list, a forwarding service used by the recipient, or just ad-hoc forwarding.
DKIM will fail any time the message is modified in transit. That can be obviously visible changes, such as a mailing list tagging a subject header or adding a footer to the body. It can also be less obvious changes, such as intermediate MTAs wrapping lines that are too long, reencoding content or repackaging the message altogether – perhaps when delivering from a mailserver that is 8BITMIME compliant to one that isn’t.

(This image has absolutely nothing to do with email authentication, but searching for stock photography about email or authentication or chains or, well, pretty much anything like that leads to horribly depressing corporate imagery. So, no. Have something colourful and optimistic instead.)
As SPF and DKIM are typically used, none of this is much of a problem. A message being authenticated provides a little extra information to the receiving mailserver, and the domain attached to the authentication can be used to look up a senders reputation, giving a potential boost to the chances of the mail being sent to the inbox. If the authentication is broken, though, the mail will still be judged on it’s merits – is it coming from an IP address that’s a source of good mail, does the content look legitimate, and all the other things a spam filter looks at.
That authentication is a (potentially big) positive signal, but lack of authentication isn’t really any signal at all is why SPF and DKIM being fragile wasn’t an issue. SPF and DKIM are positive assertions – “IF this mail IS authenticated THEN IT IS from me”.
That changed when DMARC became popular, though.
DMARC allows the owner of a domain to say “We send no mail that is not authenticated, and we promise that none of that authentication will be broken in transit”. DMARC is a negative assertion – “IF this mail IS NOT authenticated THEN it IS NOT from me”. It converts the absence of a positive assertion into a negative assertion.
This isn’t the first attempt to layer a “we authenticate everything” negative assertion on top of fragile email authentication. SPF did it, with the -all flag (which is universally ignored, leaving SPF purely as a positive assertion). DomainKeys did it, with DomainKeys policy records (which you occasionally still see published, but were never really used to reject mail). DKIM did it with ADSP – which didn’t see much use either.
The reason none of them were used much is because even when senders were telling the truth about “we send no email that is not authenticated” they were always lying, to varying degrees, about “none of the authentication will be broken in transit”.
If your domain that is solely used for bulk email. If it’s never for used mail sent by human beings, not even customer support employees. If it’s a newly created domain with no legacy usage that only sends email from a very tightly controlled infrastructure. If you only send email that’s been created via a well implemented message composition pipeline that ensures the content of the is not just RFC compliant but also “well formed”, with short lines, simple widely implemented encoding, vanilla mime structure and so on. And it’s sent out via conservatively configured smarthosts that deliver directly to the end recipients MX. And if you know that the demographics of your recipients are such that the minority that are forwarding that mail elsewhere (e.g. from their Yahoo account to their Google account or via an alumni mail alias) is a small enough group that you don’t care about them…
If all of those things are true, then your domain is going to be able to deploy DMARC pretty easily and safely. If not, though, how can you tell?
That’s the place where DMARC improves over it’s predecessors. It allows you not only to publish a DMARC policy record in test mode, so it’s not actually used to filter your mail (well, mostly, but that’s a longer story) but also to ask recipients to notify you of mail that seems to be from you but which isn’t authenticated.
You can publish a “p=none” DMARC record with notification addresses in it and wait and see what happens. You’ll get notification of mail that has your domain in the From: field but which isn’t authenticated.
As a first round of action that lets you see where you’re sending email from that you didn’t know about. Sysadmin notification email. That marketing splinter group in Sasketchwan. The outsourced survey company.
Once you’ve cleaned all that up, and made sure everyone is authenticating their mail then you can look at what’s left. The next step is likely to be mistakes you’re making in authentication or message composition that’s causing some of your mail – typically depending on content, and source and recipient domain – to become unauthenticated. Clean that up, make sure all your message composition is squeaky clean, make sure employees aren’t sending mail using that domain in ways you don’t authorize (interacting with mailing list, for example).
By that point you’ll have reduced the torrent of reports you’re getting to two types. One is mail that you send that has it’s authentication broken in transit through some process you have no control over. The other type is mail that has your domain in the “From” field but which you didn’t send. Some of that may be legitimate use of your domain by your employees, such as forward-to-a-friend services, signing up for document delivery via email, third-party notification services. By deploying DMARC you are declaring all that sort of usage to be illegitimate, and you’ll need to get all your employees to stop doing it (or, at least, know that it’s going to stop working). The rest of it is likely a mix of spam and phishing mail. The spam, that’s just using your domain in random from addresses, you probably don’t care about. The phishing you do.
You’ve finally cleaned up your mail infrastructure and policies enough to gather the data you need. How much of my legitimate email will have it’s authentication broken (and hence be silently thrown away by DMARC)? And how much hostile phishing mail is targeting my users (and using the exact domain you are)?
Then you have the information you need to make an informed decision as to how badly deploying DMARC will break your legitimate use of email (after you’ve done everything you can to minimize that) and some idea of whether it will provide you any benefit, at least in the shorter term.
That testing phase, where senders can use other peoples mail infrastructure to investigate their sending practices, gradually fix any problems and finally gather some metrics is what made gave the developers of the DMARC spec confidence that it wouldn’t break things, and made it much more deployable than previous approaches to negative assertion.
On Monday, how all that optimistic reasoning went to hell, what it broke and how we’re trying to fix it.