Recent Posts

Dodgy PDF handling at Gmail

We sent out some W-9s this week. For non-Americans and those lucky enough not to have to deal with IRS paperwork those are tax forms.
They’re simple single page forms with the company name, address and tax ID numbers on them. Because this is the 21st Century we don’t fill them in with typewriters and snail mail them out, we fill in a form online at the IRS website which gives us PDFs to download that we then send out via email.

We started to get replies from people we’d sent them to that we hadn’t included the tax ID number. Which was odd, because it was definitely there in the PDFs we’d sent.
The reports of missing numbers came from Google Apps users, so we sent a copy to one of our Gmail addresses to see. Sure enough, when you click on the attachment it’s mostly there, but some of the digits of the tax ID number are missing.

And all the spaces have been stripped from our address.

The rest of the form looked fine, but the information we’d entered was scrambled. Downloading the PDF from Gmail and displaying it – everything is there, and in the right place.
Weird. After a brief “Are gmail hiding things that look like social security numbers?” detour I realized that the IRS website was probably generating the customized forms using PDF annotations.
PDF is a very powerful, but very complex, file format. It’s not just an image, it’s a combination of different elements – images, lines, vector artwork, text, interactive forms, all sorts of things – bundled together into a single file. And you can add elements to an existing PDF file to, for example, overlay text on to it. These “annotations” are a common way to fill in a PDF form, by adding text in the right place over the top of an existing template PDF.
I cracked the PDF open with some forensics tools and sure enough, the IRS had generated the PDF form using annotations.

Is purging always effective?

Data never dies

Anyone know why…

Countless questions about email troubleshooting start with “does anyone know why.” Unfortunately, most of these questions don’t contain enough detail to get a useful answer.

In the case of email, even the smallest redactions, like the IP address and the domain in question, can make it difficult for anyone to provide help. Details matter.
Every detail matters, sending IP and domain are just the beginning. Who’s doing the sending? What is their authentication setup? What IP are they using? How were the addresses collected? What is their frequency? What MTA is used? Are they linking to outside sites? Are they linking to outside services? Where are images hosted? Is the mail going to the bulk folder or being rejected? What ISPs or filters are involved?
The relevant questions go on and on and on.
We send fairly detailed question lists to clients. I regularly look at them to try and make them shorter. But the reality is these are questions that are relevant. Without enough information we simply cannot troubleshoot delivery problems.

How to hire an affiliate

Yesterday I talked about all the reasons that using affiliate email can hurt overall delivery. In some cases, though, marketing departments and the savvy email marketer don’t have a choice in the matter. Someone in management makes a decision and employees are expected to implement it.
If you’re stuck in a place where you have to hire an affiliate, how can you protect the opt-in marketing program you’ve so painstakingly built? Nothing is foolproof, but there are some ways you can screen affiliates.

Affiliate marketing overview

Most retailers have realized that sending unsolicited email is bad for their overall deliverability. Still, the idea they can send mail to people who never heard of them is seductive.
Enter affiliate email. That magical place where companies hire an agency, or a contractor, or some other third party to send email advertising their new product. Their mail and company reputation is protected because they aren’t sending the messages. Even better, affiliates assure their customers that the mail is opt-in. I’m sure some of them even believe it.
The reality is a little different from what affiliates and their customers want to believe.

SNDS issues and new Gmail

A bunch of folks reported problems with Microsoft’s SNDS page earlier today. This afternoon, our friendly Microsoft rep told the mailop mailing list that it should be fixed. If you see problems again, you can report it to mailop or your ESP and the message will get shared to the folks who can fix it.
The other big thing that happened today was Gmail rolled out their new inbox layout.
It’s… nice. I’ll be honest, I am not a big gmail user and have never been a huge fan. I got my first account way-back-during-the-beta. I used it to handle some of my mailing list mail. I could never work out how to get it to stop breaking threads by deciding to put some mail into the junk folder. I just gave up and went back to my shell with procmail (now sieve) scripts. I still have a couple lists routed to my gmail account, and the filtering is much improved – I can at least tell it to never bulk folder certain email.
The feature I’m really interested in is the confidential, expiring email. I’m interested in how that’s going to work with non-Gmail accounts. Within Gmail makes perfect sense, but I don’t think Gmail can control mail once it’s off their system.

My best guess is that Gmail will end up sending some type of secure link to recipients using non-Gmail mail servers. The message itself will stay inside Google and recipients will only be able to view mail through the web. That’s how the vast majority of secure mail systems work.
If anyone has the secure message already, feel free to send me a secure message. I’ll report back as to how it works.

What kind of mail do filters target?

All to often we think of filters as a linear scale. There’s blocking on one end, and there’s an inbox on the other. Every email falls somewhere on that line.
Makes sense, right? Bad mail is blocked, good mail goes to the inbox. The bulk folder exists for mail that’s not bad enough to block, but isn’t good enough to go to the inbox.
Once we get to that model, we can think of filters as just different tolerances for what is bad and good. Using the same model, we can see aggressive filters block more mail and send more mail to bulk, while letting less into the inbox. There are also permissive filters that block very little mail and send most mail to the inbox.
That’s a somewhat useful model, but it doesn’t really capture the full complexity of filters. There isn’t just good mail and bad mail. Mail isn’t simply solicited or unsolicited. Filters take into account any number of factors before deciding what to do with mail.

No, I won't rate you!

Brick and mortar stores have tried to use feedback as a means of driving customer engagement for a while. Anyone who’s shopped at a big chain here in the US knows what I mean. You buy a pack of gum and end up with a 2 foot long receipt. At the bottom of the receipt there is a URL and bar code. The cashier circles the bar code and cheerfully tells you to go online and tell corporate about their service.
If you go to the website, they ask you for specific specific purchase information (time, date, store number, amount, cashier) and ask a bunch of questions about the store. Then, they offer you a chance to win something (gift card, something) if you’ll provide them with your personal information.
Note: This particular form does not allow you to continue at all unless you’ve filled in the information request. Even if you check “prefer not to answer” the page throws up an error message and tells you to provide a valid phone number.
More recently email marketers have jumped on the asking for feedback bandwagon. Over the last few weeks multiple companies have sent me emails asking how my visit to their website was. It… was a website? I mean I went to your website and checked my credit card bill, it told me how much I owed. Your tech support told me they couldn’t fix my problem over chat, I’d have to take my laptop in for repairs. My package arrived and if it didn’t you can be sure I would have reached out to you.
And it’s not just online services that do this. Hotels send followup surveys, which if you’re a frequent traveler turns into a full time job. Yes, I visited your hotel it’s very nice. If I’m in town and that’s where the conference I’m attending is hosted, I’ll probably be back.
I get it, the more chances you provide for people to interact with your brand the more engaged they are and the more likely they are to purchase from you. But a simple search of my mailbox shows over a dozen messages from companies over the last few weeks, all of them asking me for feedback on their services. I’d like a little less email, please. The bank, the mortgage company, the credit card company, the food delivery service I used, the clothing website, the travel website, the ride share service, the hotel… the list goes on and on.
If only a few companies did this, it wouldn’t be such a big deal. But as more and more companies adopt the triggered email followup (and the followup reminder and the final reminder and the final final reminder), recipients are going to get tired of the messages. Some of the requests don’t even have opt-outs, although the majority of the ones in my mailbox do.
I get that each company is only responsible for the mail they, in particular, are sending. But the user has a different frame of reference, and maybe it’s time to consider that using surveys and triggered emails to drive engagement may not be a long term sustainable business model. The rest of the companies out there using the same strategy are going to ruin it for everyone.

Laposte rejections

Update: The issue seems to have been resolved and Laposte say they’re no longer sending the 519 responses as of April 25th 2018.
Laposte.net are having a bad couple of weeks. There’ve been reports from customers of their IMAP service being unusable, with attempts to move or delete messages timing out and expected emails simply not arriving.
Several delivery friends have mentioned that they’re rejecting mail with errors that look like this: