Recent Posts

New FBLs

There are two new FBLs in production. Synacor and Fastmail.fm. I’ll be updating the Wiki and FBL page today.

I had someone ask on Facebook about getting some documents off of Pacer. I thought the information may be of use to other people out there.
PACER (public access to court electronic records) provides access to public documents filed in the Federal court system. Each court has their own website, but there is one login and the search and document display are the same. Documents cost 8 cents a page, capped at $2.40 for a single document.
Access to PACER isn’t always immediate. When I signed up there was a 7 – 10 day delay as usernames and passwords were sent by mail. There does seem to be a way now to get a password faster, for those of you who want data NOW!
Once you’ve got a username and password now you’re in business and can start digging up all these documents.
The first step is determining which court website to check. Generally, I’m looking for details because I saw a news report that does mention what court the case was filed in. So I just plug the court name (Northern District of California) in a search window and go from there. PACER also provides the facility to look up where a case is on their website. This wasn’t an option when I signed up for PACER so I’ve never used it, but it is there.
The court websites are often not very flashy (Web 0.5!) but there will be a link to retrieve documents or view documents through PACER. This is the link that will take you to the login page. Put in your username and password and click go. If you’re not filing, you don’t need to bother with the checkbox for the Notice of Redaction Responsibility to get in, nor do you need to add a client code.
Once you’re logged in you’ll notice a blue bar across the top of the page. This is your (web 0.5!) navigation bar. Click on Query to bring up the case search window. If you have the actual case number, you can put that in the top box and hit search. Otherwise, you can enter in a party name. For my recent research, I just enter “holomaxx” in the box marked Last/Business Name and click Search. Being web 0.5! you have to actually click the button, pressing enter doesn’t work.
That will take you to the Select a Case window. In this case, Holomaxx is a safe search because it brings up exactly the two cases I’m interested in: Holomaxx v. Yahoo and Holomaxx v. Microsoft. Clicking on the case number brings up a window with some basic information (the judge, last filing date) and a number of links.
The link that will show you documents is, unsurprisingly, History/Documents. Click there, and click again on All events to bring up a list of documents filed with the court.
The first column is a clickable link that lets you look at the document. The second column is that date it was filed. The third column is the title of the document. Generally when I’m looking at a new case I grab something that looks like “complaint” or “motion” to orient myself.
When I’m looking at PACER I tend to download everything I look at on a case, just so I only have to pay for it once. I also make extensive use of tabs and new windows, so I don’t have to reload the case page.
Download names vary by the actual court. For instance, the Northern California court gives me all the documents with the same name: show-temp.pl. But other courts give names like 384972395.pdf. In either case, you’re going to want to rename the documents to something useful before you have a disk full of show-temp-*.pl files. In some cases, there are documents and exhibits in a single filing. You will be asked if you want to download everything as a .zip file. I suggest you do this.
For a while I was trying to name things intuitively but then gave up because it gets too confusing. My current organizational technique is to set up a directory with the case name HolomaxxvYahoo_4926 and HolomaxxvMS_4924. The numbers are the last 4 digits of the case number and are there to make it easier to file and sort documents.
If you download a zip file, it opens up a directory containing all the files. The courts name these pretty simply: documentnumber-main.pdf, documentnumber-1.pdf, documentnumber2.pdf. The document numbers correspond to the order the documents were filed with the court. Once the file is unzipped, I copy the files into the directory I’ve set up for that case.
Now that you have the documents somewhat organized, you can shut PACER down and go read at your leisure. If you spend more than $10.00 on documents in a quarter, then you will get a bill from the Federal court system. If you haven’t spent that much, the court doesn’t bother billing you that quarter.
Some state courts have similar systems, but not all of them do and you can’t use a PACER login to access them.
In the course of writing this, I discovered new documents filed in the Holomaxx case filed by the defenands. Tune in tomorrow. Same bat-time. Same bat-blog.

Who leaked my address, and when?

Providing tagged email addresses to vendors is fascinating, and at the same time disturbing. It lets me track what a particular email address is used for, but also to see where and when they’ve leaked to spammers.
I’d really like to know who leaked an email address, and when.
All my inbound mail is sorted into “spam” and “not-spam” by a combination of SpamAssassin, some static sieve rules and a learning spam filter in my mail client. That makes it fairly easy for me to look at my “recent spam”. That’s a huge amount of data, though, something like 40,000 pieces of spam a month.
Finding the needle of interesting data in that haystack is going to take some automation. As I’ve mentioned before you can do quite a lot of useful work with a mix of some little perl scripts and some commandline tools.
I’m interested in the first time a tagged address started receiving spam, so I start off with a perl script that will take a directory full of emails, one per file, find the ones that were sent to a tagged address and print out that address and the time I received the email. I can’t rely on the Date: header, as that’s under the control of the spammer, and often bogus. But I can rely on the timestamp my server adds when it receives the email – and it records that in the first Received: header in the message.

The weak link in security

Terry Zink posts about the biggest problem with security: human errors. Everyone who is looking at security needs to think about the human factor. And how people can deliberately or accidentally subvert security.

Holomaxx doubles down

Holomaxx has, as expected, filed a motion in opposition to the motion to dismiss filed by both Yahoo (opposition to Yahoo motion and Hotmail (opposition to Microsoft motion). To my mind they still don’t have much of an argument, but seem to believe that they can continue with this.
They are continuing to claim that Microsoft is scanning email before the email gets to Microsoft (or Yahoo) owned hardware.

Gmail shows authentication data to the recipient

Yesterday Gmail rolled out some changes to their interface. One of the changes is that they are now showing end users authentication results in the user screen.
It’s really the next step in email authentication, showing the results to the end user.
So how does Google do this? Google is checking both SPF and DKIM. If mail is authenticated and the authentication matches the from address then they display the email as:

If we click on “details” for that message, we find more specific information.
In this case the mail went through our outgoing mailserver to gmail.
Mailed-by indicates that the message passed SPF and that the IP address is a valid source of mail from wordtothewise.com.
Signed-by shows the domain in the DKIM d=. In this case, we signed with the subdomain dt.wordtothewise.com. That’s what happens when you sign using the domain in the From address (or a subdomain of it).
For a lot of bulk senders, though, their mail is signed using their ESP’s domain instead. In that case Gmail shows who signed the mail as well as the from address.

And when we click on “details” for that message we see:
This is an email from a sender using Madmimi as an ESP. Madmimi is handling both the SPF authentication and the DKIM authentication.
As an aside, this particular sender has a high enough reputation that Gmail is offering me an unsubscribe option in their interface.
Gmail is distinguishing between first party and third party signatures in authentication. If the mail is authenticated, but the authentication appears to be handled by a separate entity, then Gmail is alerting recipients to that fact.
What does this mean for bulk senders?
For senders that are signing with a domain that matches their From: domain, there is no change. Recipients will not see any mention of your ESP in the headers.
However, if you are using an ESP that is signing your mail with a domain they own, then your recipients will see that information displayed in the email interface. If you don’t want this to be displayed by Gmail, then you will need to move to first party signing. Talk to your ESP about this. If they’re unsure of how to manage it, you can point them to DKIM Core for an Email Service Provider.
Gmail blogpost about the changes
Gmail help page about authentication results

URL Shortening and Email

Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.
(9/18/17: Closing comments because this post attracts spam comments)

Bit.ly gets you Blocked

URL shorteners, like bit.ly, moby.to and tinyurl.com, do three things:

Well designed email program

I so often talk about the failures of various email marketing programs that it’s only fair I mention when someone gets it right.
We spent the past week with family on the east coast. Our flight back to the west coast was very, very early Sunday morning so I booked a night at the airport hotel. That way we could just stumble to the shuttle at some horrible hour and not worry about trying to coordinate drivers and cars and all that other stuff.
As we were headed to the airport, I pulled out my phone to confirm directions. I found a new message in my mailbox offering me the opportunity to check-in online. I decided to see how it worked.

The frequency conundrum

What is the perfect frequency to send mail? Is it daily, weekly, monthly, hourly, minutely (is that even a word?) or randomly? Any number of experts will give you a definitive answer to this question, but I don’t believe there is a single answer.
The frequency recipients will respond to depends on the type of mail, the recipient expectations, the sender and a host of other factors.
For one example look at the mail sent by social networks. Many people, myself included, will accept dozens of emails a day telling me someone wrote on my Facebook wall or retweeted something I said or wants to link to my network on LinkedIn. Another example is when I’m traveling or waiting to pick up someone who is, I am thrilled to receive multiple updates an hour from the airline.
This willingness to receive frequent commercial or bulk emails doesn’t necessarily translate to marketing emails. When Sur la Table started sending double digit amounts of email a week, I down-subscribed, and had they not let me pick an acceptable-to-me frequency I would have unsubscribed completely.
A lot of marketing experts insist that mailers don’t send frequently enough. That increasing frequency increases ROI. What a lot of people miss are all the caveats in the fine print. In their minds, increasing frequency goes hand in hand with increased segmentation, targeting and recipient specific emails.
The idea isn’t simply to mail the entire list more frequently but to mail those who are more open to increased frequency. This is an idea I wholeheartedly support.