A brief history of TXT Records

When the Domain Name System was designed thirty years ago the concept behind it was pretty simple. It’s mostly just a distributed database that lets you map hostname / query-type pairs to values.
If you want to know the IP address of cnn.com, you look up {cnn.com, A} and get back a couple of IP addresses. If you want to know where to send mail for aol.com users, you look up {aol.com, MX} and you get a set of four hostname / preference pairs back. If you want to know the hostname for the IP address 206.190.36.45 you look up {45.36.190.206.in-addr.arpa, PTR} and get a hostname back.
There’s a well-defined meaning to each of those query types  – A is for IP addresses, MX is for mailservers, PTR is for hostnames – and that was always the intent for how DNS should work.
When DNS was first standardized, though, there was one query type that didn’t really have any semantic meaning:

TXT RRs are used to hold descriptive text. The semantics of the text depends on the domain where it is found.

TXT records didn’t really have a use. Some domain owners used them to provide their contact information for the domain, and there were some funny messages scattered around, but that was about it.
Around the year 2000 email people started thinking about publishing data in the DNS. Email people are not the same as DNS people. To a DNS person the obviously correct way to do that would be to define a new query type, persuade people to agree on that definition and then make it a standard by publishing an RFC.
But email people have a long history of piling standards on top of other standards and deploying ad-hoc approaches (e.g. X-Headers, Uuencoding, PGP encapsulation and ascii armoring) without more than the bare minimum of agreement on the right way to do things. That approach helps with fast-yet-gradual deployment of new solutions, but also leads to a certain impatience with the bureaucracy of an actual standards development process.
“Just stick it in a TXT record!” became the rallying cry.
SPF was the first widely deployed protocol to do that. There was an attempt to migrate SPF from using TXT records to using a dedicated SPF record type, but given the deployed base of users already using TXT records it was doomed to failure.
Whenever someone is rolling out a new protocol that needs a domain owner to publish something – whether it’s something fairly standardized or pretty much ad-hoc – they tend to reach for a TXT record.
Some more things you need to know about TXT records on Monday.