TidBITS Policy on Challenge-Response
An anti-spam technique called challenge-response is becoming increasingly popular these days. Simply described, challenge-response compares the sender of each incoming message against the contents of your email address book (or a similar list generated in another way, such as by extracting the senders of every piece of your stored mail). If the sender of the incoming message appears in your address book, the message comes through as you’d expect. However, if that incoming message is from an unknown address – either someone from whom you’ve never received email or an acquaintance using a new address – the challenge-response system sends an email reply to the sender, asking her to click a link, reply to the message, or in some way indicate that her original message came from a real person. Once verification has happened, the message is delivered appropriately, as are all subsequent messages from that sender.
Challenges to Challenge-Response — Challenge-response systems are fairly effective, since most people receive mail from roughly the same subset of senders, and the effort to any individual sender is relatively low. These systems suffer from a number of important problems, though.
Spammers often forge headers so the spam you receive appears to come from other email addresses at the same domain, or even from your own email address. It’s not uncommon for me to receive spam "from" myself, or "from" another member of the TidBITS staff. In smaller organizations, it’s likely that most people with email addresses at that domain would be in each other’s address books, so spam "from" those addresses would bypass a challenge-response system.
Challenge-response puts an additional burden on senders, which is why it’s effective against spam. However, it also tends to engender ill will among normal people who feel as though you’re asking them to jump through hoops (which you are). It’s in your interest to make the process as easy as possible for legitimate senders.
There are many legitimate reasons why you might receive email that’s sent automatically, such as an order receipt from an online vendor or a mailing list subscription confirmation request. You’re unlikely to have such email addresses in your address book, so those sorts of messages can be stopped erroneously. Most of the time, no person would even see the challenge since those systems run on auto-pilot. Ironically, this could even create mail loops between systems as your challenge is answered not with a response, but with a competing challenge.
As a special case to the above, consider mailing lists to which you subscribe. Depending on how the challenge-response system is set up, you could end up sending challenges to everyone who posts a note to a discussion list (this happened on TidBITS Talk recently, annoying a number of people). Or, in the more generic case of TidBITS, we could end up receiving hundreds or even thousands of challenges from subscribers who turned on a challenge-response system but didn’t have <[email protected]> in their address books.
Ever More Challenges — There are certainly technical solutions that could ameliorate each of these problems (such as a quarantine area that users can check for legitimate mail that’s been held but hasn’t been verified by the sender, and special cases for mail from lists), but with different systems appearing from a variety of companies, such as SpamArrest and Mailblocks, there’s no telling which features will be commonly available, or how they will require senders to respond.
Challenge-response technology is about to become significantly more widespread, though, with EarthLink about to test such a system for its 5 million customers. EarthLink is currently the third-largest ISP in the United States, and it serves over 2,000 TidBITS subscribers (second only to AOL, and well ahead of Mac.com).
Our Challenge — Although we’re always in favor of individuals and ISPs working to control the pestilence that is spam (by the time you read this, I’ll have received more than 21,000 spam messages so far in 2003), we’ve also spoken out in the past against approaches like arbitrary content filtering that actually increase the damage spam causes to the global email system.
We don’t view challenge-response as being nearly as concerning as arbitrary content filters, but it does raise problems for us. We send email to nearly 50,000 people each week by the time you take all of our versions and translations into account, and dealing with hundreds of individual challenges each week would utterly overwhelm us. We don’t have the staff resources to do that and keep everything else running. We’re not unusual in this regard; most mailing lists on the Internet will run into similar problems.
So consider this article a heads-up to anyone who is thinking about using a challenge-response system. Please be a good Internet citizen and make sure you add mailing list distribution addresses to your address book and work to avoid situations that will cause irritation for others in your particular parts of the Internet.
Closer to home, be warned that we will not answer any challenges generated in response to our mailing list postings. Thus, if you’re using a challenge-response system and not receiving TidBITS, you’ll need to figure that out on your own. Also, if you send us a personal note and we receive a challenge to our reply, we may or may not respond to it, depending on our workload at the time.
In short, do what you feel is necessary to control your spam problem, but remember that it’s your responsibility to make it possible for people to send you email that you request.