Thoughtful, detailed coverage of the Mac, iPhone, and iPad, plus the TidBITS Content Network for Apple consultants.

Introducing the TidBITS Commenting System

After weeks of discussion, design, and development, Glenn Fleishman and Jeff Carlson and I are pleased to unveil our new TidBITS Commenting System (TCS). For many years, we've had the TidBITS Talk mailing list for discussions of our articles. We've tried to integrate it more tightly with our articles, but it's difficult to connect email and a Web-based content management system in a reliable and coherent fashion that doesn't require constant intervention.

In the meantime, nearly every other publication and blog has added a commenting feature. But commenting systems have suffered their own growing pains, caused largely by the need to deal with spambots and trolls, and it's become sufficiently onerous to use many commenting systems that we no longer bother. (Trolls are people who specifically pick fights to make others respond; they often have no stake in the opinion they're stating.)

So we thought about what we wanted in a commenting system, and after many discussions and design tests, we've come up with one that we quite like. It has worked well in testing and in a quiet rollout last week, and we'll see how it scales as more people start using it.

On the TidBITS home page and each of our section pages, each article summary now displays the number of comments in the metadata line beneath it, and each full article shows the number of comments at the top. In either case, clicking the word "Comments" takes you to the comments section at the bottom of the article.

Lightweight, Yet Secure -- Our first goal with the TCS was that it be easy to use. Twitter has been successful largely because it's so easy to post tweets. You don't have to worry about writing much or formatting your text, and your tweet appears immediately. Since we've put untold hours of work into moderating our TidBITS Talk discussion list, we wanted a system where posts could appear immediately, without any intervention on our part. Moderation was good for stopping spam and keeping discussions on track, but forcing people to wait hours or days for their posts to appear isn't ideal.

The problem with immediate posting, of course, is comment spam. Spammers have figured out how to write comment bots that do nothing but fill up commenting systems with link-filled posts as a way of attracting a few clicks and gaming search engine rankings. Most comment systems we saw dealt with this by either requiring users to type a CAPTCHA for each post or to set up an account before posting. CAPTCHAs have their place, as do accounts, but for a quick comment, both were more than we wanted to use for the TCS. (We use CAPTCHAs if you want to post a tip or send email to a TidBITS author.)

Instead, the first time you comment on an article by clicking the "Add a comment" link, it asks you for your name and your email address in addition to your comment. The TCS then generates an email message to you, asking you to click a link in the message to post your comment on the site. Clicking the link tells us that the email address you entered is valid and that it's your address. That gives us enough confidence to allow your post to appear on the site. It won't stop trolls, of course, but I'll go into more detail on how we can deal with that problem shortly.

Confirming via email for each comment would be tedious, so the TCS also sets a cookie in your Web browser when you click that confirmation link. Thanks to that cookie, the TCS then knows who you are and doesn't require confirmation of any subsequent posts, as long as you use the same browser and don't delete your cookies. Posting comments becomes even easier, too, because identified users don't have to enter their names and email addresses in the Post Comment dialog.

If you post a comment from another computer or another browser, you must confirm via email again, but as long as you use the same email address, the TCS will know that you're the same person in multiple browsers. We have to store your email address in our database for this to work, but your address isn't displayed in any way, so there's no worry about it being harvested by spam trawlers. For information about how we treat private information, see our privacy policy.

Maintaining this lightweight identity is important because we give identified users the ability to edit their own posts. So, let's say you submit a comment and notice as soon as it's posted that it has a typo in it. Click the Edit button and you can make a change instantly and repost your edited comment. The ability to edit your own posts lasts for 7 days, after which the TCS locks the posts down.

There's one other neat little trick that Glenn added toward the end. What if you enter your name incorrectly, or more or less formally than feels appropriate once you see it on the page? Just click your name (a subtle yellow highlight appears when you mouse over it) and enter a new one. Simple and unobtrusive.

Threaded Discussions -- Another of our goals with the TCS was that it be threaded, but again in a lightweight way. Some commenting systems are entirely flat, which is fine when there are relatively few comments per article, and where the comments seldom refer to each other. But once there are a lot of comments and they start referencing one another (often with manual email-style quoting), it gets out of hand. Threaded systems are less common, probably because they're harder to write, and they often suffer from design problems that make it difficult to group related messages in your head while reading.

We addressed these concerns in the TCS in several ways. To add a top-level comment, click the "Add a comment" link at the bottom of any article. But if you want to reply to a particular comment, click its Reply button instead, after which your comment will be indented under its parent and any previous replies; it also has a different background color.

Jeff's design for this is intentionally minimal. He created two light background colors that match with the rest of the site color scheme, and each level of threading uses a different color. So the top-level comments always use the light purple, the second-level replies use a light blue, third-level replies go back to the light purple, and so on. This has the effect of subtly grouping replies to a particular comment and making it clear when levels are changing without hitting you over the head with an outline.

Although it's not specifically related to the threading, we also implemented a 1,000-character limit on comments. The idea is not so much to prevent people from writing a lot (though it will have that effect), but to encourage people to keep comments concise and to the point. Instead of one long comment that addresses an issue in the article, and responds to several other comments, we'd prefer that you post a short comment for each topic, replying directly to other comments as necessary to maintain the structure.

Right now, there's no way to collapse replies into their parent, but if threads start to grow long enough to need a way to condense them, we'll look into adding it.

Preventing Abuse -- The final design goal with the TCS was to keep it easy to use without opening ourselves up to comment spam or trolls. Our abuse-prevention features include:

  • The email-based verification for the first time a person posts, which should stop most spambots.
  • An AJAX-intensive interface that will hopefully resist being controlled through automation.
  • Admin-level controls that allow us to edit a comment, delete a comment, prevent a user from posting new comments, and block a user (deleting that user's posts) entirely. Although we don't plan to use a heavy hand, we won't put up with anything that's abusive.
  • Email notification for several staff members of the actual comments, so we can see what's being posted quickly and act on it if need be.
  • Automatic disabling of comments on articles older than 30 days, since few people are likely to see comments on old articles, and it lessens the risk to our massive article database if something does break through our safeguards.
  • A master switch to turn comments off for the entire site. This is our "Oh no!" fallback if we see abuse happening and need time to figure out the best way to stop it technically.

Honestly, though, I believe the main thing that will help keep our comments clean is the TidBITS audience. I've been distressed by the behavior I see (and hear my colleagues complain about) at other sites, but we just don't have those problems with TidBITS Talk and direct email responses from TidBITS readers. Universally, TidBITS readers are polite and thoughtful, even when they don't agree with something we've written.

My hope is, therefore, that by having useful, considered comments by long-time TidBITS readers setting the tone, people who are new to TidBITS will respond in kind. And if we see comments that are abusive or wildly off topic or just plain icky, we'll delete them as soon as we notice.

Whither TidBITS Talk? The release of the TCS raises the question of what will be happening with our TidBITS Talk mailing list. For the moment, all will continue as it has, although we're hoping that the combination of some mail server changes and traffic diverted to the TCS will allow us to stop moderating posts from subscribed users. We'd still look at anything from someone who wasn't on the list, mostly to prevent spam.

We anticipate that TidBITS Talk will remain useful as a place to have more-involved discussions about topics that are perhaps only peripherally related to an article, or that are about a topic about which we haven't yet written an article. As such, we don't have any plans to change it in the near future, and we'll play it by ear going forward.

Future Plans -- We've already started to keep a list of things we want to consider adding to the TCS in the future, such as including comment links in our email issues, optionally informing authors of comments on their articles, allowing user-level email notifications of new comments, and some sort of integration with Twitter. We're not committing to any of these ideas, and we're open to others, so if you have a particular suggestion, leave a comment!


Make friends and influence people by sponsoring TidBITS!
Put your company and products in front of tens of thousands of
savvy, committed Apple users who actually buy stuff.
More information: <http://tidbits.com/advertising.html>

Comments about Introducing the TidBITS Commenting System
(Comments are closed.)

Adam Engst  An apple icon for a TidBITS Staffer 2009-07-03 07:53
One thing we found interesting was that during the quiet rollout, a couple of articles generated a number of comments that helped us refine the articles in useful ways. We often tweak articles in small ways in the days before publishing them in email. After the email issue goes out, we almost never make any textual changes to an article, so comments will be a good way to continue adding information on the topic.
Mike Rose  2009-07-04 17:52
Looks pretty snazzy so far. Congratulations on the rollout!
John Nemo  2009-07-03 07:54
Thank you for the new comment system.
Devesh   2009-07-03 09:10
Sounds neat - hope it works out as planned. Care to share some of the technology used to implement this? Is this all custom programing, or something commercially available, or perhaps a cocktail of both?
Glenn Fleishman  2009-07-03 10:02
It's all developed in house by yours truly, and it's so integrated into our homebrew CMS, I can't see how to release it separately -- yet!
AHoffman   An apple icon for a TidBITS Supporter 2009-07-03 13:13
Isi it worth patenting some of these ideas?
Glenn Fleishman  2009-07-03 13:35
I hope not! It's too late, because we deployed before filing. Anyway, I wouldn't want to get into the ugliness of looking for prior art. If we're lucky we a) developed something de novo and b) are far below the radar.
Jim Rea  2009-07-03 09:29
This sounds like a great system - congratulations. I think the e-mail confirmation before allowing the first post is especially clever. And it's also very cool that I can edit my comments for seven days, which I just did!
Franois Granger  2009-07-03 09:36
Nice move, carefully planed and well executed.

Any plan to add the corresponding feeds for all comments of the site, all comments for an article?
Glenn Fleishman  2009-07-03 10:03
Yes, we're thinking about that. It wouldn't be difficult. We're collecting ideas for "2.0" (or 1.1) based on how people like and use this initial version.
Spartan  2009-07-03 12:00
Great idea to make this easy to use. The email confirmation is no big deal and worth having if it helps keeping spammers at bay.
Adam Bell  2009-07-03 12:29
Interestingly, the comment system doesn't work from NetNewsReader -- I sent one from there and didn't get a confirming email but this comment from Safari was confirmed almost immediately.
Glenn Fleishman  2009-07-03 13:09
Could be a defect in NetNewsReader's WebKit support? The form uses JavaScript with AJAX elements.
AHoffman   An apple icon for a TidBITS Supporter 2009-07-03 13:30
I've had lots of problems with NetNewsWire for commenting systems. It just doesn't hold onto cookies as it should.
Jeff Porten  An apple icon for a Friend of TidBITS 2009-07-05 09:36
Testing whether NetNewsWire works here. If this posts, then I had no problems with Safari 4's WebKit.
AHoffman   An apple icon for a TidBITS Supporter 2009-07-03 13:25
Easy enough to do from an iPhone during a family committment. Score!
(actually, I'm testing the cookies and need a second comment. It's all good from an iPhone. Now, how long aare these cookies going to last?)
Glenn Fleishman  2009-07-03 13:39
We set it up so that the cookies will last a year.
AHoffman   An apple icon for a TidBITS Supporter 2009-07-06 10:35
As I mentioned above, NetNewsWire doesn't hold on to cookies as it should. I understand that you set them to expire after a year, but I don't know how Mobile Safari treats them.
Jeffrey McPheeters  2009-07-03 14:11
This is aesthetically and technically pleasing to use. Thanks so much! After all these years, TidBITS continues to be the number one most trusted resource in my personal arsenal of Mac and Internet news and opinion. You make it possible for a digital immigrant such as myself to stay current and on focus with what I do, thanks to the many informed writers and thoughtful comments covering a wide range of pertinent subjects.
More words have never been written about comments than these. I mean wow. Could've just flipped your CMS to Wordpress or Movabe Type.
Glenn Fleishman  2009-07-03 19:09
Adam and I have been looking at, developing, supporting, deploying commenting systems since -- I dunno, 1994? Earlier? We don't want something that's irritating to our users.

Your suggestion isn't unreasonable, but having worked with both WP and MT, they just don't meet our needs for content management or commenting.

TidBITS has always documented our process to get to what we do, and how we do it. Our readers ask for it (those that don't just don't read the articles on it).

We find this encourages openness and collegiality that's led us to share info with other people and get information shared with us that's helpful.
Sure. Me too since the days of debating pagination on webpages. Makes sense if you're all homegrown. Many just start /blog. With some advanced techniques, in blog engines, you can block spam without captchas, but do require a sign in. Do that and comments will decline. Macworld is some odd, forum hybrid with redirects of confusion. The compendium of comment systems is impressive.
Dan Butcher  2009-07-04 02:58
I appreciate the detailed article on the issues and solutions involved in bringing comments to the site. I learn from seeing how others have approached a problem, and it helps me make better decisions in my own web work.
Jeff Carlson  An apple icon for a TidBITS Staffer 2009-07-05 21:18
Another reason the TCS is homegrown is because it allows us to add features without wondering if a company's CMS will support it. And, I think, because Adam likes being able to get Glenn on the line and say, "Make it so!"

Uh oh, maybe I'll get banned for that. :-)
Jeff Carlson  An apple icon for a TidBITS Staffer 2009-07-05 21:16
I'm glad to hear kudos about the aesthetics. We looked at a lot of commenting systems and... well, a lot of them are either ugly or confusing or both. Some of my early sketches fell into that camp ultimately, so we made a 180 degree turn near the end and went for simplicity.
Dainius Blynas  2009-07-04 00:50
very nice system. comments are having a come-back lately ...
That's right, comments are back baby! Joking aside, publishers are sweating now how to monetize traffic across social networks. We'll have entire threads on twitter none of which touches our blog or clicks.
Jack Ziegler  2009-07-05 11:52
Great job, I've been reading your stuff for years, and have learned so much. Thanks
donald g  2009-07-07 09:37
All of the Good comments are deserved - But those of us with weak, unreliable dial -up connections are now forced to also go to the web, not just email, to 'hear' what is going on. Not a happy solution for us
Adam Engst  An apple icon for a TidBITS Staffer 2009-07-07 10:05
We're looking into ways of delivering comments via TidBITS Talk as well, if that's what most people on TidBITS Talk want. Further email services will probably have to wait until we have a full account management system.
Echoing many others here, but it looks really nice. I'm impressed that you've managed to come up with a system that shows the threads and sub-threads clearly, something that I note even powerhouses like Macworld haven't been able to do.