This article originally appeared in TidBITS on 2017-11-20 at 3:21 p.m.
The permanent URL for this article is: http://tidbits.com/article/17644
Include images: Off

Next-Generation TidBITS Infrastructure in the Works

by Adam C. Engst

Has it really been a decade since the last redesign of the TidBITS Web site? (See “Designing a Modern Web Site for TidBITS [1],” 10 September 2007.) In fact, not everything has remained static over the past 10 years. We’ve moved the site from an Xserve to Rackspace [2] to Linode [3] and continually refined our tools and publishing practices. But at its core, the site has remained largely the same for a long time.

We’re currently working at full speed with our developer, Eli Van Zoeren [4], to rebuild nearly everything about our Internet presence with the goal of launching in early 2018. That includes the Web site and email issue design, along with our systems for article creation, content management, membership coordination, commenting and community discussion, account handling, email delivery, and more. We’ll be switching Web and email hosts too. Special thanks to our TidBITS members [5], whose contributions are making this transition possible!

I suspect that why we’re doing this is clear to most people, though perhaps not so painfully obvious as from our perspective. Our current Web site design looks and feels like it’s a decade old, and instead of seeming ironically retro, it makes the site hard to read on an iPhone and hurts our Google search rankings.

Why not update just the site design? Because of how it’s integrated into our Perl-based homegrown content management system, redesigning only the site would have been extremely difficult — it’s all intertwingled.

On the email side, a redesign would have been more straightforward, but we’ve struggled with deliverability problems over the years, so we’re looking forward to reworking our email system so we can outsource it to a company like SendGrid [6] that specializes in sending massive amounts of email. I’ve spent far too much of my life fighting with email — it’s time to let experts deal with it.

Because of being written in custom Perl code, our content management system is brittle and difficult to tweak, improve, or explain to another developer. Even Glenn Fleishman, who wrote it originally, has had trouble fixing problems since so much time has passed since he was immersed in Web programming. So we’re headed to WordPress [7]. It’s not perfect, but it’s robust, well-maintained, and highly extendable. If WordPress is good enough for the New York Times, it’s good enough for TidBITS.

The big win of WordPress is its extendability, both in terms of existing plugins and custom code. For instance, we plan to move TidBITS member management from eSellerate to the Paid Memberships Pro [8] plugin for WordPress, with payments processed through Stripe [9]. (In fact, I set up the TidBITS Content Network [10] site using WordPress and Paid Memberships Pro, so I have some experience there already.) Plus, Eli can write custom plugins for TidBITS-specific behaviors such as building Watchlist and ExtraBITS articles for the email issue.

The big unknown right now is how our article creation process will change. This won’t affect readers, of course, but is how Josh and I spend a lot of our time. We’ve been writing in BBEdit and committing new and updated articles to a Subversion repository that our content management system then reads. Aspects of it are great, but it is finicky to set up and use, which gets in the way of fluid collaboration with writers. Since we already rely on Google Docs for collaborative editing, we hope that WordPress’s Chrome add-on for importing posts from Google Docs [11] will eliminate the need for the copy-paste dance. If that doesn’t work, we’ll try Wordable [12] before investigating a custom solution.

Another advantage of moving to a new system is that it gives us a chance to clean up accounts. Until we sold Take Control Books to Joe Kissell (see “Take Control Books Acquired by Joe Kissell [13],” 1 May 2017), TidBITS and Take Control shared an account database. That’s no longer true, and we’re taking this opportunity to remove all users whose accounts were associated only with Take Control. We can also reduce the amount of data we manage since TidBITS-only accounts won’t track book ownership.

We’re also hoping to eliminate some confusion that comes about because our commenting system works with TidBITS accounts, but doesn’t require them, and because our Mailman-hosted TidBITS Talk mailing list uses entirely separate subscription accounts. All services going forward should rely on the same account information.

Speaking of commenting and Mailman, we’re replacing both. The commenting system works well but suffers from some annoying formatting limitations and other bugs we couldn’t fix easily, and Mailman requires too much work to maintain. The current plan is to use Discourse [14] for both per-article comments and general TidBITS Talk discussions. Discourse has a Web focus (and lots of interesting capabilities there) while still allowing email-based discussions, and we’ll see how it works.

Running WordPress and Discourse will require separate hosting accounts, but the combination should still be vastly simpler to deal with than our current CentOS-based virtual private server. While I can monkey through many Unix tasks, I find administering a production Unix server highly stressful and time-consuming. We’re running our staging version of WordPress at ArcusTech [15], which provides a fully managed server, and we’ll install Discourse at Digital Ocean [16] because ArcusTech doesn’t support the Ruby on Rails environment Discourse needs.

In the end, we’re looking forward to providing you with a Web and email presence that’s more visually appealing, mobile friendly, and modern. It will also be simpler — we plan to drop features that no longer make sense or that require too much work. For instance, the hard-wrapped text versions of the email edition require a lot of hand-coding while writing, complicate the email management interface, and look lousy in most of today’s email clients. Simultaneously, though, it should become easier for us to add new capabilities because we’ll have a platform that supports tons of plugins and that many developers understand.

You will certainly notice these changes, and you may need to log in to your TidBITS account again, tweak email filters, or make other minor adjustments to how you interact with TidBITS. But we’ll do our best to alert you to important things in advance and make the transition as easy as possible. We hope that you’ll appreciate all the changes from the front end as much as we’re looking forward to the new back end.

[1]: http://tidbits.com/article/9168
[2]: http://tidbits.com/article/11541
[3]: http://tidbits.com/article/12770
[4]: https://elivz.com/
[5]: http://tidbits.com/members.html
[6]: https://sendgrid.com/
[7]: https://wordpress.org/
[8]: https://www.paidmembershipspro.com/
[9]: https://stripe.com/
[10]: https://tcn.tidbits.com/apple-pros/
[11]: https://en.blog.wordpress.com/2017/03/07/wordpress-com-google-docs-collaborative-editing/
[12]: https://www.wordable.io/
[13]: http://tidbits.com/article/17200
[14]: https://www.discourse.org/
[15]: https://www.arcustech.com/
[16]: https://www.digitalocean.com/