Photo by Gerd Altmann
It has been a long week, with hundreds of messages — in email, in Discourse, in SlackBITS, and on Twitter — about the new TidBITS Web site. Most were complimentary, and thank you for those! There were more login problems than I was happy about, and our new server suffered some performance issues. Eli Van Zoeren has been fabulous about snuffing out bugs as we find them, Lauri Reinhardt has been patiently helping numerous users with email and password issues, and I’ve been flitting from problem report to problem report, trying to reproduce, respond, submit bugs, and test solutions.
Site Performance and Reliability
The most severe problems we suffered last week revolved around the site responding very slowly or failing entirely. Building a modern Web site is a far cry from the process of writing HTML and CSS of yesteryear, and Eli spent a lot of time optimizing the site to reduce the number and impact of SQL queries for each page, dropping from 200 queries per page load down to 40. Another problem revolved around WordPress’s wp_cron function; we eliminated some freezes by switching to a true cron task for running scheduled events.
On the “hardware” front (it’s all virtual), we jumped from an ArcusTech $25-per-month virtual private server account with 2 CPUs and 1 GB of RAM to a $50-per-month account with 3 CPUs and 2 GB of RAM. It was impossible to predict how much RAM we’d need before going live, but 1 GB of RAM just wasn’t sufficient for the load the site placed on MySQL. That helped performance a good deal, and we’re watching to see if another upgrade will be necessary in the future.
When we signed up for a SendGrid account, the level that seemed to make sense was the Essential plan, which cost $20 per month for 100,000 messages. Unfortunately, all accounts at that level share a pool of IP addresses from which the messages come, and some companies that use SendGrid don’t have good email practices, with the result being that SendGrid shared IP addresses often get added to real-time blacklists (RBLs) like SpamCop. As your account ages, and if you maintain a good reputation within SendGrid, you start sharing with senders of similar age and quality, which reduces the chances of having your IPs end up on RBLs.
We didn’t realize all this initially, and a number of our messages in the first two days didn’t get through because of it. Since many of those were password reset messages, they caused no end of headaches for readers who never received their password reset links, and for Lauri and me in trying to figure out why in each case. SendGrid is good about reporting what happened to each message, but it was a lot of work.
To resolve this, I switched to SendGrid’s Pro plan, which costs $80 per month for 100,000 messages and slightly more for each message after 100,000 — you can see why I didn’t start there. With the Pro plan, we got a dedicated IP address that isn’t on any RBLs. So if you do any whitelisting of IP addresses, our mail will be coming from send.tidbits.com or 18.104.22.168.
Email deliverability should be better now, and once things settle down in a few months, we may look into switching to something like Sendy, which relies on the Amazon SES mailing service, and would cost a lot less per month.
Perhaps unsurprisingly, although most comments on our redesign have been overwhelmingly positive, some people don’t like it, or at least aspects of it. For a few, it seems to be just personal preference — they liked the old look. For others, the most common criticisms are that the new design is too bright and has a low information density.
I don’t have brain space to focus on visual design tweaks yet, since I’m dealing with much bigger issues, but I do appreciate the feedback, and once things settle down a bit, we’ll revisit the design and see if there are solutions.
As far as the information density goes, it’s fascinating to see how some people feel that white space is somehow wasteful, that if there’s an empty area in a window, it should be filled with something. We could increase the information density in a variety of ways, but all the ones we’ve thought of would detract from the usability and readability of the site, or would just be worthless. For instance, allowing the column width to expand with the window size would increase information density, but at the cost of long lines that reduce readability. And eye-tracking studies show that people read in an F pattern, so putting much of anything down the right side is largely useless. I’m open to suggestions, but please include pointers to sites that illustrate what you’re talking about.
Comments on Discourse Commenting
I’ve become fond of the Discourse system we’re using for article commenting in a very short time because it has a boatload of features that make reading, commenting, and administering the site a breeze. The notifications take a little getting used to, and we’re still figuring out how they interact with email, but we have a topic to explore that in the Site Feedback category.
Two annoyances have cropped up, however. First, readers expect there to be a field for leaving a comment at the bottom of each article, whereas instead there’s just a “Start/Join the discussion in the TidBITS Discourse forum” link at the very bottom of the article page.
The reason for that is that Discourse’s editor is highly competent, so much so that it can’t be embedded on non-Discourse pages. So yes, there’s an extra click to jump over to our Discourse site to leave a comment. But it’s such a brilliant commenting environment that it’s worth the extra click. You can type in Markdown, BBCode, or HTML for formatting, drag in images, @mention users or #link to categories, and most important, select short bits of previous messages and click a quote button to quote the selected text in your reply. It’s resizable with the handle in the blue bar at the top, it provides a live preview, and it will even warn you if your post is nearly identical to another. Plus, you can go back and edit your post if you see a mistake, and I have full admin editing capabilities too.
Second, and this is sort of related, there aren’t currently top-level ways to move back and forth between our main site and the Discourse site. That’s fixable, and we’ll be working on how to do that better this week.
TidBITS News App
We’re working on building the necessary custom feed for the TidBITS News app, so we hope it will be able to come back soon. In the meantime, I’ve removed mention of it from the Get TidBITS menu.
Apple News and iTunes Podcast Feeds
Speaking of Apple News, we’ve installed a WordPress plug-in that sends our articles to Apple News in Apple News Format as we publish them. There appears to be some approval process that Apple itself is doing on our content, but assuming that goes well, Apple News should provide a better experience for reading TidBITS than it has in the past. To follow TidBITS in Apple News, tap Following in the News app’s toolbar, search for TidBITS, and tap the heart button to the right.
Similarly, we believe we have brought the iTunes podcast feed back online, so the next time we produce an audio version of an article, that feed should update. Needless to say, there hasn’t been time in the past week to record anything.
RSS Feeds and Readers
If you’re a TidBITS member and are logged in to the site, you can now get the link for the full-text RSS feed by clicking the Get TidBITS menu, and then RSS. It’s also available on your TidBITS Account page; just click Account in the site header.
One thing we’ve noticed is that many RSS readers do not honor the image size tags in our feed, which results in some images displaying at the full width of the window or at least the full size of the image, rather than the size we specified. That was especially problematic with some inline graphics that we used. If you’re using an RSS reader that displays our articles particularly well, let us know.
Email Client Formatting Problems
If you read our email on an iPhone in portrait orientation, we’re fighting with some style-related problems that can cause the column width to be half what it should be. Flip to landscape orientation or read on any larger-screen device and it should be fine.
A couple of readers wrote in to say that the TidBITS issue wasn’t formatting well in Thunderbird and Postbox. The solution turned out to be easy — make sure you’re running the latest version and then, if you’re still having formatting problems, choose View > Message Body As > Original HTML. On the plus side, the internal table-of-contents links work well in both Thunderbird and Postbox.
More heartrending were the messages from a few readers who said that they were still using the much-beloved Eudora, which couldn’t parse the new format either. I’m not surprised — Eudora’s final update to version 6.2.4 happened in late 2006, and the state of, well, everything has changed significantly in the last decade. Plus, Eudora is a PowerPC app that requires the Rosetta environment, which means that anyone still using Eudora is also relying on Mac OS X 10.6 Snow Leopard or earlier and hasn’t updated since at least 2011.
Alas, there’s nothing we can do for people running an 11-year-old email app in a 6-year-old operating system. (And honestly, I’m surprised TidBITS would still be interesting, since you couldn’t run any software we talk about or even access many linked Web sites.)
No More Plain Text or Minimally Formatted Email
A handful of people wrote in to express regret that we no longer offer plain text or minimally formatted HTML email versions of the issue. Creating those issues was possible only because we were writing everything in Markdown, which is itself plain text and which can create only simple HTML. Even still, there was quite a lot of work that went into getting the issues to format well each week.
In the new system, we’re working first in Google Docs for collaborative editing and then in WordPress’s visual HTML editor. That enables us to do more formatting with graphics in particular, a limitation that was a constant irritation in the previous system. In the new system, for instance, we can resize graphics so they’re visually appropriate for the text around them, align them, and add ALT tags and captions.
That said, we’re now using html2text to generate plain text from our HTML and insert that in issues and articles as a text/plain MIME part. So if you can configure your email client to use the text/plain MIME part instead of the text/html MIME part, you might end up with a readable text version of our issue.
1Password, LastPass, and Safari
Finally, we’ve received more reports of problems with password managers like 1Password and LastPass than I’d like to see, and one or two people have also said that they’ve experienced trouble with Safari saving and autofilling passwords.
We haven’t been able to reproduce these problems here, so we’re not entirely sure what’s going on. However, TidBITS reader and LastPass user David Cuddy discovered that when he updated his TidBITS login details in LastPass, the password manager would no longer autofill them when logging in. He did some sleuthing and realized that the names of the form fields have changed. On our old site, they were
tb_password_pup, whereas in the new WordPress-based site, they’re
pwd. That discrepancy is likely confusing the password managers, since they’re looking for fields with the old names and not finding them.
David’s solution was to delete and recreate the LastPass entry for TidBITS, after which autofill worked fine. So if you’re having issues with your password manager not logging you into the TidBITS site, try copying out the recorded username and password to a new entry and deleting the previous one. (You might also be able to update the field names in the entry, but that’s likely harder.)
Thanks for Your Patience!
Please do keep the constructive criticism coming, and I’ll do my best to respond. I’m the sort of person who wants to make everyone happy, but that’s not possible when some people would be happiest if nothing changed at all. I haven’t shared all the technical problems with our previous system, but rest assured that many of them were intractable — the only solution was to start over. Change is necessary, particularly when it comes to technology, but we’re doing our best to make something that’s better, not just different.