Skip to content
Thoughtful, detailed coverage of everything Apple for 28 years
and the TidBITS Content Network for Apple professionals
Pretty picture of digital spheres

Photo by Gerd Altmann

40 comments

Juggling TidBITS Infrastructure Issues

It has been a long week, with hundreds of messages — in email, in Discourse, in SlackBITS, and on Twitter — about the new TidBITS Web site. Most were complimentary, and thank you for those! There were more login problems than I was happy about, and our new server suffered some performance issues. Eli Van Zoeren has been fabulous about snuffing out bugs as we find them, Lauri Reinhardt has been patiently helping numerous users with email and password issues, and I’ve been flitting from problem report to problem report, trying to reproduce, respond, submit bugs, and test solutions.

Site Performance and Reliability

The most severe problems we suffered last week revolved around the site responding very slowly or failing entirely. Building a modern Web site is a far cry from the process of writing HTML and CSS of yesteryear, and Eli spent a lot of time optimizing the site to reduce the number and impact of SQL queries for each page, dropping from 200 queries per page load down to 40. Another problem revolved around WordPress’s wp_cron function; we eliminated some freezes by switching to a true cron task for running scheduled events.

On the “hardware” front (it’s all virtual), we jumped from an ArcusTech $25-per-month virtual private server account with 2 CPUs and 1 GB of RAM to a $50-per-month account with 3 CPUs and 2 GB of RAM. It was impossible to predict how much RAM we’d need before going live, but 1 GB of RAM just wasn’t sufficient for the load the site placed on MySQL. That helped performance a good deal, and we’re watching to see if another upgrade will be necessary in the future.

Email Deliverability

When we signed up for a SendGrid account, the level that seemed to make sense was the Essential plan, which cost $20 per month for 100,000 messages. Unfortunately, all accounts at that level share a pool of IP addresses from which the messages come, and some companies that use SendGrid don’t have good email practices, with the result being that SendGrid shared IP addresses often get added to real-time blacklists (RBLs) like SpamCop. As your account ages, and if you maintain a good reputation within SendGrid, you start sharing with senders of similar age and quality, which reduces the chances of having your IPs end up on RBLs.

We didn’t realize all this initially, and a number of our messages in the first two days didn’t get through because of it. Since many of those were password reset messages, they caused no end of headaches for readers who never received their password reset links, and for Lauri and me in trying to figure out why in each case. SendGrid is good about reporting what happened to each message, but it was a lot of work.

To resolve this, I switched to SendGrid’s Pro plan, which costs $80 per month for 100,000 messages and slightly more for each message after 100,000 — you can see why I didn’t start there. With the Pro plan, we got a dedicated IP address that isn’t on any RBLs. So if you do any whitelisting of IP addresses, our mail will be coming from send.tidbits.com or 168.245.8.75.

Email deliverability should be better now, and once things settle down in a few months, we may look into switching to something like Sendy, which relies on the Amazon SES mailing service, and would cost a lot less per month.

Redesign Issues

Perhaps unsurprisingly, although most comments on our redesign have been overwhelmingly positive, some people don’t like it, or at least aspects of it. For a few, it seems to be just personal preference — they liked the old look. For others, the most common criticisms are that the new design is too bright and has a low information density.

I don’t have brain space to focus on visual design tweaks yet, since I’m dealing with much bigger issues, but I do appreciate the feedback, and once things settle down a bit, we’ll revisit the design and see if there are solutions.

As far as the information density goes, it’s fascinating to see how some people feel that white space is somehow wasteful, that if there’s an empty area in a window, it should be filled with something. We could increase the information density in a variety of ways, but all the ones we’ve thought of would detract from the usability and readability of the site, or would just be worthless. For instance, allowing the column width to expand with the window size would increase information density, but at the cost of long lines that reduce readability. And eye-tracking studies show that people read in an F pattern, so putting much of anything down the right side is largely useless. I’m open to suggestions, but please include pointers to sites that illustrate what you’re talking about.

Comments on Discourse Commenting

I’ve become fond of the Discourse system we’re using for article commenting in a very short time because it has a boatload of features that make reading, commenting, and administering the site a breeze. The notifications take a little getting used to, and we’re still figuring out how they interact with email, but we have a topic to explore that in the Site Feedback category.

Two annoyances have cropped up, however. First, readers expect there to be a field for leaving a comment at the bottom of each article, whereas instead there’s just a “Start/Join the discussion in the TidBITS Discourse forum” link at the very bottom of the article page.

Screenshot of TidBITS Discourse link

The reason for that is that Discourse’s editor is highly competent, so much so that it can’t be embedded on non-Discourse pages. So yes, there’s an extra click to jump over to our Discourse site to leave a comment. But it’s such a brilliant commenting environment that it’s worth the extra click. You can type in Markdown, BBCode, or HTML for formatting, drag in images, @mention users or #link to categories, and most important, select short bits of previous messages and click a quote button to quote the selected text in your reply. It’s resizable with the handle in the blue bar at the top, it provides a live preview, and it will even warn you if your post is nearly identical to another. Plus, you can go back and edit your post if you see a mistake, and I have full admin editing capabilities too.

Screenshot of Discourse editor — which is really cool

Second, and this is sort of related, there aren’t currently top-level ways to move back and forth between our main site and the Discourse site. That’s fixable, and we’ll be working on how to do that better this week.

TidBITS News App

We’re working on building the necessary custom feed for the TidBITS  News app, so we hope it will be able to come back soon. In the meantime, I’ve removed mention of it from the Get TidBITS menu.

Those who like the offline option of the TidBITS News app might look into following TidBITS in Apple News or employing another RSS reader app, such as Reeder, Feedly, or Unread.

Apple News and iTunes Podcast Feeds

Speaking of Apple News, we’ve installed a WordPress plug-in that sends our articles to Apple News in Apple News Format as we publish them. There appears to be some approval process that Apple itself is doing on our content, but assuming that goes well, Apple News should provide a better experience for reading TidBITS than it has in the past. To follow TidBITS in Apple News, tap Following in the News app’s toolbar, search for TidBITS, and tap the heart button to the right.

iPhone screenshots of finding TidBITS in Apple News

Similarly, we believe we have brought the iTunes podcast feed back online, so the next time we produce an audio version of an article, that feed should update. Needless to say, there hasn’t been time in the past week to record anything.

RSS Feeds and Readers

If you’re a TidBITS member and are logged in to the site, you can now get the link for the full-text RSS feed by clicking the Get TidBITS menu, and then RSS. It’s also available on your TidBITS Account page; just click Account in the site header.

One thing we’ve noticed is that many RSS readers do not honor the image size tags in our feed, which results in some images displaying at the full width of the window or at least the full size of the image, rather than the size we specified. That was especially problematic with some inline graphics that we used. If you’re using an RSS reader that displays our articles particularly well, let us know.

Email Client Formatting Problems

If you read our email on an iPhone in portrait orientation, we’re fighting with some style-related problems that can cause the column width to be half what it should be. Flip to landscape orientation or read on any larger-screen device and it should be fine.

A couple of readers wrote in to say that the TidBITS issue wasn’t formatting well in Thunderbird and Postbox. The solution turned out to be easy — make sure you’re running the latest version and then, if you’re still having formatting problems, choose View > Message Body As > Original HTML. On the plus side, the internal table-of-contents links work well in both Thunderbird and Postbox.

Screenshot of TidBITS issue in Thunderbird

More heartrending were the messages from a few readers who said that they were still using the much-beloved Eudora, which couldn’t parse the new format either. I’m not surprised — Eudora’s final update to version 6.2.4 happened in late 2006, and the state of, well, everything has changed significantly in the last decade. Plus, Eudora is a PowerPC app that requires the Rosetta environment, which means that anyone still using Eudora is also relying on Mac OS X 10.6 Snow Leopard or earlier and hasn’t updated since at least 2011.

Alas, there’s nothing we can do for people running an 11-year-old email app in a 6-year-old operating system. (And honestly, I’m surprised TidBITS would still be interesting, since you couldn’t run any software we talk about or even access many linked Web sites.)

No More Plain Text or Minimally Formatted Email

A handful of people wrote in to express regret that we no longer offer plain text or minimally formatted HTML email versions of the issue. Creating those issues was possible only because we were writing everything in Markdown, which is itself plain text and which can create only simple HTML. Even still, there was quite a lot of work that went into getting the issues to format well each week.

In the new system, we’re working first in Google Docs for collaborative editing and then in WordPress’s visual HTML editor. That enables us to do more formatting with graphics in particular, a limitation that was a constant irritation in the previous system. In the new system, for instance, we can resize graphics so they’re visually appropriate for the text around them, align them, and add ALT tags and captions.

That said, we’re now using html2text to generate plain text from our HTML and insert that in issues and articles as a text/plain MIME part. So if you can configure your email client to use the text/plain MIME part instead of the text/html MIME part, you might end up with a readable text version of our issue.

1Password, LastPass, and Safari

Finally, we’ve received more reports of problems with password managers like 1Password and LastPass than I’d like to see, and one or two people have also said that they’ve experienced trouble with Safari saving and autofilling passwords.

We haven’t been able to reproduce these problems here, so we’re not entirely sure what’s going on. However, TidBITS reader and LastPass user David Cuddy discovered that when he updated his TidBITS login details in LastPass, the password manager would no longer autofill them when logging in. He did some sleuthing and realized that the names of the form fields have changed. On our old site, they were tb_email_pup and tb_password_pup, whereas in the new WordPress-based site, they’re log and pwd. That discrepancy is likely confusing the password managers, since they’re looking for fields with the old names and not finding them.

David’s solution was to delete and recreate the LastPass entry for TidBITS, after which autofill worked fine. So if you’re having issues with your password manager not logging you into the TidBITS site, try copying out the recorded username and password to a new entry and deleting the previous one. (You might also be able to update the field names in the entry, but that’s likely harder.)

Thanks for Your Patience!

Please do keep the constructive criticism coming, and I’ll do my best to respond. I’m the sort of person who wants to make everyone happy, but that’s not possible when some people would be happiest if nothing changed at all. I haven’t shared all the technical problems with our previous system, but rest assured that many of them were intractable — the only solution was to start over. Change is necessary, particularly when it comes to technology, but we’re doing our best to make something that’s better, not just different.

Subscribe today so you don’t miss any TidBITS articles!

Every week you’ll get tech tips, in-depth reviews, and insightful news analysis for discerning Apple users. For 28 years, we’ve published professional, member-supported tech journalism that makes you smarter.

Registration confirmation will be emailed to you.

Comments About Juggling TidBITS Infrastructure Issues

Notable Replies

  1. Congratulation on the changes, everything is very positive from my point of view.

    I did have a little bit of trouble with my new membership credentials – after the expected password reset 1Password insisted that both my email address and password were the gogbbledegook string I asked it to create. After manually fixing that, log-in works just fine.

    I really love the formatting of articles through the web interface. For many years now I have used Safari’s “Reader” interface for virtually everything I read on the Web but the new TidBits HTML format actually looks better to me than the Reader version. TidBITS is one of only two web sites that I can say that about (the other is MondayNote).

    Keep up the great work Adam and everyone.

  2. Tonya experienced the same thing — no idea why 1Password would put the password in the username field for some people.

    I’m so glad to hear that!

  3. I’m really liking the new site. The article formatting is nice and clean and readable. The old design, while charming, was getting pretty dated. The only thing that seemed less than ideal with the new system is not being able to jump back to the main article from Discourse, but I’m happy to see you’re working on that. I think the Discourse editor is going to make me happy. Markdown formatting is a great feature. Formatting in the old system was hit-or-miss, especially adding links.

    So far, everything is working great in Reeder on my 10.5” iPad Pro.

  4. You wrote,

    As far as the information density goes, it’s fascinating to see how some people feel that white space is somehow wasteful, that if there’s an empty area in a window, it should be filled with something. We could increase the information density in a variety of ways, but all the ones we’ve thought of would detract from the usability and readability of the site, or would just be worthless.> Blockquote

    First, yes, the information density is lower than necessary.
    Second, whitespace carries information regarding change of context.
    Third, text to edge of window greatly hampers readability for some of us.
    Fourth, Reader mode allows great variability in text box shapes – nice for those special reading circumstances.
    Last, I don’t look for massive changes, just tweaks to satisfy the First comment.

  5. I’m using sendy since about 3 years, with Amazon SES and there were almost zero problems with this setup. Some features might still require some extra coding/setup (e.g. signup forms), but I’m very happy with it. The non-self-hosted offers were just too expensive for my case.

  6. On reading this lovely gem…

    More heartrending were the messages from a few readers who said that they were still using the much-beloved Eudora, which couldn’t parse the new format either. I’m not surprised — Eudora’s final update to version 6.2.4 happened in late 2006, and the state of, well, everything has changed significantly in the last decade. Plus, Eudora is a PowerPC app that requires the Rosetta environment, which means that anyone still using Eudora is also relying on Mac OS X 10.6 Snow Leopard or earlier and hasn’t updated since at least 2011.

    …I was reminded of this tweet sent to me four years ago.

  7. Looking good to me. Hang in there.

    I logged in with 1Password - no problem.

  8. Well I like the new changes!
    Going with “Discourse” was a great chose. I have a quick question.
    I have the Discourse app and use it for several other lists. I like it a lot.
    Will you folks be giving out invites so that I can read TidBits on the Discourse app?
    Or will it be web only?

    For me having one interface for all the lists, and discussions I want to keep up on, is a bonus.

    Anyway great job on the new site!

    My bad!.....
    The app I was talking about is "Discord" ...Not Discourse... 
    I thought they were the same.
    Please disregard the above.
  9. I think you’re smart to ignore (for now) “design-related” commentary.

    People always claim to want change, but they don’t actually like it. It’s been my experience that 90% of the time, people get used to the change fairly quickly, so addressing design-related changes immediately is a waste of your time. The other 10% may have valid concerns, but 50% of them are using it as an excuse to leave (when in reality, they simply weren’t engaged with your site anymore regardless of the design). What’s left is generally minor tweaks that you can do in off times after the infrastructure issues are dealt with.

    As for me, this redesign will actually put TB back at the top of my visit list after a very long time of barely even being on it.

  10. I like the new designs, both with your e-mail and your web site. I know they’ve been years in the making. Adam has been talking about a site upgrade since forever. :smile:

    I had to make the switch from Eudora a few months ago. Not everything made it through the conversion to Apple Mail so I’m keeping Eudora around in case I have to refer back to something that didn’t make the move. I was using Eudora OSE rather than Eudora 6.x, which was based on Thunderbird, a slightly more modern browser, using a plug-in to render the Eudora interface. Eudora’s toolbar icons are still best in breed, in my opinion, even after all these years. But OSE development was also dropped some years ago, making an upgrade of some kind inevitable. Frankly, I chose Apple Mail because Thunderbird development has been discontinued as well, except for occasional security fixes. Apple Mail may not be the best, but we can be pretty sure it will be around next year, something that cannot be said of any third-party e-mail client.

    I had trouble with my password reset, as described in the article. I’m heartened to learn I wasn’t alone. I doubt my replies to Lauri had much if anything to do with it, but I said you had erred in your selection of an e-mail service. As it turned out, you were able to solve the problem by selecting a higher level of service so you didn’t have to share IP addresses with bad actors who were blacklisted by spam blocking services. I would think that SendGrid would find a way to deny access to their IP addresses by such people. But even if they did it would take a while to get the IP addresses cleared by watchdogs like SpamCop. Frankly I don’t know how they justify using, let alone charging for, a service that is so open to abuse. I guess it’s just another case of caveat emptor.

    Anyway, Lauri did a great job helping me. She even took the time to dialogue a bit with me about it, rather than just sending me pro forma replies. Kudos.

    As for bounding back and forth from Discourse to your web site, when I clicked the Discourse link, it opened in a new page. I could just as easily have opened it in a new tab. The procedure may be different, but it’s no less functional.

    I’ll have to give some thought to upgrading my TidBITS membership in light of your new e-mail server costs. :frowning: There’s obviously a big difference between $20 a month and $80. It’s clearly a burden on a small business, and an unexpected cost at that.

  11. I too was struck by the text about Eudora users. I have such fond feelings about Eudora and although I use Gmail, sometimes loading it into the Gmail Web app or into the Mailplane wrapper app on the Mac, and sometimes loading it into Apple Mail where I can delete unwanted messages so much more efficiently, it’s not the same and I just deeply miss the ability to file my email effectively and efficiently. Anyway, I was smiling from ear-to-ear, Andrew, when I saw your post. :slight_smile:

  12. If you’re using Mac Mail, I highly recommend MailHub. I came across it when researching my switch away from Eudora a few years ago, and, dare I say it, it makes filing and managing my email much more efficient than in Eudora. It will even file messages as you send them. It also adds in some Eudora niceties like the combined to/from column (identifying emails you sent and displaying the recipient instead of the sender). Why that’s not a standard mail client feature, I’ll never know.

  13. For my part, I’d stopped participating in TidBITS Talk because I never figured out a good way to manage the volume of messages when switching between my phone and computer. I missed TBT, but am back in the game as it’s easy to keep on top of things with Discourse. :grin:

  14. In what ways would you suggest it be increased?

  15. That’s great to hear, and I might loop back to pick your brain about it after things settle down. SendGrid is doing a good job and has great features, but I worry that it’s going to end up costing us about $150 per month overall, whereas Sendy and Amazon SES might be around $35 per month.

  16. x-eudora-settings will live forever.

  17. Well said! My experience with major magazine redesigns is that there are often some more radical bits that end up getting toned down after a few issues, and there are some useful little things that often got lost in the redesign. I figure that after all the big stuff settles down, we’ll have the brain space to look closely and see what small changes really would be appropriate.

  18. It turns that one of my failings is that I don’t drive major infrastructure projects quickly, often because I’m so busy with producing and editing content each week, and helping readers with questions. But it’s also because I’m (rightly!) scared of how much work is involved, and how much it will all end up costing. It’s always easier to do nothing. But I can’t express how happy I am to be on this new system, despite the stress of getting it all working right.

    @lauri.reinhardt is indeed fabulous at what she does. We met her when she was doing tech support for eSellerate, and she was so good to work with that when we heard she’d left eSellerate and was looking for more work, we jumped at the chance to get her to help us with TidBITS and Take Control.

    Hmm, I wonder if that’s a browser setting. When I click a Discourse link on our main site, it opens in a new tab for me.

    Thank you for such consideration and the ongoing support!

  19. Hey Adam:

    FYI, the new infrastructure has messed up the formatting of the weekly email message on my machine. I am reading them in Outlook365 on a Windows 10 machine. The problem seems to be that Outlook’s CSS doesn’t recognize the image size in your messages. The images are much bigger than the window in which I am reading them, and consequently the text doesn’t wrap in the window but follows the size of image. It is similar to the problem described at https://medium.com/@jasemiller/a-fix-for-outlook-image-issues-in-html-email-campaigns-b8dd1c8f7d16.

    I can open the message in my browser and read it there, but it would be nice to be able to read it in email.

  20. This is my problem as well. If I try to read TidBITS at work I’m stuck with Outlook on a Windows 7 machine. I have to scroll left/right to read each line of text. Using “Zoom” doesn’t help as it merely changes the text size but does nothing for the width of the wrap.

  21. $%&(^%# Outlook… :slight_smile:

    Thanks for the reports, folks, and I’ll pass along that workaround to @elivz so he can take a look.

  22. Here is a perhaps odd behavior for TidBITS and RSS readers. I use igHome to create a homepage in Safari with links to sites I want to follow (I liked iGoogle until it went away…). igHome uses RSS feeds and “gadgets” to display the sites. I found the RSS feed for TidBits and am using that. The Heading says TidBITS, but if I click on that heading, I get the message “No RSS reader is installed …Safari can’t open Feed:https://tidbits.com/feed because Safari can’t display RSS feeds.” I have other web sites in igHome where this works fine - clicking the heading opens the web page with all the articles listed (such as Daring Fireball, Mac Stories, etc.) I wonder why TidBITS behaves differently - this would be a really handy way to access the complete current TidBits article collection.

    Incidentally: is there a way to “paste” in screen images (png files)? I could get that to work.

    David

  23. If you’re talking about in this forum, then yes. If the image is in your clipboard, you can just hit cmd-V (paste) when writing your message in the editor box. If you have an image file, you can hit the upload button in the editor toolbar (here I’m going to hit cmd-V to paste an image of the toolbar):

    image

  24. You can? How do you open the weekly issue in your browser?

  25. You can also drag and drop an image, which is pretty great.

    And you can drag it in the 03 middle of text

  26. Try it in a different browser — when I loaded our RSS feed into igHome, it worked. No images, but that’s a different issue, I think.

    55%20PM

  27. I tried that and got a long list of letters, numbers and symbols - not the images I wanted.

    David

  28. I tried that in Chrome and got the source code.

    David

  29. Ah, I understand now. What’s happening is that igHome is using the RSS feed as the link behind the TidBITS heading in its box. Since Safari can’t handle RSS feeds, it throws an error, and since Chrome displays them as code, it does that.

    Nothing we can do about igHome’s behavior, I’m afraid.

  30. Hi there Adam,

    For the most part I really like your new, 21st-century site (and components). It takes some getting used to, but everything new does.

    As for the “information density” issue, I agree with you. If you pack everything into a smaller space, it will look cluttered and clunky. As things stand now, I really don’t see a problem in this regard.

    –Phil V.

  31. When you are reading the weekly issue, go to “Actions” in the “Move” tab. You will see “View in Browser” as an option.

  32. Not sure what you are using to read the weekly issue, but I do not see an action nor a move tab.

    I thought you meant you were able to open the weekly issue in it’s own view from tidbits.com, but the “issue” page only shows summaries.

  33. Not as of yesterday sometime! Now there’s a Show Full Articles link on the right side of the issue page, which loads the full text of all the articles in the issue.

  34. I access TidBits via Feedbin in Chrome under OS X High Sierra. The behavior of the site since your upgrade has been perfect. I only faced a problem with my password when I went to log directly into the site. Apparently the new parameters are different than the old password requirements. Once my password was reset, everything works fine.

    Other than that, I’m all good.

  35. I am impressed with the new site. Athena-like emerging full-grown from the old TidBits. But I was disappointed at the lack of a serif font. Robin Williams (you know who I mean) would not approve. A serif font would be a nice touch for the Jobs’ quotes and perhaps some other places. Maybe you can even use a different font for each quote.

  36. That’s a good idea! Honestly, I was moving so fast when I made the first couple Jobs quotes that I just used something familiar.

  37. For myself I’d never participated at all, but I do now. Again, because of Discourse.

    In fact, it’s inspired me to create my own install of Discourse for a different community.

  38. The other huge benefit, in my opinion, is that the previous split between comment articles and article threads in TidBITS Talk was irritating. I tended to comment on the article as I wanted it to remain ‘attached’. But a lot of interesting discussion went on ‘behind the scenes’ on TB Talk (which over the past 18 months I’ve missed out on).

  39. Being able to combine article comments and general TidBITS Talk discussions in the same platform was one of our big goals, especially since TidBITS Talk had completely separate subscription info in the previous system, making it a lot more work to administer.

    My hope is the people will come here for article comments, and Discourse will show them lots of cool other conversations that are going on throughout the system, so they’ll stay and participate more.

Join the discussion in the TidBITS Discourse forum

1 more reply

Participants