Yahoo-Backed POP Connections Cause TidBITS Formatting Error
A few months ago, I wrote about a quirky, sporadic problem that readers would report to us on occasion (see “LittleBITS: TidBITS Formatting Bug, Ransomware Protections, More OCR in Images,” 15 November 2021). For these readers, an article in the TidBITS issue would occasionally be formatted as a column of text that’s a single character wide. I tracked down the cause of each problem to corruption in the CSS styling for the issue that caused style="padding: 0.25em" to change to something like style="padding:25em". That corruption transformed a slight indent into a massive one.
If the problem had happened once, I could have chalked it up to a communications error somewhere along the way. But it was happening to multiple users, and some of them saw it multiple times, so there had to be some other commonality.
Our support whiz Lauri Reinhardt and I received reports from multiple ISPs, but upon closer examination, they weren’t as separate as they might have seemed. That’s because Verizon bought AOL in 2015 and acquired Yahoo in 2017. At some point, Verizon transitioned both aol.com and verizon.net email accounts to the Yahoo Mail backend. (Yahoo and AOL were acquired by Apollo Global Management in May 2021 and are no longer part of Verizon.) At some point, Frontier Communications also entered into a partnership with Yahoo for its email, adding its primary frontier.com and frontiernet.net domains to the list of those that rely on Yahoo Mail behind the scenes, along with earlier domains like citlink.net, ctaz.com, ctaz.net, and newnorth.net.
That was the first piece of the puzzle, but we have over 2100 subscribers from those domains plus yahoo.com, so if that’s all there was to it, we would undoubtedly be receiving more reports of the problem. After some back and forth with the people who reported the error, I figured out the remaining piece of the puzzle—they all used POP to download their email from the server. The vast majority of Internet email app users rely on the server-focused IMAP protocol for accessing email because it makes working on multiple devices much easier, but POP remains available at many ISPs.
TidBITS reader Bill Gruber provided a possible explanation. He had encountered a similar problem with periods disappearing and discovered that a line that contains just a period would terminate the transaction. (I can’t quickly parse POP3’s RFC 1939, but there is mention of CRLF.CRLF terminating a multi-line response.) Bill found that some ISPs were incorrectly stripping leading periods from a wrapped line, presumably to protect against an inadvertent termination. So if a line were broken such that it started with a period, the ISP would strip that period. The problem was impossible to reproduce because email headers are of random lengths, so the point where a line would be wrapped could change on every send. Needless to say, removing periods can play havoc with formatting, as we experienced, and Bill pointed out that it could be problematic in other ways, such as changing an instruction to administer .25 ml of medicine to 25 ml.
If POP were widely used and if it was easy to connect with Yahoo Mail support, I would report this bug. (Here’s an example of the difficulty of connecting with Yahoo support. Earlier this week, an acquaintance’s email account was compromised, and the attacker sent me multiple emails trying to fool me into sending him an Amazon gift card. After the first message, he tried to redirect our conversation to a similar-looking Yahoo account that wasn’t controlled by my acquaintance. After alerting my acquaintance to the attack via a text message, I looked for some way to report this obvious abuse to Yahoo but could find no appropriate support channel.)
Instead, for those who experience this problem, I suggest you switch to IMAP. If that’s not something you wish to do for technical or philosophical reasons, you can read TidBITS articles that get corrupted in email on our website instead—just click the article title in the issue to load it in your browser.
No company is more appropriately named than “Yahoo.”
But my friend who is my ISP and I still use POP, we both like the concept of “my email is on my machine.”
ISPs that mess with content deserve to be shot. It might be acceptable if they were collectively competent, but that’s a huge “if”.
I maintained my POP accounts for that reason… wanted everything on my device. But eventually it became too much to edit the same emails on multiple devices. I’d suggest that, if that is NOT a problem, finding a ‘non-Yahoo’ provider is critical. If Adam can’t contact them with a serious support issue, they shouldn’t be providing anyone a critical service!
POP has been around for how many decades? And a major provider like Yahoo can’t get it to work. Shameful.
You can still do that with IMAP, if you like. Just configure a script/filter on your mail client to copy or move everything to a local mailbox (e.g. a folder under the “On My Mac” mailbox, if you’re using Apple Mail).
Now your message is stored locally and not on the server. And with the added bonus that you can always copy/move specific messages back to the server if you have a need (e.g. to sync to another computer).
That’s just demented,
POP says that the POP server needs to fold lines so that no response is more than 512 characters long (including trailing CRLF). Every response must end with CRLF, and it signals the end of the mail with a .CRLF (a line containing only “.”).
To stop inadvertent signalling of EOF, if any line (as sent by the server!) would start with a “.”, then the server adds an extra “.” at the start of the line.
The *client", if it sees any line beginning with a “.” that is not the EOF indication (a line containing only “.”) just removes it. (Note that it doesn’t just convert a leading “…” to “.”, it strips any leading “.”. A POP server could in fact start every line with an extra “.” and it would work, although it would technically not be adhering to the standard).
The Yahoo POP server is broken. Or else something upstream of it is broken.
That would be less important if a number of ISPs worldwide had not got out of the ISP business and outsourced their business to Yahoo.
You should be able to reach a Yahoo technical human by checking out the steps at Whois yahoo.com
I still have a couple of old “Yahoo” accounts which are really from the older ATT days but it’s my understanding that these accounts are maintained by a division of Verizon now. So even though you go into your ATT account to make changes such as passwords or add a sub accounts, they don’t maintain those as they did previously. I have very few issues with my POP accounts so the TidBits one doesn’t bother me since I can use another email account like gmail if need be.
Thanks! But wow, that’s convoluted—you have to request a link to a webmail form, and the webmail form itself failed to actually work.
They’re probably back to the new Yahoo, now that Verizon sold it again.
While I should be surprised, I am merely disappointed.
RFC822 (the defining rfc for email) actually requires that [email protected] exists:
Of course, I’m not holding out much hope that Yahoo! actually adheres to this part of the standard.
The Postmaster address probably does exist.
Whether that mailbox is read by anybody responsible is another question altogether.
@billgruber, who helped me track down this problem, also shared some instructions for Frontier Communications users who want to switch to IMAP. That’s harder than it seems because Apple Mail auto-configures Frontier addresses to use POP. Here’s what Bill suggests:
Wow, fantastic detective work!!
And what a royal PITA!
I can’t believe that Yahoo got this basic protocol parsing issue wrong.
Signaling like this shows up elsewhere too. You can talk raw SMTP like this:
That last dot ends the transmission. And the protocol knows not to include that as part of the message body.
That’s a fairly embarrassing bug
Change their name to Yikes!?
A similar issue, perhaps the same one? You might just want to skip down to the TL;DR below.
I wonder… before the Before Time, I used Verizon’s own mail via POP, and never had issues.
A few years before the Before Time, Verizon bought Yahoo! and eventually stuck is email customers with AOL email, which features zero customer support with cash app front. Since I was familiar with it, I chose to stay with POP. All of a sudden, URL links in HTML-rich emails would often (but not anything like uniformly) lead to Safari heartburn when clicked… because a period was missing in the string that got communicated to Safari. It happened just often enough to be a minor nuisance, and just infrequently enough that I could never summon up the gumption to investigate. And of course I was using POP, because years ago Steve Dorner had admitted how worthless he thought IMAP was. And because I don’t want my email on anyone’s sever after I’ve downloaded it. (I also hate push email but that’s another issue.)
And yes, occasionally, I’d get formatted emails parts of which displayed as one character per line.
When Verizon sold AOL, I figured the handwriting had at least begun to appear on the wall for any train of accountability to the customer, so I solicited opinions for a good alternative that didn’t involve the company whose name used to begin with “G.” You know, the folks who first made customers and their data the product. The best reviews were for fastmail, so I decided to go with them, despite the fact that it meant paying a nominal annual fee, the first time I’d ever had to pay for email service. I didn’t switch over immediately, just kept the fastmail account for a couple of contacts and a rainy day.
It started to rain last year, when AOL really got obnoxious, and decided to implement (purportedly) better security by having all its verizon.net email users literally destroy their accounts, and reenter the configuration info in their clients, along the new security config. Fair enough, but it indicated yet another bit of “the heck with the customer” attitude on AOL’s part. Why not provide an automatic, self-configuring file, as fastmail does?
At that point, it became obvious that the thing to do was simply to switch to fastmail, so I did. It was pretty easy to get my regular and some not so regular, correspondents to use my new address, though I didn’t recognize how persistent numerous semi-legitimate ones (charity and political donation seekers) persisted in using my old address, even after I’d either changed my profile settings at their sites, or unsubscribed (hah!) from emails to the old address.
TL;DR: Since I switched to fastmail’s IMAP service, I have experienced exactly zero instances of the mysterious, disappearing full stop in a URL link from an HTML-formatted email. So here’s to IMAP, or maybe just a decent email service provider.
This was indeed great sleuthing! Thanks for the reporting. It’s a minor thing, but small inexplicable failures like this can make a person crazy.
And wow it’s hard to believe that Yahoo’s POP implementation is to blame. I remember writing custom POP handlers and MIME parsers back in the 90’s. One would expect this to be a solved problem by now!
I dropped my Yahoo account years ago, which aside from switching to IMAP is probably the only “solution” here. I expect that even if you could report this issue to a real human who believes you and understands the situation, Yahoo as a company would not care enough to fix this.
This is POP-related, though not quite the same subject.
I still use a swbell.net eddress that’s 20 years old. Originally it was accessed via POP using Eudora. But after SWBell merged into AT&T, it was eventually handled by Yahoo under subcontract to AT&T. In August 2014, the POP access suddenly stopped working–no error message from Eudora, but every attempt at access just quit quietly. Since then, I’ve used the Web access thru att.com to yahoo.com, but I dislike not being able to archive my mail on my own machine. This thread makes me wonder whether there’s a way to regain POP access to Yahoo-based email. Suggestions, please?
Alternatively, is there any other way to download years worth of emails from that Yahoo-based account? The Web interface certainly doesn’t seem to offer one.
You should be able to if it’s similar to other ATT addresses that go way back such as yours and att.net, sbcglobal.net etc. The POP info can be found if you Google for that information but the main settings in your email program have to be:
incoming mail server: inbound.att.net
outgoing mail server: smtp.att.yahoo.com
When creating a new account, you may have to do what others have mentioned in this thread and other places: you have to trick the initial setup by putting in the correct email address but the wrong password which will eventually take you to the advanced settings where you can enter the information from above.
Whatever mail is on the server now will download once you set up your POP account assuming it was deleted at some point.
Now, if your password does not work when setting it up you may have to go into your yahoo.com page and find the section where you create a secure mail key. If your Eudora program could not support Open Authentication, that could be why it stopped working a few years ago. Yahoo/ATT sent numerous emails about that issue at the time but even if your email program supports it like Apple Mail, you might still need that new secure password since there is some glitch in their system that can cause that to happen. I’ve always used POP for my old ATT accounts with no issues, only some password ones when I changed them on their website. Then I had to make a secure mail key for Apple Mail even though there should not have been an issue.
See the following regarding the secure mail key if needed:
Thanks! Now, recommendations for a good Mac-based email program? I don’t want to use Mail, because that already handles another eddress for me, and I want to avoid any possibility of confusion. I want something that will run under Mojave, which is my main workhorse at the moment.
I have five different addresses I use for various reasons and Apple Mail does fine with all of them. Four are POP and one is IMAP so you can mix the accounts. I’m sure others here can recommend other programs but Apple Mail works fine for me.