Resetting Recent iCloud Bouncing Subscribers
Three weeks ago, starting with TidBITS #1574, Apple’s iCloud Mail servers decided that there was something about the TidBITS issues they didn’t like and started rejecting issues with this error:
SMTP Errors: 554 5.7.1 [CS01] Message rejected due to local policy. Please visit https://support.apple.com/en-us/HT204137
It took our support expert Lauri Reinhardt a little while to piece together questions from subscribers and her own tests. Once she had assembled a coherent picture of what was going on, I reported the problem to iCloud Mail support. Two days later, the iCloud Mail team told me that they had made appropriate changes to resolve the delivery problems. My tests confirmed that, but there was more to do.
The problem was that Sendy, our mailing list management software, had marked over 300 iCloud users as bouncing. (That may seem like a lot, but we have well over 5000 iCloud subscribers all told. I can’t explain why such a small percentage bounced.) I have reset all the affected icloud/me/mac.com addresses, so in theory, everyone who failed to receive the last issue or three should get TidBITS #1577 on 30 August 2021.
I say “in theory” because there are two other variables. First, if you subscribe from a custom email address that forwards to iCloud and doesn’t receive this week’s issue, you may still be affected; contact Lauri for help.
Second, Sendy gives us an interface for funneling messages to the Amazon Simple Email Service for the actual delivery. Amazon SES maintains a global suppression list that prevents email from going out to an address that has recorded a hard bounce for any Amazon SES user. We can remove addresses from Amazon SES’s global suppression list when we know they’re good and have fixed whatever problem was causing the bounces. But we have to do that one address at a time, with an image CAPTCHA, and there’s no way to query the list or even know if a removed address was on the list—all that Amazon SES tells us is that the address was removed if it was on the list. Doing that one by one with over 300 addresses would be a royal pain.
Amazon’s user guide says that Amazon SES automatically removes addresses from the global suppression list, increasing the amount of time an address stays on the list after each bounce. The longest an address will remain on the list is 14 days, and since TidBITS subscribers wouldn’t have bounced more than three times, I’m hoping Amazon SES has already removed all those addresses. If not, it should happen before TidBITS#1578 comes out on 6 September 2021.
With any luck, then, this situation will go away either this week or next. Our apologies for any confusion it may have caused—it’s yet another example of how systemic anti-social behavior by some (spammers) causes ongoing headaches for the rest of us.
Now you mention it, I forward mail sent to my personal domain to iCloud and I just noticed I did not receive a single issue of TidBITS after #1560 on 27 Apr 2021. I guess I didn’t miss them because I’ve been reading TidBITS via the website and used the mail issues as archive. So it appears Apple’s iCloud mail servers have been rejecting TidBITS issues a bit longer than 3 weeks?
Thanks for the update. I seem to be getting the Tidbits articles themselves (after a noticeable delay) but this Discourse seems to keep running into trouble delivering to my iCloud address. I follow with RSS too, so it’s possible that I’ve been missing other mail too and just didn’t notice, but either way it’s clear iCloud has been having trouble with Tidbits.
Sendy looks interesting, though it seems to be rather tightly wedded to Amazon SES. I did look around for information on using it with SMTP and it seems that bounce tracking isn’t in the package, but a patch adds the functionality. Setting up an MTA isn’t a walk in the park, but it’s not impossibly hard either, so perhaps if Amazon SES (which seems to have a reputation for being the way to avoid deliverability problems) gives you enough trouble, you could consider doing that instead and be the master of your own destiny. It’d probably also be faster since your MTA can probably run more efficiently in parallel too, even if you configured it ultimately to relay via Amazon. There again, I certainly appreciate how it would be much more convenient to avoid the overhead and configuration. Yup, spammers ruin everything.
Frans, we see an occasional bounce back from the iCloud servers now and then but this latest issue was more global for all those email addresses. I suspect your initial bounce back in April was probably related to something else. Send your email address to me at [email protected] and I’ll make sure your account is OK moving forward.
Thanks for giving the details. I noticed last week I was not getting the weekly email, the last one received was 8/2/21. As far as I know I don’t use iCloud but these things are subtle at times. I was pleased to get the email on 8/30 so it seems my .gov email was included in the issue.
Discourse has great reporting, so I can see that you bounced twice on August 6th and once on June 10th. I’ve reset your account to send email again.
I can’t tell exactly what the errors were. All we got back was:
I’m glad that delivery worked, but it was probably unrelated, since the iCloud Mail changes would affect only people with icloud/me/mac.com addresses or other addresses that forward to one of those addresses.
Thanks; it’s working.
Must you intervene every time? When I started bouncing, there seemed to be nothing I could do to restart mail. Perhaps Discourse could be made less sensitive or to restart delivery automatically sooner. If you didn’t intervene, perhaps it already does this: after not receiving mail for a while, I started again, before being bounced again, corresponding to the dates you gave or thereabouts.
I think so—I don’t see anything in Discourse that allows users to reset themselves. It seems like the sort of feature Discourse might offer, so I’ll ask.
In the meantime, I’ve slightly increased the bounce score necessary before you stop getting email, and I’ve lowered the number of days after which a bounce score is automatically reset from 30 to 5. We’ll see if that makes a difference—at most people would go 5 days without getting email.
I don’t know if it has anything to do with this behavior, but a couple times I have caught TidBITS in my spam folder instead of my InBox, including today. The email address I use is my webmaster address which is forwarded to my Gmail account. Weird!
We have trouble with spam filters because our issues are so long that they often run afoul of spam filters that use additive scoring systems and mark email that ends up with a too-high score. No individual article would have problems, but put them all together and it’s more likely.
Thanks for the explanation, I appreciate it. At least now I know why it ends up there. But in this day and age, there should definitely be a work-around or solution for that. So if I was to write a long article or a ten page email and send it to someone, it would wind up in their spam folder instead of in box because of the length? Crazy!
Sadly, it’s just how spam scores work—the more text you feed something, the more likely it is that some rule or another might be matched. Each one that’s matched adds to the score, and as far as I’m aware, there’s no acknowledgement that message size could increase that final score by increasing the possibility of more rule matches.
I had wondered where Tidbits emails had gone… and was about to message you when the latest issue appeared! Thanks for sorting!