Sierra PDF Problems Get Worse in 10.12.2
One of the first problems with macOS 10.12 Sierra revolved around PDFs created by Fujitsu’s ScanSnap scanners (see “ScanSnap Users Should Delay Sierra Upgrades,” 20 September 2016). Those problems turned out to be less severe than initially feared (see “ScanSnap Conflicts with Sierra Easily Avoided,” 3 October 2016), and Apple resolved them in macOS 10.12.1 (see “macOS 10.12.1 Sierra, watchOS 3.1, and tvOS 10.0.1 Mostly Fix Bugs,” 24 October 2016). Now, however, it seems Sierra’s PDF-related problems go deeper, and you should exercise caution when editing PDFs with Preview.
The first I heard that Sierra’s PDF-related problems might affect more than ScanSnap scanners came in a comment left on one of those articles on 26 October 2016. Developer Craig Landrum, who founded the document management system company Mindwrap, said:
The primary problem with Sierra with respect to PDFs is that Apple chose to rewrite the PDFKit framework in macOS 10.12 and it broke a number of things that PDF-related developers relied upon (I write scan-to-PDF software and know other developers who were impacted). Software that uses third-party PDF libraries probably runs fine, but those of us in the development community who relied upon Apple’s PDFKit library were really slammed — and we have no way to fix the problems ourselves. There have been numerous bug reports sent to Apple on the several serious issues found with PDFKit and we hope Apple addresses them in an upcoming point release.
Since Craig Landrum’s comment came after the release of 10.12.1 and the fixes for ScanSnap, I filed his criticism of PDFKit away as something that likely had been true but was no longer. However, throughout the next few months, additional complaints kept surfacing. Eric Bönisch-Volkmann, head of DEVONtechnologies, told me that they’ve spent a significant amount of development time working around Sierra’s PDF-related bugs in DEVONthink. Christian Grunenberg, DEVONthink’s lead developer, characterized the rewritten version of PDFKit in Sierra as “a work in progress,” saying:
Apple wants to use a common foundation for both iOS and macOS. However, it was released way too early, and for the first time (at least in my experience) Apple deprecated several features without caring about compatibility. And to make things worse, lots of former features are now broken or not implemented at all, meaning that we had to add lots of workarounds or implement stuff on our own. And there’s still work left to be done.
10.12.2 introduces new issues (it seems that Apple wants to fix at least the broken compatibility now) and of course fixed almost none of the other issues. It’s not only DEVONthink — a lot of other applications (such as EndNote, Skim, Bookends, and EagleFiler) are also affected.
In fact, Michael Tsai, developer of EagleFiler, just published a blog post confirming his problems with PDFKit:
I ran into a lot of PDF bugs in macOS 10.12.0. None have been fixed, as far as I can tell, and I’ve already filed two Radars for new issues in 10.12.2. It’s sad that basic functionality remains broken for so long — especially given that PDF was an area where Apple used to excel.
More concerning, and this is what finally pushed me to track down all these reports and write this article, is that the recently released macOS 10.12.2 has introduced a serious new bug related to PDFKit. Brooks Duncan of the DocumentSnap site published a note from one of his readers that warns that the OCR text layer added to scanned PDFs by Fujitsu’s ScanSnap software will be deleted if you edit the PDF in Preview. Eric Bönisch-Volkmann confirmed this, saying ruefully:
10.12.2 fixes a few bugs but kills the OCR text layer in PDFs. We worked around the earlier bugs in DEVONthink 2.9.8 and will address 10.12.2’s new problems in the upcoming 2.9.9. But yes, as soon as you edit a PDF in Preview the text layer is gone. Our customers are delighted.
Although the DocumentSnap reader said that the problem didn’t affect PDFs scanned and OCRed with other solutions, Brooks Duncan was able to reproduce the problem with scans made from both ScanSnap and Doxie scanners; he noted that both rely on the ABBYY FineReader engine.
Sonny Software’s Jon Ashwell, developer of the Bookends bibliography app, expressed significant frustration as well, saying:
We’ve been trying very hard to work around perfectly good code that was broken in Sierra. Versions 10.12.0 and 10.12.1 were bad, but 10.12.2 was a disaster for us, causing Bookends to crash when displaying PDFs with annotations. We’ve worked around that, but in the process had to shut down PDF annotations while we look for workarounds. I’ve filed a number of radars with Apple, two of which were closed as duplicates. In another case, I was asked to provide our app, but after doing so there has been only silence. I’ve never seen such a sorry case of sloppy code and indifference from Apple.
Problems with PDF annotations have plagued other developers as well, to judge from irate posts in Apple’s developer forums.
Christian Grunenberg laid the blame for the problems at Apple’s feet:
Apple supports only a subset of the PDF specification, and that support has always been buggy. For instance, PDF documents containing Eastern European characters created by the older ABBYY FineReader 8 engine are corrupted by PDFKit after editing. And issues reported by Peter Steinberger (author of the PDF framework PSPDFKit) were simply closed with the response that Apple didn’t intend to fix them.
Apps that don’t use PDFKit are immune from these problems, of course, but only to the extent that their PDFs aren’t shared more widely and edited in Preview. Greg Scown of Smile told me that PDFpen operates independently of PDFKit, but
bugs in Preview impact PDFpen customers whose document recipients use Preview rather than PDFpen to view or edit them. We have not had reports of PDFpen causing data loss of documents’ OCR layers.
Interestingly, Preview itself may suffer less from bugs in PDFKit than third-party apps. Michael Tsai said that some of the bugs he has seen don’t manifest themselves in Preview, suggesting that Apple’s Preview team is aware of the problems and is choosing to work around them rather than getting them fixed in PDFKit itself.
It pains me to say this, speaking as the co-author of “Take Control of Preview,” but I have to recommend that Sierra users avoid using Preview to edit PDF documents until Apple fixes these bugs. If editing a PDF in Preview is unavoidable, be sure to work only on a copy of the file and retain the original in case editing introduces corruption of any sort. Smile’s PDFpen is the obvious alternative for PDF manipulation of all sorts (and for documentation, we have “Take Control of PDFpen 8” too), although Adobe’s Acrobat DC is also an option, albeit an expensive one.
In the meantime, we’ll be watching closely to see which of these PDF-related bugs Apple fixes in 10.12.3, which is currently in beta testing.
PDF Expert is the best PDF reader/annotations tool I have. PDFPen is good for OCR though.
Actually, you heard at least the speculation from me before you heard it from Craig Landrum, when I posted:
"We don't know if ScanSnap used OS X system libraries and APIs to perform their PDF rendering or if they used their own or those of a 3rd party. If the former, it's possible that this problem is not limited to ScanSnap generated PDFs, but will also affect other PDFs created on the Mac (eg, MS Office). But if ScanSnap "rolled their own" PDFs, then the problem may be limited to ScanSnap."
http://tidbits.com/article/16810br/>
;-)
I had a "bad feeling about this". And Michael Tsai is absolutely right that this is particularly painful because native PDF support was one of the foundational hallmarks of OS X and for the Sierra team to so royally screw this up is very "Apple Maps" of them.
Truly unsettling.
You were right on, as it turns out!
Sounds like another bit of evidence that Apple no longer cares about Mac users. No updates for desktop Macs, mediocre to dismal updates for MacBooks, increasingly buggy Mac OS software, etc.
Preview is a HORRIBLE application. It has never, in all my years of using Macs, NOT ruined a PDF by making it unsearchable.
Also, large documents tend to cause frequent beachballs and crashing.
I highly encourage users to NEVER use preview for documents they care about and need to search regularly. (I.e. User guides, technical documents, etc.)
Preview is far from perfect, but you're overstating the criticisms against it. We pay a LOT of attention to Preview because of our Take Control books (which are long, complex documents where searching is key), and it is generally fine. Long ago, using the Reduce Size Quartz filter would delete bookmarks, but that was fixed.
For most people, most of the time, Preview is just a PDF viewer, and for that, it's usually fine. I'm sure there are PDFs out there that it can't handle, but that's not common.
Adobe Reader will spin beachballs with even moderately sized documents. With large documents you might as well go brew yourself a cup of coffee while you wait. That said, I prefer Adobe Reader to Preview for reading PDFs because it retains my view preferences. And, of course, with it PDFs are readily searchable.
What do you mean? Preview can also search PDFs.
I wonder if the new PDFKit explains why the ability to fax was removed in Sierra. Some scoff at faxing. However, for many firms in the medical and legal fields, fax is the only electronic transmission accepted. The US Robotics Model 5630 USB fax modem works great through El Capitan. In Sierra, it and other fax modems are unsupported. The menu selection for faxing is "Fax PDF." Is PDFKit the cause for loss of faxing in Sierra?
No, I don't think so. We covered that in:
http://tidbits.com/article/16826
And several readers dove in even more deeply in the comments. The removal of faxing was related to the removal of other low-level software.
Perhaps, but it's of a piece with the dumbing down of PDFKit. Apple has a long tradition of crippling or deleting useful features and software (Aperture). But I'm trying not to rant on TidBITS since I wrote a long e-mail to Tim Cook.
There's a telling thread about this problem as it relates to the app Skim.
https://sourceforge.net/p/skim-app/bugs/1109/
Preview and PDF viewing in general has been a mess since around 10.10, possibly 10.9. Somebody at Apple screwed up on the math that resizes the PDF content in the window in Preview, which made it blurry for 2 years before finally semi-fixing it in 10.12. Notice the blurry text and weird hairline border around the page: https://puu.sh/ktaeN.png
It's semi-fixed in 10.12, but it still happens sometimes in places like Quick Look and Safari especially when you try to zoom in:
https://puu.sh/ktayf.png
I tried filling a PDF form the other day, then Preview crashed and corrupted the PDF with it. Luckily file revisions still work and I was able to use that.
It seems Apple keeps wanting to rewrite perfectly functioning code, and they end up introducing bugs in everything they touch.
Mail.app, discoveryd, Preview and PDF, various bugs in the Finder and Safari... It's starting to get pretty annoying dealing with new bugs popping up in every release and almost never getting fixed properly.
yes has hallmarks of the discoveryd fiasco, doesn't it?
Tidbits continually ignores the very fine, FREE, Adobe software that is *the* gold standard for viewing PDFs - Acrobat Reader.
And for anyone that is editing PDFs for any serious purpose, Acrobat Pro is the clear choice. As the creator of PDFs, Adobe doesn't just throw compatibility to the wind like Apple does.
I know Adobe isn't a Tidbits sponsor and other companies are, but IMHO discussions about PDF should not routinely dismiss the Adobe tools (especially the free ones) as too expensive to consider.
Full disclosure, I work for Adobe, but not on Acrobat, so I am slightly more than just an Adobe fan.
I gave up on Adobe Acrobat Reader years ago because it was a slow, bloated pig with new security vulnerabilities regularly discovered.
The context of noting that Acrobat DC is expensive was in relation to editing PDFs, which Acrobat Reader won't do. And Acrobat DC is in a completely different price ballpark than PDFpen.
We do use Acrobat Pro XI for Take Control work, but we need Creative Cloud for InDesign as well. Newer versions of Acrobat won't work for us because essential plugins like Aerialist aren't compatible with Acrobat DC on the Mac, forcing us to stay with the old version. And, I have to say, I find even XI to be a UI step backward from earlier versions, since most of the controls are hidden and have to brought out manually and added to a custom sidebar.
I agree. I don't like the UI in the newer versions of Adobe Reader (it hasn't been Acrobat Reader for many years now, guys, but I guess old habits die hard). I try to avoid the Adobe Reader DC versions. For one thing their version numbers are a mess, with DC this and DC that. It's apparently up to version 15 now. I'm using Adobe Reader 11.0.18, which I think is the last pre-DC version. It seems to work find in Sierra. That said, Reader has always required at least a little tinkering in the toolbar, depending on what features you like to use. Compared to you guys, though, my needs are pretty basic, like reading Take Control e-books. TidBITS is good enough to make them available in other formats, ePub for iBooks and Mobi for Kindle. But, like I said, old habits die hard. If it ain't broke, don't fix it, which is sadly not part of either Adobe's or Apple's philosophical tool kit.
It seems to me that the DC versions of Reader try too hard to get you to use Adobe's cloud services. But, since I'm not using Creative Cloud, I haven't yet been drawn into their airy-fairy ecosystem. The CS 6 apps work OK in Sierra. Again I'm no longer into fancy document production, which means I can avoid both Adobe's and Apple's cloud briar patches. For now, anyway.
PDF Expert has been mentioned in this thread; it's been my default PDF reader on OS X since its release (because of my experience with the iOS version), and I can confirm that editing/annotating and highlighting does not remove the text layer and keeps the PDF file searchable.
Rendering of signatures is similarly broken since 10.12.2. Filing radars with Apple has no observable effect.
Hi guys, I'm Paul from PDF Expert team. Thanks for mentioning our app.
Since I see numerous concerns arising around this issue, I’d love to assure you that there’s nothing to worry about if you use PDF Expert.
We don't rely on PDF Kit and have our own file saving mechanics. Your PDFs are in good hands :)
Cheers
Since East European characters are mentioned, I'll jump in by saying that East Asian characters are also not searchable in a pdf if exported in Safari. Selecting them in Preview and you'll only see a series of blocks, rather than the characters themselves. I first notice this problem in Mavericks and Apple never got around to fixing it.
My Macs are too old for Sierra so I'm still on El Capitan. The Preview I have is version 8.1 (877.7).
1. What Preview version is on up-to-date Sierra machines?
2. Am I worried about PDFs being corrupted?
3. Things like this PDFKit problem will drive me to Windows. (I like Windows 10, so that isn't a problem for me.) I won't make that call until we see what's happening with desktop Macs, but PDFKit makes it more likely.
(My Apple ][ serial 537 is in the attic crying at this moment.)
Sierra's Preview is 9.0.
If you're not using Sierra, there's no need to worry, since you're not using the rewritten PDFKit.
the only workaround currently is using the adobe app to view / edit
Is it possible to replace the new Sierra PDFKit with an earlier version from El Cap?
Technically, yes, but then Sierra's Preview may not work properly. There was discussion of doing this in
https://sourceforge.net/p/skim-app/bugs/1109/
Actually I documented this - see my link below - and really, the issues you get appear to be much less than the issues you solve. It is not that "Sierra" isn't working properly, the only thing I did observe is that sometimes Preview may refuse to launch. But then I've also included a little script which you can call at runtime to just switch to the newer version of PDFKit.
I considered the problems gone after a hot fix, and hence no longer recommended doing that, but apparently they came back.
The only "real" drawback is that if you want to keep switching, you need to keep rootless disabled. Well, we didn't have that one for years, so I don't really care.
It's not just Sierra Preview that's buggy. I have issues in El Cap Preview as well.
For example, in PDF forms for some reason Preview will not display checked boxes. Open the same PDF in Adobe Reader and the checkboxes are checked. But you'd never know judging from Preview. There checkboxes just appear empty.
I would have switched to Adobe Reader if it weren't such a pig. A real shame Apple can't get its act together with Preview.
Superior PDF handling used to be one of the major OS X advantages. /sigh
This isn't generally true of Preview; when Josh and I wrote "Take Control of Preview," we had no trouble with checkboxes being checked. Try the IRS form 1040, for instance.
https://www.irs.gov/pub/irs-pdf/f1040.pdf
That said, PDF is a rather complex format, and it's entirely possible that some apps (including Preview) get it wrong, either in writing it out or reading it in. So I'm not surprised that Preview might have trouble with some PDFs. Acrobat Reader will be the gold standard for interpreting PDF.
Ironically I was having a lot of trouble with checkboxes with a US Government form. I-864 actually.
This so seriously sucks, as shown on the Skim forum. Apple is pushing back their most competent developers - perhaps because they're called variants of Christia(a)n? Anyway. Preview already got so much worse in like Yosemite when annotating became a multi click adventure. Now it sometimes just OOMs. And PDFKit breaks about everywhere.
Apple appears to get really everything seriously wrong these days. Laptops that for no reason drop ports and battery life, an OS that sucks each version more, etc. - and nobody from Apple seems to care as long as IPhones sell like sliced bread.
I've created not really a solution for the PDFKit problem, essentially documenting every single step you need to take if you want to restore an older version.
Maybe this helps someone:
http://www.mnott.de/how-to-workaround-the-fucked-up-pdfkit-in-sierra/
HTH,
M
Thanks for the link to the instructions!
Excellent tech journalism, thanks :-)
10.12.3 Beta 2 FIXES all the issues, well the ones that I've tested!
I can now view all my exiting PDFs, scan new ones and save them as encrypted and re-open them AND scanning with OCR the invisible text layer is preserved!!
Install it now!!! :)
I still can reproduce at least one issue:
Open PDF file generated by the ScanSnap software, edit&save it – OCR text is gone.
Apple needs to do some more homework.
All of a sudden I can't view real estate documents as pdf. Half the data of the page is missing. I've downloaded Adobe Reader as a partial workaround but this update is a mess! Electronic signing doesn't work, docs can't be viewed...I am hoping for an update soon. From what I read here this is exactly the problem.
I haven't seen anyone here commenting about Safari crashing when using Print function to create/save as a pdf file (rather than printing a document). This has happened ever since Sierra for me and others in our company. Perhaps it's related?
Seems to work OK here...
10.12.2 broke the display of the table of contents in Preview for me, but only for some PDFs: instead of chapter, etc., titles it now displays strings of garbage.
10.12.3 has the same issue. e.g. http://support.sas.com/documentation/cdl/en/proc/69850/PDF/default/proc.pdf (20 MB)
Fixed in 10.12.4
Beta 3 seems to have fixed ALL the issues! And heck, Beta 4 just came out!
Beta 4 seems fine too
The word I heard from a developer was that Beta 3 fixed the problem with Preview deleting the OCR text layer when editing a PDF, but none of the other issues with PDF handling in Sierra. We'll see what happens when 10.12.3 actually ships.
Many years ago Adobe Acrobat came with a PDF print driver that allowed you to print to PDF. Is there such a thing these days? Or just Apple's Save as PDF functions?
I actually use Adobe Reader for that. I sometimes have an issue when printing a PDF (that looks perfectly fine on screen) to our new Canon printers at work where a lot of special characters and equations come out all garbled. If I print from Reader, the printed copy looks as good as on screen though. Reader also allows saving as PDF ("Save a Copy...").
The version I have is quite old I believe. 9.5.5. But it still works.
Would be interested to know if the PDF issues are solved in today's 10.12.3 release. Beta comments here indicate at least some solved issues but wonder if others linger.
TIA
Early reports are that the bug that causes Preview to delete OCR text layers when editing a PDF has been fixed, but that's basically it.
Apago's previously excellent "PDF Shrink" application also seems to be affected. It includes a drag-and-drop applet that usually gave superior results in both quality and file size in seconds compared to what I could achieve by tweaking a PDF for minutes in Acrobat Pro. Since Sierra, however, the tool has been plagued with artifacts, including missing objects and improperly rendered masks. A very unpleasant surprise! C'mon, Apple! Your third party developers deserve better than this!
I've found Skim to be very slow since 10.12.2. I filed a bug with Skim .The developer explained it is a PDFKit issue: https://sourceforge.net/p/skim-app/bugs/1151/
I've got a book coming out that vaunts Skim and Preview. Now I've got to add this qualifier
I have a preference of Adobe Reader to view for reading PDFs because it retains my view preferences. And, of course, with it PDFs are willingly searchable.
10.12.3 kills PDFPen v.8.x which crashes on startup. Have to go back to v.7
I saw this crash in PDFpen Pro 8.3.1, but downloading 8.3.2 from Smile (click the Download Free Trial button) resolved it.
It turns out that this is entirely related to a developer certificate expiring and causing an unexpected crash — Smile just explained in email to users. Just download a new version of 8.3.2 as I suggested and all will be fine.