Apple Needs Public Betas for Mac OS X
It’s unlikely that we’ll see a Mac OS X 10.6.9, given the nearness of Mac OS X 10.7 Lion’s release, but if Apple were to release 10.6.9, I would be forced to recommend that TidBITS readers not install it immediately. After all, both 10.6.7 and 10.6.8 introduced entirely new problems that affected large numbers of people while resolving extremely specific bugs that likely affected only a small set of the user base.
That’s unlike Apple. Although there have been exceptions, Mac OS X releases have, for the most part, been either positive steps forward or at least neutral. As a result, I’ve had little reason to recommend extreme caution when upgrading. Early upgrades — the 10.x.1 and 10.x.2 releases — generally address common bugs and problems and are thus worth getting quickly. And later upgrades — those that come after 10.x.5 — are largely aimed at mopping up highly particular problems and lurking infelicities, and seldom introduce new problems.
So what explains the font problems in 10.6.7 that forced the Snow Leopard Font Update (see “Apple Releases Snow Leopard Font Update,” 26 April 2011), or the printing problems in 10.6.8 that are currently best addressed by replacing four Unix applications with their previous versions (see “Mac OS X 10.6.8 Suffers Printing and Audio Problems,” 1 July 2011)? Are Apple’s testers just getting sloppy? Are they too focused on testing Lion to give sufficient attention to Snow Leopard? What about developers? Perhaps they too have been more focused on developer previews of 10.7 than on new versions of 10.6?
Or maybe this is part of the price of success? What if these problems are directly related to the ever-increasing popularity of the Mac and the concomitant growth of the surrounding industry of software and hardware manufacturers? And, of course, to Apple maintaining this growth over the last 10+ years, which has given us not just multiple generations of software and hardware, but an expanding number of professional users who rely on their Macs for day-to-day work.
In short, perhaps the entire Mac ecosystem has grown so large that it has exceeded Apple’s internal ability to test sufficiently against the myriad real-world configurations that have sprung from each and every Macintosh purchase. And, I would argue, even if Apple doesn’t target the Mac at the enterprise market, the consequences of introduced bugs have more of an impact on professionals than ever before, simply due to the increased number of people relying on the Mac for their livelihoods.
What then is the solution? Certainly, Apple could just employ more testers and move more cautiously with Mac OS X releases. Or we users could attempt to hold developers responsible for better testing — it’s somewhat troubling that both Parallels and PGP failed to discover and address (either in code or with communication) their conflicts with 10.6.8 before it shipped.
But neither of these solutions is satisfying. Apple can’t possibly test every conceivable user configuration, nor would we want the company to delay necessary fixes for obsessive testing. And yes, it would be nice if developers did a better job of testing too, but all developers have limited resources to devote to the task.
I propose a more modern, democratic solution: public betas of interim Mac OS X releases, starting with 10.7.1. As much as I know the very concept goes against Apple’s secretive, controlling grain, I think the company should consider it.
[Update: In fact, as I’ve learned from some of the comments on this article, Apple actually has a program somewhat along these lines. Called AppleSeed, it’s an invitation-only program that enables everyday users to test pre-release versions of Apple software and provide Apple with feedback. From what I can tell (and what I read in this recent Ars Technica article) it’s almost entirely secret, though, so it’s unclear how it compares with what I suggest below, and the simple fact of the matter is that, just like Apple’s internal testing and developer testing, it
failed to bring the problems that plagued 10.6.7 and 10.6.8 to the forefront.]
Of course, I’d hope Apple would do a better job than many other companies that do public betas. For instance:
- The option to get public betas should be something that the user sets in Software Update, so participating in a public beta doesn’t require users to waste time manually downloading and installing. This is just respect for your testers — let them focus their time on actual testing, rather than fussing with manual installations.
- For those who install the public betas, there should be an option to revert to the previous installed version, and to the current shipping version. That way, if a user installs a public beta and discovers a bug that prevents everyday usage, she can revert easily to a functional version of the operating system. Again, such a feature acknowledges that testers are donating their time, and if a beta is harming their ability to earn a living, it should be easy to return a Mac to a functional state.
(In fact, now that I think of it, Software Update should work with Time Machine to create a snapshot of the operating system before installing an update for all users, so if a user experiences problems, reverting is merely a matter of clicking a button in Software Update. Although I’ve not seen this personally, my understanding is that Windows has long had a feature like this. In an involved Twitter conversation, TidBITS friend Andrew Laurence suggested that this is actually the key thing for Apple to work on, since he felt a public beta program wouldn’t serve Apple’s interests, whereas the capability to revert from a troublesome update would
help everyone.) -
Once a public beta is installed, a Submit Bug Report item should appear in the Apple menu. Choosing it should display a form that helps users make useful reports; it could even be an assistant that walks the user through several screens, collecting information about the problem in each one. Those bug reports, along with automatically collected logs, should go directly into Apple’s internal bug report database. Currently, the only way to report bugs is via Bug Reporter, a clumsy Web-based system that feels as though it hasn’t been updated in at least 5 years.
-
I don’t have a suggestion of how this could best be done (iTunes gift cards? Hardware discounts?) but Apple should reward users who consistently submit helpful and accurate bug reports. It’s a small thing, but when a large company like Apple acknowledges individual assistance, as in the credit for reporting security vulnerabilities, it goes a long way.
-
One final thought. Especially now that Apple IDs are required to purchase Mac OS X Lion, Apple could open up a private discussion forum for those participating in the public beta program, which would enable people to figure out whether any problems they were seeing were more widespread. I have no idea if Apple’s discussion software would support this, but it would be extremely cool to give users a rating based on how many good bugs they’d reported.
Again, the very concept of allowing users to see betas of Mac OS X goes against Apple’s desire to control the message, but as far as I can tell, Apple isn’t enforcing its developer non-disclosure agreements at all — many Web sites are now publishing NDA-covered information about Lion with impunity. So perhaps Apple doesn’t care about technical details leaking out, and with small updates like 10.7.1, it’s not as though there would be any damage to Apple’s competitive position if someone were to discover that a particular bug has been fixed or some other problem has been introduced.
And, on the plus side, releasing public betas could significantly improve customer confidence in Apple’s ability to provide a solid operating system. It takes a long time to win trust that can be lost nearly instantly, and the last two releases of Snow Leopard have distinctly hurt Apple’s reputation.
I think this is especially important given a recent review I've seen of Lion, suggesting it is the "Vista" of the MacOSX series. Please read the review: Mac OS X Lion: This Is Not the Future We Were Hoping For : http://lifehacker.com/5819508/mac-os-x-lion-this-is-not-the-future-we-were-hoping-for
You forgot to mention that the review is on Gizmodo. You remember them - they're the guys who bought the stolen iPhone 4 prototype. Yeah, I trust their reviews to be fair and unbiased.
Exactly, I do like Lifehacker but they have never really been that positive on Mac and as for Gizmodo, what did you expect? A positive review of something from Apple? Not going to happen.
Get there direct:
http://gizmodo.com/5819418/mac-os-x-lion-this-is-not-the-future-we-were-hoping-for
Looks to me like a sober appreciation rather than an attack.
Lion might be the "Vista" of the OSX line, but it still hasn't been released yet. I doubt they will, but hopefully Apple delays its release (like they used to do way back when) to try and resolve the issues that everyone's talking about.
Only if they had some type of Developers club where you could sign up and test new OSX software before it hits the mainstream...
First, you have to pay a yearly fee. Second, it's restrictive in a lot of directions. Third, having access to every build is different than making limited public betas available.
There is. https://appleseed.apple.com/cgi-bin/WebObjects/SeedPortal
It's by invitation only.
Interesting, I hadn't heard of AppleSeed before. I'll have to update the article. I guess the only criticism I can make of it is that it clearly isn't working perfectly if it was in place all this time.
For those who also haven't heard of it, Ars Technica had an article about it a couple of months ago.
http://arstechnica.com/apple/news/2011/04/appleseed-gives-regular-joes-pre-release-access-to-mac-os-x-lion.ars
It's also a long way from what I proposed, which is a system built into Software Update and with feedback and reversion capabilities.
Clearly the current system isn't working or we wouldn't have had the problems with 10.6.7 and 10.6.8.
Just because Lion is almost out the door doesn't mean that Snow Leopard will be a dead OS.
There will surely be one Snow Leopard update if not two before it's retired. Is Leopard retired? I don't think so, but I'm not sure on that one.
Yes, there may be a 10.6.9, though we may also just see security updates for Snow Leopard, possibly along with specific updates like the Snow Leopard Font Update that fixed 10.6.7.
Giving everyone access to beta builds of OSX would result in a support nightmare for Apple. Most developers can recover from bad system crashes or corrupted data and permissions problems that result from using unfinished software. Developers have dedicated machines for installing and testing.
If someone really wants to test beta builds they can invest $99 and buy a developer account. As it is,
developers file a great many bugs most of which are duplicates of existing internal bugs already filed. Then the issue of the bug writing itself. All too often a bug is entered as;" I opened iTunes and it crashed. There was some kind of error on the screen but I forgot to write it down. I tried again and it worked."
at least that is what it was like 10 years ago when I was there....
BTW: tidbits is one of the all time great resources for Apple users. Thanks for being here for all this time.
Glad you're liking TidBITS!
I don't think it would be a support nightmare because it would be self-selected for people who would be expecting that the betas might cause problems, and there would be a reversion option should an update cause any problems.
Also note that I'm not suggesting every build be released this way - just a public beta along the traditional definition: in other words, when Apple believes it's ready.
great article! It would be especially useful to enable snapshots in Time Machine so that system can be restored to previous points as per windows, and to make it an option before any update.
Perhaps one day we'll get a really advanced file system like ZFS or BTRFS :)
I am not a developer but I have been using Lion Os developers review 4 since it has been released,( p2p download file ) It is nice and easy to use and doesn't have any major bugs at least for me !!! the only problem I have with it is, the safari . backward and forward trackpad gesture is not working as it is suppose to be.
hope it works with the official version.
There is an informal beta testing program. It called "early adoption". Anyone who has been using computers for a while should know that the early adopters do the beta testing for the rest of us.
Well, you have a point, except that people who don't pay attention to the news, and these are the people who should be waiting the longest, are getting the update pushed to them via Software Update. And they have no idea how old it is at that point, so they're likely to just install when Software Update suggests it.
I say no Public Betas. Apple just needs to tweek it's internal testing process and segment it's testing teams in a more comprehensive and thorough way.
I firmly believe that apple's "it just works" motto died with the introduction of the iPhone. A new OS was delayed for 6 months and everything Mac went downhill frow there.
I don't know why. Perhaps the same people work on OS and IOS or perhaps the developers at Apple aren't what they used to be. Whatever the reason Mac software ain't what it used to be in terms of reliability. It's better than the competition but that doesn't say much
These are good suggestions, Adam, and seem potentially workable. I agree with (what I enterpret the) sentiment behind the article -- that Apple has really lost sight of shipping reliable software to its Macintosh user base.
Over the last few years, during which time the iDevices have taken ascendancy, it seems Apple has decided that the Mac computer and the Mac OS X are just not the thing for them to focus anything more than minimum attention on.
I find it both sad and discouraging, especially as I attempt to support and encourage myself and my Mac clients. It's tough to explain why an official Mac OS or other Apple software update introduces new problems for the users. The folks who just want it to work.
Here's to hoping Apple either allows some kind of wider beta testing, or at the very least seriously improves their internal quality control and testing before release. Especially for releases they say are "required" for future software updates (as: 10.6.8 required for Lion).
As a commenter or two have stated, this exists today. The Appleseed program. Granted, you can not request to join, or show interest, or invite your friends, just hope you are selected somehow by Apple to participate, but it is just every day users.
Yep, looks like a start. But I can't tell if it has the various other mechanisms I suggest, such as reversion and built-in feedback in an interface that works better than Bug Reporter.
Adam,
A reversioner comes with the download of the seed as well as a feedback mechanism. It's called Feedback Assistant and it works very well. I feel that your concerns are not well founded.
I was browsing headline/summaries on the iGoogle Sci/Tech page I use and saw an article that Apple should use public betas. Thinking I had read the article, I checked the link in the summary. It was to this article!
Sorry to say, I believe Apple has become preoccupied with selling istuff. They have forgotten their roots, namely the small business that put them on the map.
This is a ridiculous notion, OS X is a pay to own OS, not LINUX. The JUST released the Gold Master to developers (which is essentially the paid version of the OS). Yes, Apple is in the hardware business, but they're in the software business, too. Let's see Microsoft offer a major release of Windows for $29. NOT GONNA HAPPEN
I just tweeted my support of the idea. Adam wrote an excellent thoughtful article and his ideas about reverting to previous version, cooperative with Time Machine are perfect. I think Apple would be well served by such a program and its wider audience. Then when there is a release it won't have so many negative issues. This reinforces my idea to do some Public Beta releases of TopXNotes 2.0. Thanks Adam for the great ideas.
Adam C. Engst wrote:
> Apple should reward users who consistently submit helpful and accurate bug reports.
For the record, I have filed a total of 2212 problem reports via Apple Bug Reporter since 2004. No offer of reward from Apple so far...
Heh. The real question is how many of them remain completely unaddressed by Apple, which is true of most of mine.
Today, i received 27 messages from Apple, all saying "Please verify this issue in Mac OS X Lion GM Build 11A511 and update this report with your results." So, it is possible that they fixed these 27 bugs that I had reported. However, I am not willing to upgrade to Lion until the Quicken incompatibility is resolved somehow...