Skip to content
Thoughtful, detailed coverage of everything Apple for 33 years
and the TidBITS Content Network for Apple professionals
41 comments

How to Report Bugs to Apple So They Get Fixed

WWDC has come and gone and left in its wake a slew of new operating system betas—macOS 11 Big Sur, iOS 14, iPadOS 14, watchOS 7, and tvOS 14.

If you’re feeling brave and want to donate some of your time to helping Apple improve these operating systems for everyone, you can test these betas, either immediately if you’re in the Apple Developer program, or shortly afterward, once Apple releases the public betas.  Once you install the beta (not on your primary device, of course, because that would be foolhardy!), you’ll promptly run into bugs. That’s why it’s a beta release, after all. If it was ready, it would be shipping.

It’s easy to feel like Apple releases these betas mostly for PR reasons, but that’s far from the truth. Apple does care about receiving bug reports from developers and users alike. But, as you can imagine, hundreds of thousands of developers reporting bugs can overwhelm even Apple’s resources.

How do you report bugs so Apple actually fixes them?

Use Feedback Assistant

Apple solicits bug reports from users and third-party developers via Feedback Assistant, which is both a pre-installed app and a Web site. I recommend using the app to report new bugs. It guides you through specific questions based on the type of bug you’re reporting, automatically generates a sysdiagnose file, and prompts you for logs and reports relevant to that type of bug. The iOS version of Feedback Assistant can also report bugs on connected Apple TV, Apple Watch, and HomePod devices.

The Feedback Assistant app is installed in macOS 10.15 Catalina and later, and in iOS 12.4 and later. On beta operating systems, the Feedback Assistant app appears on the Dock (macOS) or the Home screen (iOS). If you don’t want it in the Dock on the Mac, you can find Feedback Assistant in /System/Library/CoreServices/Applications/Feedback Assistant. It’s often easiest to launch it with Spotlight. You can also launch Feedback Assistant from Safari on any device by navigating to applefeedback://

Feedback Assistant

Every Feedback Assistant bug report is reviewed by someone from Apple Developer Support. That person may answer easy problems themselves or kick the report back if it’s obviously missing key information. If your bug report looks good, they copy it from Feedback Assistant into Radar, Apple’s internal bug tracking system. Radar then sends the report to the appropriate engineering team, where it may or may not be fixed, or even read.

Apple has published an introduction to reporting bugs that’s worth a look, but read on and I’ll tell you how to maximize your chances of getting your bug resolved.

Write Good Bug Reports

Between WWDC in June and the iPhone launch in September, Apple operating system engineers work 12-hour days, six or even seven days a week. They’re finishing up features that their boss told the next boss up were already done (oops!). They’re fixing bugs reported by Apple Quality Assurance (QA) engineers. They drop everything to fix the occasional bug reported by an Apple vice president. (Yes, Apple management uses pre-release products, and they report bugs too.) After all that is done, they’re also supposed to fix bugs reported by third-party developers and users of the public betas.

When I was an Apple software engineer, this was my job. I’d get dozens of new external bug reports daily, and I had time to address only some of them. Many developers wrote great bug reports, but some looked like they were written by a third-grader who skipped their ADHD medication. You can guess which ones got more attention. Here’s what you need to do to write a good bug report. Some of these are specific to developers, but most apply to everyone:

  • Craft a descriptive title. A good bug report starts with a good title. Make the title clear and concise. Bug meetings are long, grueling affairs in which engineers and product managers prioritize hundreds of new bugs. Bugs are listed by title, so your bug’s title is just one in a very long list. A descriptive title makes it easier for engineers to know what it covers without having to open the bug report and read the description. It’s all too likely that bug reports with generic or too-short titles will get passed over with a vague promise to come back later.
  • Describe the problem. Explain what actually happens, and what you think should happen. Don’t assume that it’s obvious what’s wrong. If Apple developer documentation describes what should happen, link to it. Don’t assume I know all the Apple docs by heart.
  • Include clear steps for reproducing the bug. Don’t forget to list any initial conditions, and don’t skip any steps. If the bug doesn’t happen every time, note how frequently it occurs.
  • Be concise. I have a stack of bugs to fix after yours. Please use my time efficiently. The more easily I can reproduce your bug, the more likely I am to fix it.
  • Include screenshots or video. If possible, include a screenshot or screen recording showing how to reproduce the bug, including the bug actually happening. A picture makes the problem obvious. It also makes it harder for an engineer to ignore the problem. Out of sight, out of mind.
  • Clarify the context. Does the bug occur on the device, on the simulator, or on both? Does it occur in both the debug build and release build? Have you tried different device models? Include all this information.
  • Include code. If I’ll need code to see the bug, include code. At a minimum, include a few lines of code that demonstrate the bug. If at all possible, attach a sample Xcode project that demonstrates the bug. Don’t attach your entire app; I don’t want to wade through it. Create a new, empty app in Xcode and add just enough code to show the bug. Add a big, obvious comment saying, “The bug happens here!” so I don’t have to look through all the source files. Attaching an Xcode project has one more advantage, which is that it includes all your build settings. Some bugs happen only with certain build settings.
  • Specify build numbers. If you discuss operating system versions, include the build number (such as 17E262) in addition to the operating system version. (To find the build number in macOS, choose About This Mac from the Apple menu and then click the macOS version number. In iOS, go to Settings > General > About and tap the version number.) Inside Apple, operating system builds are identified by build number. Apple engineers don’t always know offhand which build number corresponds to a particular external beta or release. They can look it up, of course, but anything that saves them time makes it more likely your bug gets fixed.
    Build numbers in macOS and iOS
  • Include sysdiagnose and logs if relevant. If you use the Feedback Assistant app, it will automatically include a sysdiagnose and prompt you for logs relevant to that type of bug. A sysdiagnose is a report that includes a metric shipload of information about what’s running in your system. It’s extremely useful for those who know how to read it. If you use the Feedback Assistant Web site, you’ll have to generate a sysdiagnose by hand, which isn’t hard. Be sure to attach any relevant console logs or other logs that might contain relevant information. Apple provides instructions for generating a sysdiagnose, along with all kinds of specialized logs for specific types of bugs.
  • Specify the Xcode version. Each operating system beta release typically includes a corresponding Xcode beta release, so be sure to use the latest Xcode. The bug could be in one of the frameworks included in Xcode, not in the operating system itself. You don’t want to report a bug that has already been fixed because you tested it with an old version of Xcode.

Report Bugs Promptly

Apple creates a new internal operating system build each night, called the daily build. Apple engineers install the daily build each morning. Some daily builds are fairly stable; others are not. Every few weeks between WWDC and shipping, a stable build becomes a beta candidate. It gets additional testing to ensure all aspects seem stable. If approved, it gets a beta number and is released to developers and users participating in the public beta.

By the time the beta is posted, Apple engineers already consider it old since they’ve installed weeks of daily builds in the meantime. By the time you download it, test it, file a bug report, your bug report is copied from Feedback Assistant to Radar, and an Apple engineer looks at it, they think that beta is positively antediluvian. The longer it takes you to report a bug, the harder it is for engineers to recreate the situation you’re seeing.

So when a new beta comes out, test it and file bug reports immediately. Make your bug report as current as possible when an Apple engineer sees it.

Check Each Subsequent Operating System Release

It would be nice if Apple informed you if your bug was fixed. It would be nice if the tooth fairy was real, too. Apple receives dozens, if not hundreds, of bug reports about each issue from outside the company. Internal Apple engineers may have already written up the bug as well. One bug report, usually the one with the clearest explanation of the problem, is designated the original. The others are marked as duplicates, or dups, and at least in theory, all bug reporters will be notified when the bug is fixed. If your description doesn’t closely match the description in the original, it may not get detected as a dup, and you may not be notified when the bug is fixed.

As a result, for each newly released beta, you’ll want to test to see if your bug is fixed. If not, consider writing a new bug report against the new release. That brings the bug back to Apple’s attention and lets the engineers know it wasn’t fixed in the latest update.

Even if you think your bug report will be marked duplicate, report it anyway. Think of your report as a vote for that bug to be fixed. When engineers are sitting in bug meetings and deciding which bugs to prioritize, a bug with 100 dups gets more consideration than a bug with 3 dups because it’s obviously affecting a lot more people.

If your bug is fixed, it won’t be marked fixed in Feedback Assistant until the operating system version with the fix ships, whether that’s a beta or the final release. Apple doesn’t like to announce fixes until they ship.

Developers, if a bug is blocking a key feature of your product and doesn’t seem to be getting fixed, email Developer Support and ask if there’s a status update, or if they have any suggestions for a workaround. Wheels sometimes need to squeak to get the grease.

If you have a friend who works in Apple engineering, you might ask them to look up the bug in Radar, to see if it looks like it’s going to be fixed. There are several things that could prevent this. Your bug may not have been copied from Feedback Assistant to Radar yet. Not all engineers can view all Radar bugs for security reasons. And your friend may not feel comfortable doing this for you. If they do, they’ll probably give you general information, like “it’s on track to be fixed in a coming release.” Even knowing this can be useful. Don’t ask your friend to pressure Apple to fix the bug. Chances are that your friend doesn’t work in the department that owns the bug and doesn’t know the engineer it’s assigned to. Apple employs thousands of engineers. They’re not supposed to discuss bug status with non-employees, so don’t put your friend in an awkward spot.

One final note. Feedback Assistant and Radar use separate bug numbering systems. Apple engineers can translate one to the other. Radar is not an acronym. It has been Apple’s bug reporting system for 30 years. Developers used to have a Web portal directly into the Radar database called RadarWeb, though they were restricted to the bugs they reported. Feedback Assistant replaced RadarWeb, and it provides a separate database for tracking bugs reported by third-party developers and users.

Respond to Apple Quickly

Sometimes an Apple engineer will ask for more details or logs. Try to short-circuit this by including any relevant logs with the initial bug report. Occasionally Apple asks for specific additional data or sends a profile that will generate a special log file. Respond quickly. The longer the bug sits, the more likely the engineer will move on to another task. Check your open bugs regularly.

Sometimes it seems like an Apple engineer asks for more info just to stall. When I worked there, I saw that tactic deployed against bugs written by other Apple engineers as well as third-party developers. An engineer asks for some plausible additional information, which takes time to collect. Half the developers never respond, and the engineer can then ignore those bugs, as they’re “waiting for additional information.”

Other Bug Report Sources

Apple’s Crash Reporter automatically detects and reports crashing bugs. Teams are rated on how many active crashes are in their code. Apple has a good system, both technically and procedurally, for fixing crashing bugs. That’s not to say you shouldn’t report crashing bugs—bug reports can still increase the urgency of the fix.

When reporting crashing bugs or kernel panics, hardware bugs, or printing problems on the Mac, you’ll need to include a System Information report. Use the System Information app in the Utilities folder to generate this report (just choose File > Save).

Apple engineers do pay attention to some developer forums, blogs, and podcasts. Complaints that pop up repeatedly may be written up as bugs in Radar. I’ve seen text from a developer post copied into a Radar bug report. But Apple engineers rarely post in developer forums because the company discourages it. Or at least that used to be true: Apple’s redesigned developer forums encourage developer participation (see “Apple Announces WWDC 2020 Schedule,” 11 June 2020). Although it’s rarely acknowledged, Apple engineering does track developer opinions.

Understand Bug Priority

Apple assigns all bugs a priority, from P1 to P5. P1 is a crashing or data loss bug, and such bugs generally must be fixed. P2 is a feature that doesn’t work, and Apple almost always fixes such problems. P3 is a feature that doesn’t work correctly; Apple likes to fix P3 bugs, but some get postponed. P4 is a user interface issue or other minor bug, but such bugs often get postponed. P5 is an enhancement request; such requests tend to get ignored.

Some Apple engineers just fix the bugs they’re assigned, working from high priority to low priority. Often they never make it to the low priority bugs. But more experienced Apple engineers also cherry-pick some lower priority bugs that they know will annoy users and make sure they get fixed. Apple engineers use Apple products too, and want them to work well. Some departments even give out prizes to the engineer who fixes the most bugs.

When the ship date is far away, Apple engineers are allowed to check in pretty much whatever code they like. As we get closer to alpha, beta, and eventually golden master, there are more constraints. First, only P1 or P2 bugs can be fixed. Then only P1 bugs from an approved list. Each bug and fix is rated, for the seriousness of the bug, and the risk of the fix. No one wants a high-risk fix for a minor bug. As release nears, even serious bugs can be postponed. Having a bad, but rare, bug with well-understood characteristics is better than a poorly understood fix with insufficient testing time. The fix will get added to the first patch release, after more rigorous testing.

Apple’s Development Cycle

To maximize the chance of getting your bugs fixed, it helps to understand Apple’s development cycle. After the major X.0 releases of iOS and macOS ship, usually in September, development work starts on the next year’s X+1 releases for each. Development continues through the end of the year, and by February or March, QA starts seriously testing builds and reporting bugs. Over time, the development version becomes more stable, going from barely usable to not that bad. By May, the daily build is starting to look decent. Then comes the rush to put together the WWDC beta. All those unfinished features and ignored bugs suddenly get more attention.

Beta 1 at WWDC may look pretty rough when you first install it (on a test device!), but it still represents a ton of work. That’s when you want to start reporting bugs! Apple engineers have been assigned to read bug reports and fix problems. Take advantage of the attention and report any bugs you find.

More beta releases will come in July and August. Test each one, to see if your bugs are fixed, and look for regressions, which are previously working features that broke. Remember, if a bug you reported in a previous beta still isn’t fixed, don’t be afraid to report it again.

As September approaches, the betas become more stable, but Apple also makes fewer changes. If you found a minor bug that isn’t fixed by August, it will probably still be there in the golden master. Report it anyway, and Apple will hopefully fix it in one of the early patch releases.

The new operating system usually ships in September as version X.0.0. The patch releases follow immediately. X.0.1 and sometimes X.0.2 are planned even before X.0.0 ships. X.0.1 has a few important bug fixes that didn’t make it into the X.0.0 release. X.0.2 has less vital bug fixes that were too late for X.0.0. In a bad bug year, there may even be an X.0.3 release.

Once those first patch releases are out of the way, Apple engineers start working on the next major release X+1. They don’t have much bandwidth to fix bugs in the currently shipping operating system anymore. Continue to report bugs, but the likelihood they’ll get fixed drops considerably.

(As an aside, Apple teams are usually divided by area or feature, not by main release versus patch release. If you’re a power management engineer, for instance, you’ll work on that feature in both the initial X.0 release and all patch releases. It works poorly to have one engineer write the initial code and another engineer come along later and fix bugs because the second engineer isn’t intimately familiar with the code, why it was designed that way, what ideas were tried and discarded, and so on. It’s better to have an engineer own a particular piece of code, including all releases.)

There will probably be an X.1 and X.2 release, but those typically introduce new features, like iCloud folder sharing and battery health management in Catalina. These releases will have some bug fixes, but that’s not their primary mission. It never hurts to report bugs during this time, but unless they’re particularly bad, don’t expect them to be addressed until the next major release at the earliest.

In the end, if you want Apple to fix the bugs you find (and who doesn’t?), follow these simple rules. Write clear, complete bug reports. Include screenshots, video, and sample code, if possible. Report bugs as early as possible. Work within Apple’s development cycle.

And most important, keep sending in your reports. Bugs that aren’t reported don’t get fixed.

Subscribe today so you don’t miss any TidBITS articles!

Every week you’ll get tech tips, in-depth reviews, and insightful news analysis for discerning Apple users. For over 33 years, we’ve published professional, member-supported tech journalism that makes you smarter.

Registration confirmation will be emailed to you.

This site is protected by reCAPTCHA. The Google Privacy Policy and Terms of Service apply.

Comments About How to Report Bugs to Apple So They Get Fixed

Notable Replies

  1. FWIW The Feedback Assistant app is also installed in versions of macOS going at least as far back as Sierra (10.12.6) in /System/Library/CoreServices/Applications. You can find or start it the same way the article outlined for macOS Catalina.

  2. Perhaps I need to point out that it’s highly likely that any feedback submitted in this manner on a macOS version that is not currently under beta testing will be ignored. You really need to be formally enrolled in one of the beta test programs and pay close attention to David’s recommendations on preparing a quality report. Also, reports from participants in the public beta are generally considered in bulk rather than individually and submissions will rarely receive replies. That’s one reason for the recommendation that everyone seeing an issue needs to report it for maximum effectiveness.

  3. Wow. So Al, are you actually saying if somebody is on the latest shipping macOS (so 10.15.5 right now) bug reports they send are basically ignored?

  4. It’s amazing that no one with any Catalina beta ever used Mail!

  5. I was replying to Gary about using Feedback Assistant for Sierra and above. It’s clear to me that. bug reports about an unsupported Sierra issue will be ignored and probably anything other than a Security issue found in High Sierra or Mojave, since neither of those are patching regular bugs any more.

    Regarding 10.15.5 specifically, I doubt that the use of Feedback Assistant by a non-beta tester will accomplish anything, but if a currently enrolled beta tester were to submit a 10.15.5 problem report, it would most likely be rolled into consideration for either10.15.6 (should there be one) or 10.16 (or whatever number turns out to be the next major release).

  6. I have to guess that there were more issues raised concerning Mail during beta testing than all the complaints that have been registered after each public release.

  7. Thanks, Al. Good to know. If Apple tends to ignore anything that comes from non-dev non-beta users, I do wonder why they solicit it in the first place. Besides Feedback Assistant, there’s also https://www.apple.com/feedback

  8. Sorry if I wasn’t clear, but I specifically said “non-beta testers” not “not-dev users” if I understand what you mean by that term. All beta feedback is considered, some more intensely than others. Developers and enterprise IT admins have narrower focuses in what is important to their particular needs vs. public beta testers who are rather random and hopefully representative of the average users needs.

    But you are correct, for all other user that are not involved in beta testing Apple has provided the feedback site.

  9. So are you saying regular users should use the feedback site while only beta testers should be using Feedback Assistant?

  10. I’m not even sure that regular users can use Feedback Assistant, since you have to use your AppleID which only gives you access send report on the OS’ that you’ve signed onto.

  11. Ken

    I’m signed up to the public beta program and that gives me access to Feedback assistant. I assume that Apple has tools that make it easy to identify similar bugs and they all get added togetherWhen MacOS X was under development and I was developing software I put a few reports in. Never heard back, so I don’t know if they had an effect, but the bugs disappeared.

    With Mail I seem to have lost everything from before I upgraded. It is still there, as I have copied the folders, and then I search using BBEdit. Rebuilding doesn’t work. I don’t know how anyone could get it that wrong.

  12. Sorry, but I’m even more confused now. I was able to use Feedback Assistant to report a bug in the current shipping 10.15.5. I’m not in any beta program. Did I just send a bunch of information to /dev/null? Should I instead have used the feedback website?

  13. I wish I could answer that for you, but I have no idea how or if those are being handled nor even who at Apple would be able to answer, but thanks for verifying that a non-beta tester is able to at least prepare and send reports. I know that the various groups of beta test submissions are routed differently to the engineers responsible for managing them. As mentioned before, they are sorted and transcribed into a “radar” report (at least that was the system before the latest reorganization) so that the appropriate engineering section can take responsibility for it. You might get some feedback as to the status of your individual reports and how many similar reports have been submitted. You might get a request for additional information. That would tend to indicate that your reports are being handled. Just be aware that most submissions, even by beta testers, are never replied to.

  14. Your experience isn’t unique. I don’t recall that I have ever heard that a public beta program participant ever heard anything back. They do get aggregated with similar reports, so I would suspect that they pay closer attention to the ones with the most reports. That doesn’t always insure that they will get fixed as they may be too hard or just receive a lower priority.

  15. Thanks, Al. Appreciate the background. My Feedback Assistant lists recent similar reports as none and ‘resolution’ as none which I believe is the same it said right after I filed. So from all I can tell nothing has happened (so far).

  16. Actually Al, I was just making a grammar nerd correction of the statement in the original article: “The Feedback Assistant app is installed in macOS 10.15 Catalina and later.” The implication of that statement, especially to someone who is new to macOS, is that it is an app that was introduced by macOS Catalina. I was merely pointing out that it has existed for quite some time. Whether or not it has any utility in previous versions of macOS was not stated nor meant to be implied. Hence the use of FWIW (For what it’s worth) at the beginning of my comment.

    Having said that, I really appreciated your detailed and insightful explanations throughout the rest of the thread to date. Thank you for that background!

  17. Great article. Thanks for sharing it.

    I have a question on the process when I report a bug in a third party app running on a OS X public beta. One of the categories in Feedback Assistant is to report third party app bugs.

    Am I wasting my time to report a bug in a third party app using Feedback Assistant?

    Most developers it seems do not ever load a Mac OS beta. So if the beta OS causes bugs to appear in their app, the developers don’t care.

    They assume the Mac OS bug will get fixed or the OS feature causing the bug in their app may not even appear in the Golden Master. So they don’t start to triage their app bugs until the crunch right before public release.

    In other words, App Developers seem to only deal with bugs in the current Apple public OS release. In other words, developers are behind by a release.

    So what happens to those Feedback Assistant reports that report issues that occur in a specific third party app while running a Mac OS Public Beta?

    What happens to Crash Reports that relate to a crash in a third party app that are sent to Apple? If the app bug caused a Crash Report should I also report it in Feedback Assistant?

    If a third party app crashes repeatedly, generating multiple crash reports, does Apple in any way work with the developer to fix the problem? Does Apple ever apply subtle pressure on App Developers to fix bugs in their app? Or are the Crash Reports simply forwarded to the app developer and forgotten?

    I’m not a developer so I do not understand the process of third party app development and how closely Apple Engineers work with the Third Party App engineers.

  18. The resolution of a bug also seems to depend on the “Which area are you seeing an issue with?” reported with the bug.
    I have seen the same issue reported against a particular area get comments on Recent Similar Reports: Less than 10, while the same issue if reported against another area gets Recent Similar Reports: None.

    Why does this happen.

  19. This article was really useful… In my 30+ years of using Apple gear, I’ve never heard of this Feedback Assistant app. I always used Apple’s “Product Feedback” page which does not ask for any details. And who knows who reads the input from that link…

    Correction, I did get a request from an Apple engineer once regarding a bug I submitted using “Product Feedback”. He asked for a Quicktime video showing how the bug occurs. I sent that in and followed up later to see. He responded and said the issue was being reviewed. Nice.

  20. Feedback Assistant is relatively new. For years the Bug Reporter web site was how 3rd party developers to report bugs to Apple. There wasn’t really a single, coordinated way for users to report bugs. Apple does track calls to AppleCare and Genius Bar visits, to see what users are having trouble with.

  21. The categories you see as outsider reporting a bug are not the same categories Apple uses in Radar. Radar has thousands of categories and subcategories, too many for outsiders to wade through, and many for secret, unreleased products.

    If there’s no obvious category for the bug you’re seeing, you’ll pick one that seems close. The category in your bug report is mapped to a category in Radar, which is owned by an engineering team, which reads your bug report. But it might be mapped to the wrong team for that feature, there’s really no way for you to know. Hopefully the team that gets your bug will reassign it to the right team, but sometimes people get busy, and bug reports fall through the cracks.

    If it’s not fixed in the next beta release, don’t be afraid to write a new bug report, and perhaps try a different area, if that seems applicable.

  22. Unfortunately, I’d say reporting 3rd party bugs to Apple is probably a waste of time. I worked at Apple before Feedback Assistant, but AFAIK bugs reported in 3rd party apps were almost never passed on to those 3rd parties.

    For small companies, I’d try contacting them directly. Often there’s just a couple of tech support people, who can pass concise, reproducible bug reports on to their engineers.

    Crash reports are collected automatically by the OS, and are available for developers to read in their Apple Connect account. Many developers also use 3rd party crash reporting libraries.

    I don’t think that’s true, most developers I know definitely test on beta OS releases, they want to be ready when the new OS ships. But Apple won’t let them ship updates for a beta OS, or even mention it in their release notes, because betas are still officially unreleased software. That’s why when the new OS finally ships, there’s a crush of developers trying to ship updates.

    If it’s an important app like Photoshop, Apple will definitely alert them about problems. Sometimes Apple even changes the OS to work around the problem. But most 3rd party developers don’t get this personal attention.

  23. The purpose of this category is to determine whether or not the issue is due to a bug introduced by Apple in the beta or a problem that the developer needs to adapt to. If the former, then Apple should try to fix there bug. If it’s the result of something that Apple intentionally changed or deprecated, then they may get with the developer to let them know, but that’s a big MAY. I suspect that if the developer is a big enough partner such as Microsoft or Adobe then there will be some communication between the two, since the bigger developers generally are already running betas and they have corporate engineering representatives that routinely communicate such things. None of the small developers that I know have ever told me that they received such communications from Apple.

    As far as beta testers are concerned, the NDA does not allow those users to contact developers about issues they have with macOS betas, which is designed to give Apple the first opportunity to fix any issue that is their bug so that developers don’t waste time trying to fix something that will be resolved before macOS release. But I’ve seen that restriction widely violated by testers. I’ve also know Apple to occasionally direct testers to contact the developer about an issue.

  24. Thanks for the answer.

    Can you please describe what happens to CRASH REPORTS that are generated for all crashes, not just public beta testers?

    Does anyone actually read them! Are they routinely also coied to the developer? Are they tallied or countedsoan app that crashes 100 times gets moreattention that an app that crashes 5 times?

    Is all that information about the different threads and the list of instructions and what crrashed of practical use to the developer? They are gibberish to me.

  25. You might find this interesting:

    Technical Note TN2151: Understanding and Analyzing Application Crash Reports.

    Developers can get crash reports. As @das wrote, Apple collects them on a server. Developers are notified about new crash reports and can download them for analysis.

    When used in conjunction with an application’s source code (originally used to build the app) and its debug information (generated by the compiler alongside the app), the crash report includes a list of system exceptions (describing why the OS killed the app) and a stack trace for every thread in the application at the time it crashed. It also includes the contents of the CPU registers for every thread.

    This information is all critical information for any developer trying to identify and fix a non-trivial problem.

    The only thing more useful would be a full core dump containing the complete contents of the app’s memory at the time of the crash. Core dumps can allow the developer to perform post-mortem debugging, where they re-create the app’s (crashed) state in memory so debugging tools can be used to view it.

    Apple doesn’t provide core dumps with crash reports because they can be very big (potentially gigabytes of data) and there can be serious privacy issues (since they contain all of the app’s data, including stuff you may not want the developers to know about).

  26. In this case, your best bet is to call Apple support and work through them. Reporting bugs is only useful if you (and thus Apple) can reproduce the problem at will. Particularly in a situation like this, there are simply too many variables for an engineer to be able to track down the problem.

  27. Crash reports are generated automatically, if you agree to send analytics info to Apple when you set up your device.

    Crash reports are taken very seriously at Apple. Crash reports for Apple software is assigned to the team that owns that software. There’s a list of the top crashing bugs for each team, and they’re expected to fix them. Crashes that happen more often are ranked higher.

    As @Shamino said, crash reports for 3rd party apps are made available to developers in their developer account, it’s up to developers to read those and fix them.

    All that gibberish really is important info, and very helpful in tracking down a crash. The tech note @Shamino linked to has a good explanation of what’s in a crash report, if you want to learn about it.

  28. Although this is certainly true for regular users, Apple support (both AppleCare and Genius Bars) are instructed not to deal with beta OS issues. That includes troubleshoot a Mac with a beta OS installed. I’m sure there have been exceptions, but that is what has been put out.

  29. As a Catalina beta tester, I filed this bug. Apple said there were more than 10 similar reports.

    It has gotten a lot better, I only see the error occasionally so it might just be physical movement of the device.

    But it used to generate that Not Ejected Properly error repeatedly just sitting on the desk and not being used.

    My suggestion is to update to the latest release version of Catalina. Catalina is very stable at this point.

    Work has shifted to Big Sur although there was a release of another Catalina beta build for Mac OS and iOS 13 just last week.

    So maybe they are still working on some aspects of Catalina for a final public release. I am sure they want to make sure the transition to Big Sur is flawless.

    Sierra to High Sierra release (not beta) on V1 update bricked a number of 2012 MacBook Pros. Mine included. The 2012 MBP was compatible with everything in High Sierra except the new file structure.

    When the updater tried to rewrite the drive, the result was a bricked machine. Apple had to give those affected brand new free replacement machines direct from China in exchange for the bricked machine.

    And V1.1 of the updater was released within 72 hours. I would love to hear the true inside story of that screw-up and if anyone got fired over such an expensive mistake.

    I am holding off on testing Big Sur until the release notes are down to one or two lines. And I think I will load it on an external drive once I learn how. Because the beta build Apple sends doesn’t seem to allow choice of drive.

    Maybe boot from an external drive with Catalina and it will update the startup external drive to Big Sur?

  30. Howard Oakley just wrote this interesting piece basically encouraging most users to stop reporting bugs to Apple. He claims we’re enabling Bad Apple and urges us to stop in order to force the company to finally get serious about testing and QC.

  31. I saw that, and I can understand the frustration, but it doesn’t feel to me like Apple QA would then devote more resources to looking for bugs on their own. It would just mean fewer bugs reported and fewer bugs fixed, which doesn’t help anyone.

  32. Well his take on that is to instead devote time/effort to shame Apple publicly. That he claims is the one thing Apple reacts to and he believes that will eventually get their QA in line.

    The way I see it we have been unsuccessfully doing the whole testing and reporting thing for the past decade or so while we watched reliability steadily decline (“it just works” has started becoming a faint memory) and the cadence between update and update-to-fix-the-last-update became shorter and shorter. Well, if you keep doing the same thing but the outcome doesn’t improve, to me that’s usually a sign it’s time to change strategy. I therefore hesitate to dismiss what he says.

  33. I’ve done more than my share of shaming Apple publicly over the years, and my experience is that unless something is significant enough to get picked up across the Apple media ecosystem, it gets ignored. So that’s fine for something like the free space in the Big Sur installer problem, but it won’t work for bugs that are smaller or less serious. Which is not to say that reporting bugs works reliably either, but at least they’re in the system for Apple to fix, which won’t be true otherwise.

  34. I remember reading a column from decades ago where the writer mused about the perception that Apple released better quality software (as in fewer bugs) than Microsoft did. He noted that Apple didn’t release bug-free software or even software with fewer bugs, but it released software with the “right” bugs. Back then, Apple made the effort to track down and fix even minor bugs that would detract from the user’s experience. Nowadays, at least to me, that doesn’t seem to be the case.

    I think there needs to be a change in the internal culture of Apple before we’ll see any significant increase in quality. It doesn’t matter if we report bugs or not, if all that happens is Apple engineers look at a bug report and routinely decide it’s not worth spending the time to fix it.

  35. Alas, there’s no way to know if the grass really was greener in the past, given that we seldom hear about the bugs Apple is undoubtedly fixing behind the scenes all the time. And, of course, today’s systems are orders of magnitude more complex. :-)

    The iOS 14/Big Sur cycle of releases seems significantly more bug-free than the iOS 13/Catalina cycle, which isn’t surprising given how much flak the company took last year. It’s worth re-reading @das’s 2019 article on that topic and then returning to the one whose comment thread we’re in:

  36. Let’s not confuse the improvement from iOS 13->14/Catalina->Big Sur with the overarching decade-long trend towards reduced reliability and increasingly buggy behavior. “It just works” is no longer the motto. But many of us still remember the times when that was actually to large extent the case. To those who worked through both periods there is no confusing these two very different realities. And IMHO this is not the time for relativism if we expect to see things change for the better.

  37. Well, if a trend is ever to reverse, it has to start reversing at some point, which would seem to be the case with iOS 14/Big Sur in comparison with the previous releases. And while there’s no question that there are bugs, some of which I even notice and report, it’s extremely uncommon for any of them to actually prevent me from getting my work done. My Mac and iPhone and iPad and Apple Watch and Apple TV and AirPods do just work, and the bugs are mostly annoyances.

    Of course, lots of things in the wider world don’t always work the way I want, too. I just don’t get that exercised about things I can’t control.

  38. You’re right. I should perhaps lay the blame on the process rather than the culture. With the much more complex nature of our devices these days, there are simply more ways for a user to encounter buggy behavior. Even if the bugs per lines of code (or whatever is the appropriate metric) has remained constant over the years, the bugs-encountered-per-user rate seems to have gone up noticeably.

  39. If a reported bug is critical, can we expect it to be fixed in 10 days from being reported, and released in the just immediately next beta?
    Or does a bug get fixed and released only after around 2 months?

  40. You and I have absolutely no way of knowing. Apple doesn’t publish their procedures for things like this.

    We can assume, however, that they prioritize bugs based on how critical their engineers and managers consider them. The most critical bugs (security, bugs that affect system stability or data integrity) will get fixed pretty quckly - either as a part of the next system update or a special mid-update patch.

    Other bugs will get either rolled into the next scheduled update or may be put on hold until the next major OS version (e.g. if it changes the user interface).

Join the discussion in the TidBITS Discourse forum

Participants

Avatar for ace Avatar for Simon Avatar for alvarnell Avatar for schwartz Avatar for gmg Avatar for chirano Avatar for amcarter3 Avatar for fogcitynative Avatar for das Avatar for Shamino Avatar for ken10 Avatar for uandme72