Crash Reporter Roundup
July 28th, 2009Crashes suck. When an application experiences a crashing bug, it’s likely to stop running and take with it any unsaved work which you may have had open. Generally speaking, developers who take pride in their code also take pride in ensuring that it is resistant to crashing.
Unfortunately, it can be difficult for developers to know just how crash-prone an application is, because many users are not proactive about sending in bug reports when an application crashes. Who can blame them? They just had to suffer the indignity of a product gone completely wild, and now you want to be generous with their time and help you out?
Apple recognizes this, which is why they make it incredibly easy for users to report crashes on the Mac. When an application goes down in flames, a dialog appears offering to send the details of the crash directly to the company. This is great for Apple, but relatively useless for developers. You see, although Apple is happy to collect information about every crashing application on your Mac, they don’t share the information with the developers of the product in question.
Yeah, it sucks.
Rolling Our Own
It’s obvious that Apple should be helping developers by sharing this information. It’s good for us and it’s good for the platform. It’s good for Apple. A stable environment is so important to perceived quality, that it took less than a year of supporting 3rd party software on the iPhone before they decided to share crash information with iPhone application developers.
Who knows, we may get lucky. Apple’s generosity with this information may take a turn for the better at any moment. But for the time being if you want to have any sense of the stability of your application in the wild, you’re going to have to arrange an easy way for users to send the crash information directly to you.
I recently decided to bite this bullet, and I discovered that while there is some really useful open source code out there to help achieve this goal, the pluses and minuses of each package are not well documented. I hope this roundup of several options helps direct your search for the most appropriate solution in your product.
UKCrashReporter is a popular option that is proudly low-tech in its operation. It relies on finding crash logs in the user’s home directory. If they’re there, UKCrashReporter asks the user if it may transmit the crash information to your web server via an HTTP POST request.
SFBCrashReporter and CMCrashReporter are each quite similar to UKCrashReporter, detecting crash logs at application launch time, and offering to submit them via HTTP POST to a URL of your choosing.
FeedbackReporter also relies on launch detection of crash logs, but offers to report uncaught Cocoa exceptions in your application right as they occur. While other solutions listed here are focused tightly on crashes, FeedbackReporter also provides a general-purpose feedback interface, which users could invoke from a menu item in your application for general, non-crashing feedback.
HDCrashReporter adopts the same approach as UKCrashReporter, but opts to deliver the crash information by email instead of HTTP request. It uses Apple’s NSMailDelivery class, or if that fails, attempts to open a mailto: URL, engaging the user with their favorite email client.
ILCrashReporter and its branch ILCrashReporter-NG tackle a shortcoming of all the choices listed so far: that a user will not be presented the option of submitting the crash log until and unless they launch the application again. To be sure that users immediately have the opportunity, ILCrashReporter starts a second watchdog process, that sits around waiting for your application to terminate unexpectedly. If it does, the crash information is gathered up and sent, like HDCrashReporter, by email. Unlike HDCrashReporter, the email is delivered by connecting directly to an SMTP server either deduced from the user’s settings, or else explicitly provided by you.
plcrashreporter is one of the newest additions to the bunch, and takes a decidely different approach in that it ignores Apple’s own crash logs, opting instead to scrutinize the state of memory and write its own, custom-formatted crash log to disk. It installs custom signal and exception handlers, and includes code that has an in-depth understanding of how to examine memory on different platforms including iPhone, Mac i386, and Mac PPC. Although the crashreporter is active at the moment of the crash, it requires an approach such as that of UKCrashReporter or HDCrashReporter, of detecting and then submitting the crash informaton on the next launch of the application.
Breakpad is the crash reporting facility used by Mozilla, and is unique among solutions here in that it is cross-platform, supporting Mac OS X, Windows, and Linux. Crashes and exceptions are detected as they occur, spawning a separate crash-reporter application that offers to submit crash information via an HTTP POST. Also unlike other solutions, the project includes server-side software for collecting crash information and for browsing it via a web interface.
Unsanity’s Smart Crash Reports takes a more vigilante approach, installing software that gets loaded into every application on the system. By hijacking the standard crash report dialog, SCR is able to repurpose it as a developer-friendly dialog that offers to send the report to both Apple and the application’s developer. Developers can choose to have crash information sent by email or by an HTTP POST to your server. Of all the solutions listed here it is probably the most fragile because of the way it hijacks Apple’s UI. It also requires installing software that has an impact outside of your application, which may be more than many developers are willing to ask of their users.
Great! So Which One Is Best?
If you want crash reporting in your app you’ll need to decide from among the above solutions, or starting your own solution from scratch. The problem of course, is none of the solutions listed is the obvious best choice. It depends on your own particular priorities and how you answer many of the questions at hand, including:
- How invasive/fragile should the solution be?
- Report crashes immediately, or after the next application launch?
- What platforms and architectures must be supported?
- What kind of user interface should be presented to users?
- Should reports be delivered by email, HTTP, or some other means?
Hopefully the capsule reviews above give you some direction in evaluating the options that are available, and making an appropriate choice. Of course, after evaluating and discarding the options here, I’m sure some of you will decide to hunker down and add another project to the list. Please let me know if you do.
The Best Solution Of All
The list of solutions above was gathered in one afternoon (with the help of my Twitter followers!), and I’m sure it’s not exhaustive. What the insane variety of choices tells me first and foremost is that Apple could end a lot of suffering if they just shared their dang crash reports. Seriously, look that this list. The number of choices and variety of implementation details is out of control. And these are only some open source solutions. I’ve talked to a number of colleagues who have implemented similar plans from scratch for their own purposes.
Apple: stop the madness! Free the crash reports and spare us agony of reinventing this wheel again, and again, and again. Good for us, good for the platform, good for you. Thanks.
July 28th, 2009 at 10:07 pm
If there’s madness to be stopped, I think it’s the AppStore first, and crash reports some time there after. ;-) I wish they’d just allow a Info.plist entry that directs the crash text to a dev specified URL.
July 28th, 2009 at 10:09 pm
Note that you can support Smart Crash Reports without installing it by getting an ID from Unsanity and putting it in your app’s Info.plist. If a user happens to already have Smart Crash Reports installed — and a decent number of users do — you’ll get the crash report without having to ship any code.
That being said, Apple: stop the madness!
July 29th, 2009 at 5:07 am
Just a note that UKCrashReporter also allows sending general-purpose feedback (orderFrontBugReportWindow: in UKFeedbackProvider class).
Breakpad is also used (and developed) by Google, however it seems very complicated, as it is cross-platform.
July 29th, 2009 at 11:45 am
*Microsoft* shares their crash reports if you register with them…
July 29th, 2009 at 7:30 pm
I use Uli’s UKCrashReporter in conjunction with Wolf’s JRFeedbackProvide. It works great; I get the reports by email, and the users get a chance to tell me what they were doing just before the crash.
July 30th, 2009 at 1:53 am
Dmitry, Breakpad recently added a framework on OS X that abstracts away most of the complexity (on the client side, at least). Adding it to Camino was incredibly simple.
August 8th, 2009 at 2:08 am
Holy cow, Daniel, what great timing. I’m just about to look at building a crash reporter into my app before releasing my first Alpha. How strange you should post such a well timed article. Now I have the full info to make my choice.
Thanks!
September 2nd, 2009 at 5:26 pm
Strangely, I just noticed a crash on Snow Leopard of Google’s Quick Search Box. The resulting dialog said the report would go to Google! (Not Apple.)
So I’m going on a hunt. I wonder if Apple did add some features to the reporter… I’ll let you know.
September 2nd, 2009 at 5:34 pm
Ah. No, I was wrong.
Google Quick Search Box includes Breakpad. So that’s another vote in favor of Breakpad, then.
So I’ll file another vote for Apple to add support for open crash reporting. Do they really want reports for X thousand 3rd party apps?
September 3rd, 2009 at 7:42 am
So which one did you choose Daniel?
September 3rd, 2009 at 11:28 am
Hi Gleb- I ended up working from ILCrashReporter-NG, hacking a bunch of stuff out that I don’t need.
The challenge is really finding the feature set that you do want, with none of the liabilities for features you don’t want (e.g. library dependencies, “hacks”, etc.).