Brent Simmons writes on the Anatomy Of A Feature, using his recent work in NetNewsWire to add support for the popular Instapaper service:

It’s tempting to think that adding a feature like this is just about adding the functionality — but there’s a bunch more to it than that.

Here you see the gory, deliberate details that a responsible developer must consider when adding even what seems like an incredibly “easy feature.”

Without a doubt, the part of my job that slows me down the most is exactly this kind of consideration. I want to add a new feature, but not until I have evaluated how the feature best fits in the structure of the application as it exists today.

Sometimes even an “easy feature” requires massive overhaul of the application, so that it doesn’t feel tacked-on or cheaply done. If the feature is important enough, it’s worth practically rewriting your application, just to add something that looks like it must have taken all of 10 minutes to do.

Crashes suck. When an application experiences a crashing bug, it’s likely to stop running and take with it any unsaved work which you may have had open. Generally speaking, developers who take pride in their code also take pride in ensuring that it is resistant to crashing.

Unfortunately, it can be difficult for developers to know just how crash-prone an application is, because many users are not proactive about sending in bug reports when an application crashes. Who can blame them? They just had to suffer the indignity of a product gone completely wild, and now you want to be generous with their time and help you out?

Apple recognizes this, which is why they make it incredibly easy for users to report crashes on the Mac. When an application goes down in flames, a dialog appears offering to send the details of the crash directly to the company. This is great for Apple, but relatively useless for developers. You see, although Apple is happy to collect information about every crashing application on your Mac, they don’t share the information with the developers of the product in question.

Yeah, it sucks.

Rolling Our Own

It’s obvious that Apple should be helping developers by sharing this information. It’s good for us and it’s good for the platform. It’s good for Apple. A stable environment is so important to perceived quality, that it took less than a year of supporting 3rd party software on the iPhone before they decided to share crash information with iPhone application developers.

Who knows, we may get lucky. Apple’s generosity with this information may take a turn for the better at any moment. But for the time being if you want to have any sense of the stability of your application in the wild, you’re going to have to arrange an easy way for users to send the crash information directly to you.

I recently decided to bite this bullet, and I discovered that while there is some really useful open source code out there to help achieve this goal, the pluses and minuses of each package are not well documented. I hope this roundup of several options helps direct your search for the most appropriate solution in your product.

UKCrashReporter is a popular option that is proudly low-tech in its operation. It relies on finding crash logs in the user’s home directory. If they’re there, UKCrashReporter asks the user if it may transmit the crash information to your web server via an HTTP POST request.

SFBCrashReporter and CMCrashReporter are each quite similar to UKCrashReporter, detecting crash logs at application launch time, and offering to submit them via HTTP POST to a URL of your choosing.

FeedbackReporter also relies on launch detection of crash logs, but offers to report uncaught Cocoa exceptions in your application right as they occur. While other solutions listed here are focused tightly on crashes, FeedbackReporter also provides a general-purpose feedback interface, which users could invoke from a menu item in your application for general, non-crashing feedback.

HDCrashReporter adopts the same approach as UKCrashReporter, but opts to deliver the crash information by email instead of HTTP request. It uses Apple’s NSMailDelivery class, or if that fails, attempts to open a mailto: URL, engaging the user with their favorite email client.

ILCrashReporter and its branch ILCrashReporter-NG tackle a shortcoming of all the choices listed so far: that a user will not be presented the option of submitting the crash log until and unless they launch the application again. To be sure that users immediately have the opportunity, ILCrashReporter starts a second watchdog process, that sits around waiting for your application to terminate unexpectedly. If it does, the crash information is gathered up and sent, like HDCrashReporter, by email. Unlike HDCrashReporter, the email is delivered by connecting directly to an SMTP server either deduced from the user’s settings, or else explicitly provided by you.

plcrashreporter is one of the newest additions to the bunch, and takes a decidely different approach in that it ignores Apple’s own crash logs, opting instead to scrutinize the state of memory and write its own, custom-formatted crash log to disk. It installs custom signal and exception handlers, and includes code that has an in-depth understanding of how to examine memory on different platforms including iPhone, Mac i386, and Mac PPC. Although the crashreporter is active at the moment of the crash, it requires an approach such as that of UKCrashReporter or HDCrashReporter, of detecting and then submitting the crash informaton on the next launch of the application.

Breakpad is the crash reporting facility used by Mozilla, and is unique among solutions here in that it is cross-platform, supporting Mac OS X, Windows, and Linux. Crashes and exceptions are detected as they occur, spawning a separate crash-reporter application that offers to submit crash information via an HTTP POST. Also unlike other solutions, the project includes server-side software for collecting crash information and for browsing it via a web interface.

Unsanity’s Smart Crash Reports takes a more vigilante approach, installing software that gets loaded into every application on the system. By hijacking the standard crash report dialog, SCR is able to repurpose it as a developer-friendly dialog that offers to send the report to both Apple and the application’s developer. Developers can choose to have crash information sent by email or by an HTTP POST to your server. Of all the solutions listed here it is probably the most fragile because of the way it hijacks Apple’s UI. It also requires installing software that has an impact outside of your application, which may be more than many developers are willing to ask of their users.

Great! So Which One Is Best?

If you want crash reporting in your app you’ll need to decide from among the above solutions, or starting your own solution from scratch. The problem of course, is none of the solutions listed is the obvious best choice. It depends on your own particular priorities and how you answer many of the questions at hand, including:

• How invasive/fragile should the solution be?
• Report crashes immediately, or after the next application launch?
• What platforms and architectures must be supported?
• What kind of user interface should be presented to users?
• Should reports be delivered by email, HTTP, or some other means?

Hopefully the capsule reviews above give you some direction in evaluating the options that are available, and making an appropriate choice. Of course, after evaluating and discarding the options here, I’m sure some of you will decide to hunker down and add another project to the list. Please let me know if you do.

The Best Solution Of All

The list of solutions above was gathered in one afternoon (with the help of my Twitter followers!), and I’m sure it’s not exhaustive. What the insane variety of choices tells me first and foremost is that Apple could end a lot of suffering if they just shared their dang crash reports. Seriously, look that this list. The number of choices and variety of implementation details is out of control. And these are only some open source solutions. I’ve talked to a number of colleagues who have implemented similar plans from scratch for their own purposes.

Apple: stop the madness! Free the crash reports and spare us agony of reinventing this wheel again, and again, and again. Good for us, good for the platform, good for you. Thanks.

Brent Simmons writes beautifully about a frustration that most indie software developers can relate to: the payoff proposition. He highlights the all-too-common scenario where a user offers a cash bounty, $50 for instance, to add a particular feature:

“The developers I know would rather rip up $50 bills, long sequences of them, than do something that, in their best judgment, is against the best interests of the software and its users.”

This reminds me of a listener question from a soon-to-be-released episode of Core Intuition, the podcast I do with Manton Reece. The question asks, roughly, “how can managers who don’t code relate better to their programmer employees?”

Non-coder managers and customers have a lot in common: they have a great vision for the future of a product, with little understanding of the work it will take to get there.

This is no slight on customers, or managers. Puhleez, I need customers, and if everything goes right with my business, I’ll be a manager before too long.

The naivete of customers is not a problem in itself, in fact it can be inspiring and motivating. “Yes, that does sound impossible to me, but you’re right, it would be freaking awesome if I can make it happen!”

Freaking awesome is a far better motivator than a stack of bills. You have to give the developer some credit for knowing how to evolve a product. After all, you must like what they’ve done so far, or you wouldn’t be trying to hijack their development schedule.

It boils down to something Brent nails in his write-up:

“If it’s the right feature and the right time to do it, the developer will do it. If it’s not, then it won’t get done.”

This is harsh but honest. Developers are always making judgment calls, for better and for worse. Yes, these judgment calls sometimes consider monetary compensation. We have to make a living, so great applications are built by choosing features that satisfy the intersection of “freaking awesome” and “financially viable”.

If your payoff proposition is freaking awesome, then you won’t need to pay a developer to consider it. They’ll be overjoyed just to hear the suggestion. Otherwise, it will take a lot more than $50 to pull them away from what they know is best for the product. In the end, this is good for you. Not only do you get to keep your $50, but you get to enjoy a carefully-designed application, as well.

This very post is delivered to your browser or news reader by the famous and fabulous WordPress blogging system. In my work as the developer of MarsEdit, I am exposed to countless blogging options, each with its own strengths and weaknesses. And yet, I stick with WordPress because it strikes a balance of power and ease of use which feels comfortable to me. Not to mention that Joseph Scott and others are tirelessly working to improve its API.

WordPress is licensed under the terms of the Gnu Public License (GPL) which, in a nutshell, stipulates that you are free to use the software however you like, but if you make changes and distribute those changes, then you must share those changes under the same terms. This simple, radical restriction means that you are prohibited from taking a GPL project and incorporating it with a closed-source project.

Violating The GPL

Violating the GPL is easy. All you have to do is write some code, intermingle it with some GPL code, distribute a changed copy of the original, and refuse to share your contributions. Bam! You’re toast. Assuming the original authors discover your violation and decide to pursue a resolution.

Once a violation occurs, it might be settled privately, or could escalate to legal court procedures. But the most obvious form of resolution is for the author of the changes to release their code to the public under the terms of the GPL.

Depending on how much code you “mixed” with the GPL code, this could mean only a small portion, or it could mean the entire source code of your project. This so-called “viral nature” of GPL is what scares the bejeezus out of companies, large and small, who fear the consequences of having to give up their own intellectual property to the public.

The terms of the GPL sound pretty simple at first read, but due in part to the epic consequences of a violation, there has been a great deal of debate and uncertainty about what legally constitutes a violation. Most of the debate seems to boil down to two questions:

1. What counts as a change to the original product?
2. What counts as distribution of those changes?

If you can legally justify that your additions to a GPL project either don’t change or derive from the original product, or haven’t technically been distributed, then you are not subject to the restrictive terms of the license.

Take GIMP, the popular GNU-licensed image editing application. The application supports plugins, analogous to the types of plugins you might find for the commercial, closed-source application Photoshop. A savvy developer may argue that a plugin doesn’t meet the criteria of changing the original application, because the original application still runs in its unaltered condition whether the plugin is there or not.

But promoters of the GPL take the position that plugins, by nature of being loaded into the same code space as other GPL code, do constitute a modification of the original, and are therefore subject to the terms of the GPL. As far as I know this is not a question that has been well-tested in courts.

Let me take a moment to make this abundantly clear: I respect the rights of authors to license their software under whatever terms they choose, including the GPL. In my opinion, all the legal mumbo jumbo ceases to matter once the original author’s intentions are made clear. So if the author of GPL-licensed code clarifies to me that it cannot be run on Sundays, then their GPL means it cannot be run on Sundays. But this is one of the problems with the GPL: its terms are not often understood, even by the authors of GPL-licensed code.

WordPress Themes & Plugins

WordPress supports two explicit forms of extension, each of which may affect the appearance and functionality of the product. Themes tend to work as a “skin” for the appearance of a blog, while plugins tend to introduce completely new features. Since plugins in WordPress are analogous to GIMP or Photoshop plugins, it would stand to reason that they would also be covered by the terms of the GPL. But what about themes?

Themes have been controversial in the WordPress community, as a few commercial business models have sprung up to take advantage of bloggers’ desires for high quality themes at an affordable price. One approach is to distribute “free” themes that contains commercial ads. So you might stumble upon the perfect theme for your blog, only to learn that the glaring “Brought to you by Hostess Cupcakes” line near the bottom of your page cannot be removed.

But the terms of the GPL, if themes are covered, would require that end users be granted the legal right to modify and redistribute their own copy of the theme. Zap the sponsorship, reupload to your site, and you’ve got a free, high quality theme with no ugly ads.

Today, Matt Mullenweg of the WordPress project announced his lawyer-supported opinion that themes are partly covered by the GPL:

I reached out to the Software Freedom Law Center, the world’s preeminent experts on the GPL, which spent time with WordPress’s code, community, and provided us with an official legal opinion. One sentence summary: PHP in WordPress themes must be GPL, artwork and CSS may be but are not required.

If you’re starting with the understanding that WordPress itself is GPL, and WordPress plugins are GPL, then it’s not so much extra hay on the camel’s back, to also clarify that its themes are to some extent GPL. But it got me thinking again about my own blog, and about the restrictions the GPL imposes on the kinds of things I can do with the software that runs it.

GPL Stifles Participation

Now for the most controversial point of this article, where I suggest that the GPL does more to harm collaborative development than it does to help it.

For the purposes of this argument, let me reduce all the source code in the world down to three rough categories. I recognize I have omitted some classes of license here, but for the sake of argument, most projects fall into these camps:

1. GPL code. Changes may be distributed only in other GPL products.
2. Liberal-licensed code. (MIT/BSD/Apache/etc). Changes may be distributed anywhere. Appropriate origin-attribution may be required.
3. Closed-source code. May be distributed only by the copyright owner and other explicit licensees.

Now, there are a few people in the world who, for political or philosophical reasons, will only participate in a GPL project. And for comparable yet opposite reasons, there are some who will only participate in commercial, closed-source projects. But I propose that the vast majority of developers will participate in any project that is advantageous to them.

So let’s imagine a representative, run-of-the-mill developer who is working on a project that falls into each of these three camps. If this developer is not radically committed to their own project’s license, they will naturally look to outside resources in order to bolster the success of their own work.

As the developer evaluates communities to participate in, they must evaluate the legal impact such participation will have on their own project. The closed source communities are, by definition uninviting to outsiders. GPL communities are open and embracing of other GPL developers, but generally off-putting to liberal-license and closed-license developers. Only the liberal-license communities are attractive to developers from all 3 camps.

I know what some of the GPL-enthusiasts are thinking now: “leeches don’t count as community.” Many GPL developers take comfort in the fact that their hard work can’t be quietly taken and incorporated into a commercial product, without any payback of time or money to the original project. But you’re piloting an open source project, and the first step of building a community is to get people in the door. Liberal licenses? Whoo-eee do they ever get people in the door.

If you operate from the presumption that great developers love to build great projects, the first step in any successful open source project is to get as many great developers in the door as possible.

It’s Your Party

Yes, this is just me and my crazy theories. I haven’t done exhaustive research to prove that liberal-license communities thrive more than GPL communities. But the anecdotal examples are staggering. The very foundation of Mac OS X, the operating system through which I’m typing, is thanks to the liberally-licensed FreeBSD operating system.

Looking over to the right of my screen, I’m watching this sentence appear in a live web preview as I type, thanks to the WebKit project, whose liberal license makes it compatible with closed source projects such as Safari, as well as open source efforts such as Google’s Chromium project.

For years, the problem of a generic HTTP client library that runs on every major platform has been addressed by libcurl, whose liberal license has caused it to be embraced by countless companies and projects.

The popular Subversion source control system’s liberal license enabled Sofa, a commercial software business to contribute value to the community with its extremely polished, award-winning client application. Meanwhile, the newly popular distributed source control systems presents three major choices: git, Mercurial, and Bazaar. All are restricted by the GPL-license, and therefore none is likely to inspire development of a Versions-caliber client.

I’ve touched the tip of the iceberg, and yes I’ve neglected to mention some GPL success stories such as Linux, MySQL, and gcc. These communities have thrived to some extent because the passions of the GPL community are strong, but we can’t know whether their success is in spite of the restrictions their license places on participation by the broader developer community.

Speaking of GPL succeses, WordPress is itself an example of monumental success. All of its developers have something to be immensely proud of. But whenever I am reminded that WordPress is GPL, my passion for it takes a bit of a dive. I’m more comfortable with the true freedom of liberally-licensed products. If a liberally-licensed blog system of equal quality, ease of use, and popularity should appear, my loyalties to WordPress would not last long.

It’s your party, and you’re entitled to write the guest list. But take a look around the room: not as many folks as you’d hoped for? Liberally-licensed projects are booming. Speaking for myself, a developer who has been to all the parties, I’m much more likely to pass through the door that doesn’t read “GPL Only.”