NSURLConnection Crashing Antidote
March 18th, 2008A couple months ago I reported on a particularly nasty, crashing bug in NSURLConnection. What was particularly nasty about it was how widespread it was. I had received dozens of crash reports, all containing the same tell-tale sign of a problem in this part of the system.
After getting tired of explaining again and again to customers that the bug was in Apple’s code and the best we could do was hope for a fix, I realized that maybe it was worth me writing up the bug and reporting it to Apple. I supposed maybe, even though this bug’s crash log shows up as fairly common in Google, nobody has bothered to report it yet. So I wrote the aforementioned blog post and reported a bug.
Since the issue only affects 10.4.11 users, I figured the chances of a fix might be slim. Apple naturally is most concerned with the latest releases of 10.5, although they continue to issue security fixes to protect users on 10.4.11. But this was just an extremely annoying crash, not a security vulnerability, as far as I knew.
After writing the blog post I started to hear from other developers that the crash logs were extremely common for them, too. One developer mentioned that he had no less than 100 separate crash reports in his logs, from users afflicted by the problem. Users reported that it affected them while using Safari, MarsEdit, essentially any application that uses Cocoa to access resources from the web.
It sucked, man! But would it ever be fixed? Well, I have to confess that my expectation for a fix went up when Apple contacted me a few weeks ago to ask me how I would like to be credited in a forthcoming security update. How interesting! I had not reported any bug recently which I thought had security implications. I immediately became hopeful that it would be have something to do with this nasty epidemic of a bug.
Today, Apple released Security Update 2008-002, whose release notes include the following note:
Description: A thread race condition exists in NSURLConnection’s cache management, which can cause a deallocated object to receive messages. Triggering this issue may lead to a denial of service, or arbitrary code execution with the privileges of Safari or another program using NSURLConnection. This update addresses the issue by removing an unsynchronized caching operation. This issue does not affect systems running Mac OS X v10.5 or later. Credit to Daniel Jalkut of Red Sweater Software for reporting this issue.
Everything about this description sounds like the bug I reported, except that I didn’t realize it could possibly be used to exploit the security of a system. I guess this is one of those situations where it’s lucky there was as security flaw, because without it, I don’t know if it would have ever been addressed.
Now users are not only protected from this strange security vulnerability, but perhaps more significantly, protected from the repeated frustration of crashing in their network enabled applications!
The moral of the story for other developers (and users, too): always report bugs, even if they seem so widespread as to have been “surely reported.” It turns out that my frustrated effort to bring attention to this problem might have been what Apple needed in order to spot the security flaw and ultimately decide to fix it.
Many, many thanks to Apple for fixing this problem! Of course, I am putting a lot of faith in this actually meaning it’s fixed, but it sure sounds like it is. Time will tell if the “willCacheResponse” crash logs stop trickling in.
March 18th, 2008 at 8:10 pm
Nice. Glad this finally got fixed; I’ve been seeing it a lot.
March 18th, 2008 at 8:36 pm
“May lead to arbitrary code execution” is a very broad description that applies to any sort of overrun or use of stale pointers. In the majority of cases, that”™s a rather low-probability “may”, but once it”™s reported not fixing it could be a big PR mess.
March 18th, 2008 at 8:39 pm
Ahruman: Well I am just grateful that somebody inside Apple figured out that it had a potential for security vulnerability.
March 18th, 2008 at 8:44 pm
Thanks for reporting this, Daniel. I’ve seen it too.
March 18th, 2008 at 9:38 pm
Nice. I routinely cruise security update notes looking for credit shouts from Apple and when I saw your name I took note. Oh so nerdy…
March 18th, 2008 at 10:27 pm
This is great, and congrats! FWIW, there is still (to my knowledge) a crasher in Leopard, though, with NSURLConnection, that is related to HTTPS + auth, and I think a separate one involving HTTPS + auth + proxies. My bug report was marked as a dupe, though, and I can’t check on the status of the original. I know we talked about this before, but have you seen any issues with folks using HTTPS + MarsEdit? There have to be some folks out there using MarsEdit for secure sites behind password protection.
March 18th, 2008 at 11:55 pm
I haven’t heard anything like that Justin, but I don’t think there are all that many people using HTTPS + Proxy and MarsEdit. I know some people using HTTPS without a proxy, and that doesn’t seem to be an issue.
March 19th, 2008 at 4:09 am
Congratulations, getting more famous everyday.
March 19th, 2008 at 7:57 am
Thanks for the feedback… FWIW, it is the combination of HTTPS, a proxy, and HTTP authentication using asynchronous NSURLConnection that does it, and not all the time. I’ll be sure to blog it up if/when I can get to the bottom of it.
March 19th, 2008 at 8:36 am
I saw an occurrence of the 10.5 NSURLConnection problem this afternoon, when I attempted to get TextMate to update through an authenticating proxy server on a new machine I was setting up. At least I imagine it’s using https to get at the new versions. In any case Old Faithful spewed forth onto my shiny new screen…
As per the 10.4 version of this problem, once the error kicks in it’s a difficult beast to get rid of.
March 19th, 2008 at 9:32 am
And even if a bug has been reported, it might not have been reported well. A bug report that”™s isolated the bug and provided clear steps to reproduce it can save a bunch of time on the developer”™s end, reducing the cost of fixing it.
March 28th, 2008 at 12:03 pm
It’s quite possible that the security aspects of this are as real as Apple outlined but here is what it sounds like to me:
Someone at Apple was embarassed about the problem (or generally agrees that it’s unacceptable to have a problem like this exist) and wanted to fix it. However his boss said “Security fixes only” so voila — this becomes a ‘security problem’ (which means ‘it can now be fixed’)
I see this at my work all the time — Management decides how/what they want to do but individuals who care deeply see what *needs* to be done and they figure out a way to make it work within the system.
March 29th, 2008 at 11:46 am
For folks trying to get NSURLConnection bugs to appear, I have had great luck with the _debug frameworks causing bugs in NSURLConnection to appear (https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=19915). You have to be an ADC member. Unfortunately I think we will need a new one with Daniel’s fix in it to be sure we’re not seeing his bug. I will file a Radar. Justin, this may help you out though.