Safari Keychain Woes

July 29th, 2011

I’ve been running OS X Lion for a long time and generally enjoying it. But at some point along the line, Safari 5.1 stopped correctly inserting my keychain passwords into forms for me.

For the longest time, I assumed it was simple mistakes: I had neglected to ever approve “remember this password,” or I had somehow invalidated the password in my keychain. But recently I have come to realize that Safari is no longer auto-filling passwords for me on any site. Ever. This sucks. Usually when something like this happens, the first thing I do is “Repair Keychain” from the Keychain Access application. But in this case, the keychain is reportedly not damaged. No help.

To research this problem, I started the way I usually do: Googling it. While there are many promising search results, most of them lead to “solutions” that are not applicable to my scenario. Most commonly it seems the user has somehow ended up with the appropriate “User names and passwords” checkbox being turned off in Safari’s AutoFill preferences.

So how does a programmer like myself approach a problem like this? In recent years, Apple’s Instruments application has become an incredible asset for tracking down weird behavior not only in my own apps, but in other apps and in the system itself. I opened it up and started a new “Time Profiler” instrumentation session, targeting Safari. I know that Safari used to autofill my password whenever I typed in an account name that it recognized, so I start Instruments “sampling” Safari while I type my PayPal account name into the PayPal login page, and stop it after I’m done.

I want to figure out where Safari is failing to get my password, so I take a wild leap and assume the corresponding function call in Safari contains the word “password.” Typing this into the Instruments search box yields promising results:

Instruments

Now I know that Safari is at least making an effort to fetch the corresponding password for me. I see that it’s calling -[NSURLCredential password], which in turn calls a lower-level Keychain Services function, SecItemCopyMatching(). But why isn’t it filling in my password?

At this point I switch to gdb, where i can attach to the running instance of Safari and dig a little deeper into the issue.

% gdb
(gdb) attach Safari
(gdb) b -[NSURLCredential password]
Breakpoint 1 at 0x7fff8ddff4af
(gdb) c
Continuing.

I know from Instruments that Safari will call this method when it tries to autofill my password, so I go back to Safari and enter my PayPal ID again. Sure enough, Safari “freezes” indicating that gdb has interrupted it where my breakpoint was set. Now, I realize this isn’t for the faint of heart, but you can use gdb to effectively debug an application, even if you don’t have the source code. You just have to know a bit about how Apple’s compiler translates source code into corresponding assembly language instructions, and where the arguments are placed when calling methods.

Breakpoint 1, 0x00007fff8ddff4af in
-[NSURLCredential password] ()
(gdb) d
Delete all breakpoints? (y or n) y
(gdb) po $rdi
<NSURLCredential: 0x7ffd34d62920>: [email protected]
(gdb) po [$rdi password]
2: x/i $pc  0x7fff9192760e <gdb_class_getClass+4>:
	mov    0x20(%rdi),%rax
Can't print the description of a NIL object.
(gdb)
(gdb) p (int) [$rdi hasPassword]
$11 = 1

What I’ve done here is first delete the breakpoint, so I don’t get interrupted while poking around from gdb itself. Then, I display the NSURLCredential object’s description, confirming it corresponds to the account I’m trying to log into. Finally, I call the password method on this NSURLCredential object myself, so I can see quickly and easily what the response will be. Gdb’s objection to trying to print a NIL object confirms that the password is not being returned. I am further able to confirm with NSURLCredential’s “hasPassword” method that the keychain does have a password for this account, but it isn’t being returned.

Here we have a situation where apparently Safari is trying to get my password, but the system is vexing it. Let’s confirm by setting a breakpoint a little deeper, at the Keychain Services level that we also noticed in Instruments.

(gdb) b SecItemCopyMatching
Breakpoint 3 at 0x7fff85919582
(gdb) c
Continuing.

Breakpoint 3, 0x00007fff85919582 in SecItemCopyMatching ()
(gdb) po $rdi
{
    acct = "[email protected]";
    atyp = form;
    cdat = "2007-06-02 16:17:01 +0000";
    class = inet;
    desc = "Web form password";
    icmt = default;
    labl = "www.paypal.com ([email protected])";
    "m_Limit" = "m_LimitOne";
    mdat = "2011-07-29 15:30:15 +0000";
    port = 0;
    ptcl = htps;
    "r_Data" = 1;
    srvr = "www.paypal.com";
}

So far, so good. It looks like NSURLCredential is properly passing a set of criteria to the keychain, describing the kind of password item it’s looking for. I take a peek at the second argument to SecItemCopyMatching, which is a pointer to where the result will be stored, and then I ask gdb to continue executing until the function is complete.

(gdb) x/x $rsi
0x7fff62eb3590:	0x0000000000000000
(gdb) fin
Run till exit from #0  0x00007fff85919582 in SecItemCopyMatching ()
0x00007fff86417c29 in URLCredentialInternetPassword::copyPassword ()

When gdb returns control to me, I know the function has finished calling, so I examine the result area in memory.

(gdb) x/x 0x7fff62eb3590
0x7fff62eb3590:	0x0000000000000000

Indeed, it’s NULL! What is going on, here? I decided to dust off my Usable Keychain Scripting tool, which makes it easy to use AppleScript to search and inspect the keychain. Is the inability to access the password a system-wide thing, or just something that is vexing Safari?

tell application "Usable Keychain Scripting"
	tell current keychain
		password of keychain item "www.paypal.com ([email protected])"
	end tell
end tell

The script works perfectly, first causing the system to ask permission to authorize “Usable Keychain Scripting” to access the keychain item, and then returning the expected password string. This got me thinking, is it possible I’ve somehow got Keychain Accessing refuse access only to Safari?

When I look at the “Access Controls” for the affected keychain item, it lists Safari as one of the permitted apps. What the hell, I think. I’ll remove it from the list, and then re-add it. To try to “dislodge” something. Confoundingly, everytime I try to add it, it just silently fails to appear in the list. I am able to add any other app without an issue.

AuthorizedApps

As a quick check, I change the setting to “Allow all applications to access this item,” and save the changes. When I return to Safari and type in the login name, it instantly fills in the password for me.

My system has a functional keychain but Safari seems unable to be authorized to access it, either on a one-time basis or as a “always allow” permission. I finally got the clever idea to look at /var/log/secure.log, which frankly, I should have done about 5 hours earlier. What’s this?

com.apple.SecurityServer[33]: suppressing keychain prompt
for invalidly signed client /Applications/Safari.app(2938)

Aha! So that would explain it. Funny, I don’t remember tweaking anything inside the Safari application bundle, but this is certainly a very real example of how a broken code signature on Mac OS X can cause extremely subtle, hard to track-down “bugs.” I re-signed the applicaton on my own:

codesign -f -s - /Applications/Safari.app

When I reopened the app, and went to PayPal’s site, voila! It prompted me for permission, I approved, and now everything is back to normal.

I examined the Safari.app bundle contents to try to remember what I might have tweaked. Then my memory was jogged. I had opened and edited Safari’s Info.plist file to play around with some settings. Don’t ask, it’s an even longer story than this.

I take full responsibility for getting myself into this mess. I’ll certainly be more careful in the future to consider the implications of breaking code signatures on apps. However, I do think that the failure behavior could have been more informative. That NSLog I found in the console about refusing to prompt the user because of a broken code signature, could have been presented as a dialog that would instantly inform the user something is fishy. If my copy of Safari actually had been compromised by a nefarious individual, I wouldn’t have thought twice about continuing to enter my passwords and trying my damnedest to get Keychain to share my passwords with the compromised application.

Usable Keychain Scripting For Lion

July 29th, 2011

I’m tracking down a mysterious behavior of Safari in Lion, where it seems to fail to enter my password for logins that I’ve saved to the keychain. In the process of looking into this, I noticed that “Keychain Scripting” has mysteriously disappeared from Lion. As far as I can tell you must copy Apple’s scripting addition from Snow Leopard in order to keep using it.

On the other hand, I wrote an alternative years ago, called Usable Keychain Scripting. Its main advantage over Apple’s implementation is that it is (or at least, was) enormously faster. Today I updated the app to be 64-bit compatible and to fix a pernicious bug in which the password value returned for a keychain item would sometimes have garbage appended to the end of it.

Download Usable Keychain Scripting 1.0b4

This is not a supported product, and your success with it may vary. But it has been very handy in the past for me, and hopefully it will come in handy for you if you need to script the keychain.

FastScripts 2.6.1

July 29th, 2011

I released FastScripts 2.6.1 today, which restores support for Mac OS X 10.4 and 10.5, and also includes a few bug fixes and enhancements.

One cool trick in 2.6.1 is the way FastScripts behaves when your scripts include “keystroke” commands to synthesize keyboard presses. In the past, these scripts were tricky to get right in FastScripts, because the synthesized keystroke would be mixed up with the very keys you had used to invoke the script. Now, FastScripts will suspend execution of any such script until you release the keys that were pressed to invoke the script.

FastScripts 2.6.1 Changes

  • Prevent conflicts with synthesized keystroke commands and keyboard shortcuts
  • Fix for situations where FastScripts became the front app after running a script
  • Fix the built in on-screen display windows to grow in height to fit displayed message
  • Restore support for Mac OS X 10.4 and 10.5

Unfortunately, this update is not available on the Mac App Store. Apple has now rejected it twice, citing the behavior of the app when it is used to run one of Apple’s own bundled scripts:

/Library/Scripts/Mail Scripts/Create New Message.scpt

This script is terrible to start with, but starting in OS X Lion, it simply doesn’t work. It fails with cryptic errors, and FastScripts faithfully reports them. Apple is rejecting FastScripts for the behavior of a faulty script that is bundled with OS X Lion.

The review process for App Store submissions is frustrating to start with: every release takes extra time and there is a great deal of uncertainty as to when an update will finally be made available to customers. I have to admit that sometimes the review team identifies serious bugs that I am glad to have fixed before releasing an app. But the benefit of that kind of review seems to be balanced by reviews like this one, where Apple’s own bugs are being cited as the cause for rejection my app.

I am confident that FastScripts 2.6.1 will eventually be approved for the App Store. In the mean time, any customer who owns the App Store edition can download and run the direct-sale version from my site.

 

MultiMarkdown In MarsEdit

July 27th, 2011

For a very long time, MarsEdit has included built-in support for processing text with Markdown, so users who publish in that text-based format can see in MarsEdit’s preview window how the post will look after it’s published.

It’s becoming more and more common for folks who use and love Markdown to expect it to behave the way a popular superset of the language, MultiMarkdown, behaves.

When I asked on Twitter how people would feel about MarsEdit replacing the standard default Markdown filter with one that does MultiMarkdown processing instead, the responsive was enormously enthusiastic: people want MultiMarkdown.

Because I believe that for all MarsEdit users who currently enjoy Markdown support, changing to MultiMarkdown will either be a non-issue or a god-send, I plan to make this switch in an update soon.

While I was looking into the current state of MultiMarkdown I learned that since I last looked into it, is has transformed from a Perl-script version like the original Markdown, into a blazingly fast, pure C version. This intrigued me, because it would be fantastic if while upgrading MarsEdit’s Markdown support to support more features and please more people, it could also become orders of magnitude faster.

The first hiccup I noticed when examining the source code is that it depended heavily on an LGPL open source library called GLib. While the LGPL license would be acceptable to bundle within a standalone tool in MarsEdit, it feels like overkill, and frankly, I’d rather have a standalone MultiMarkdown binary that doesn’t require any outside dependencies.

So that’s what I spent the last day doing.

My fork of MultiMarkdown adds an Xcode project, and a small but important number of code changes to substitute for what GLib was formerly providing. The result is a standalone x86_64/i386 binary that accomplishes MultiMarkdown filtering on any Mac running 10.6 or higher.

If you want to test this fancy new MultiMarkdown filter in MarsEdit:

  1. Quit MarsEdit
  2. Download MultiMarkdown Beta Filter
  3. Unzip, and copy the folder named “MultiMarkdownBeta” to:

    [Home] -> Library -> Application Support -> MarsEdit -> TextFilters

  4. Relaunch MarsEdit

Your MarsEdit preview window should now have a “MultiMarkdown” option in the preview window text filter popup. Select MultiMarkdown and see how it works for you as you preview Markdown-formatted posts in HTML Text mode.

I expect this version will have some bugs, which is part of the motivation for getting the adventurous among you to give it a try. The best way to report bugs is  by email or a message in the support forums. But if you just have low-key comments to make, this blog post would be an appropriate location as well. Thanks for your help!

Update, July 28: I’ve made major performance and correctness changes in the past day, so if you are helping to test the new MultiMarkdown filter, be sure to grab the latest version and update the copy in your MarsEdit TextFilters folder.