MarsEdit 3.5 is now available on the Mac App Store and directly from the Red Sweater Store. This is a free update for licensed MarsEdit customers.

Rich-Text Formatting Macros

When MarsEdit 3.0 was released two years ago, it marked a major transformation in the product. Previously the app was completely focused on providing a means to edit blog posts in HTML, Markdown, or other text based markup languages. With 3.0, I added the long-requested “WYSWIYG” rich-text editing that many customers prefer when working with web content.

But given the considerable work to achieve MarsEdit’s rich text editor, there were some loose ends. One of those loose ends had to do with the very powerful formatting macros available in plain-text mode. These formatting macros are completely user-customizable and allow users to add their own markup templates for virtually any kind of purpose, filling in the details with values from the editor such as the selection, the pasteboard, etc. Various complexities prevented this powerful formatting macro concept from being available to users in rich-text mode.

With MarsEdit 3.5, custom formatting macros are finally available in rich-text mode. I have already discovered some edge cases that don’t work perfectly, but for inserting arbitrary HTML snippets or “wrapping” the selected text in the rich editor with a particular style, these custom macros are very useful. Starting with this release, you can also customize the keyboard shortcut for any formatting macros, whether they are built-in or custom.

The possibilities are limitless, but to illustrate with a simple example, imagine you commonly find yourself wanting green text in your blog posts. The following formatting macro will take whatever text happens to be on the pasteboard, and insert it into the edited post with HTML markup for coloring it green:

Lion Full-Screen Mode

Shortly after Lion 10.7 was released, I added nominal support for full-screen mode to MarsEdit. The problem was, it wasn’t very useful for real-world writing scenarios. My mistakes? I merely zoomed the post editor window to occupy the entire height and width of the screen. Most users found this awkwardly huge editor space impossible to work comfortably in, so found themselves avoiding the full-screen mode, even though they liked the idea in concept.

I made a relatively simple tweak for 3.5 that should improve things dramatically. When you enter full-screen mode on Lion, the current width of your editor is preserved, and only the height is zoomed to occupy the full screen space on your Mac. Here’s a shrunken example for the MarsEdit post editor I’m using to write this entry:

Of course, it’s easy to imagine a number of ways that full-screen could be even more improved. Incorporating the preview window optionally into the space, offering to hide the Title and other fields, etc. But this is a step in the right direction.

Tumblr Tweaks

I won’t mince words: as a developer, I have a love-hate relationship with Tumblr. The service is immensely popular but in many regards their API (even the newer one, which I admit I’m not using yet) falls short when it comes to providing the services MarsEdit users expect. For example, it’s extremely frustrating that it remains impossible to upload images apart from photo posts. Over the years since I first added support for Tumblr, I have often wondered how much effort I should bother put in, when the service doesn’t seem too interested in catering to offline editing clients. But that doesn’t excuse the fact that many important features are supported by the API, and I haven’t had time to implement support for them yet in MarsEdit. I’m starting to chip away at those now.

MarsEdit 3.5 supports publishing entries to Tumblr as server-side drafts. I decided to tackle this one now because it’s one of the most frequently requested features from my Tumblr-using customers, and because it’s possible to implement correctly with Tumblr’s API. I tried to tackle “queue” type posts with this release as well, but ran into issues with the API that forced me to put that back on the shelf for now.

I also made some minor adjustments to the post editor UI for Tumblr to prevent some issues with Quote and Chat style posts.

Everything Else

I managed to fit a lot of little fixes and nagging shortcomings for this release. It’s hard to quantify all of those little details with much more than a bullet line in a list, so I’m listing the complete list of changes below. Hopefully you’ll just find this release to feel overall more finished and refined than before, and I hope to continue that trend on into the future.

• Highlighted Enhancements
  • Improved support for Lion full-screen mode
  • Custom Formatting Macros now work in Rich Text mode as well as HTML mode
  • Support for editing Tumblr Draft posts
• Post Editor enhancements
  • A number of improvements to Rich Text editor to address minor formatting bugs
  • Unified Formatting Macro menu for HTML and Rich Text modes
  • Draggging an image to the post editor now causes inserted image to go to expected target location in text
  • Fixed performance issues that could cause very slow typing in some configurations
  • Fixed keyboard-based text selection in the HTML editor to avoid getting stuck on line endings
  • Fix a slight mismatch in background color between Rich and HTML Text editing modes
  • Work around hanging issues with External Editing in some apps
• Tumblr-specific enhancemeents
  • Added support for server-stored Drafts
  • Fixed Word Count feature to work properly with Tumblr Quote and Chat posts
  • Revised Tumblr interface for Chat-style posts to prevent formatting problems
• Other changes
  • Added support for browsing Lightroom 4 libraries
  • New support for unicode characters in URLs
  • Bug fixes for improved Media Manager reliability
  • Fixed an issue where uploaded attachments would fail without prompting for password on authentication errors
  • Added support for authenticated proxy servers
  • Prevent a crash that occurred when working with Blog.de system’s categories
  • Improved XML sanitizing to increase compatibility with malformed blog content
  • Fix a bug that prevented locally added tags from becoming part of the tag-suggestion history

Black Ink 1.5 is now available on the Mac App Store and directly from the Red Sweater Store. This is a free update for licensed Black Ink customers.

I focused again on puzzle printing improvements for this update. Significantly, Black Ink now supports an option to position the puzzle grid on the bottom-left or bottom-right of the page. This is a more natural solving position for many people, because it allows you to keep more of the clues visible while writing answers into the grid.

Here’s a taste of how the printing configuration and preview panel looks in 1.5:

Complete list of changes:

• Support new options for printing puzzle grid at the bottom of the page
• Cleaner printing layout for fit-on-one-sheet printing
• Improve speed of printing puzzles

Tony Arcieri urges developers storing user-sensitive data, such as a passwords, not to use bcrypt (via Michael Tsai) for deriving the encryption key:

The first cipher I’d suggest you consider besides bcrypt is PBKDF2. It’s ubiquitous and time-tested with an academic pedigree from RSA Labs, you know, the guys who invented much of the cryptographic ecosystem we use today.

I was a little fuzzy on the distinction between encryption techniques such as AES, and the technology being discussed here, which is known as a key derivation function. Let’s break it down. With an encryption technique like AES you can use a large (e.g. 128 bits), difficult to guess private key to encrypt and decrypt data. But as a human, you can’t reasonably be expected to type in a random, 128-bit key in by hand when you want to access your data. The key derivation function is the code that takes your relatively easily-remembered password and derives a suitably monstrous, unpredictably random key from it. The quality and uncrackability of that key derivation is what Tony is questioning here.

I don’t know enough about encryption to have my own informed opinion about this. I tend to rely on the collective wisdom of the software industry, or on high-level service providers such as Apple, to suitably safeguard sensitive data in my apps. Tony included Apple’s FileVault full-disk-encryption in the list of technologies that use PBKDF2, which lent the technique an air of superiority in my mind. I know some of the folks behind Apple’s disk encryption, and they are careful, smart engineers.

I rely on FileVault for protection of my documents. But like most folks, I rely on Apple’s Keychain for the protection of passwords. I’m keenly interested to know if the Keychain is as secure as it reasonably can be, because I store not only my own passwords in it, but also e.g. my users’ blogging passwords in their respective keychains.

AgileBits, developers of the popular secure-storage app 1Password, made a conscious decision not to use Apple’s Keychain. They cite a variety of compelling reasons, including Keychain’s alleged use of a somewhat outdated encryption technique called Triple DES. Agile has written extensively about the design of their own keychain, in which they confirm that they are using PBKDF2 to derive their encryption keys.

I’m confident that Apple’s Keychain is secure for all practical purposes, but it is just sort of irksome if they are not adopting the very best protection that Mac-money can buy. Unable to find suitably authoritative documentation on the matter, I took to Apple’s open source for libsecurity_keychain, the library through which the Keychain’s data is managed. My reading of the source code for a function called SecKeyDeriveFromPassword, does show that Apple is indeed using PBKDF2 to generate the key.

On 10.7.3 they are, at least. The SecKeyDeriveFromPassword API was new to 10.7, taking over for the older CSSM_DeriveKey. Perhaps the default behavior of that function did not use PBKDF2. In any case, it sure sounds as if on top of Tony’s urging, FileVault’s use, and 1Password’s adoption of PBKDF2, Apple’s decision to use it as the mechanism in their latest versions of the Keychain only adds to the impression that it’s a fine choice.

For the upcoming Gatekeeper feature in Mac OS X 10.8, Apple will make it easy for customers to prevent software from running that has not been digitally “signed” by developers with a certificate from Apple called the Developer ID certificate.

Many developers already choose to sign software using self-generated signing certificates. I wrote many years ago about the value of doing this for applications that access information from the user’s Keychain. With an unsigned application, every new version of your app needs the user’s approval to access the same keychain item (e.g. a blog password). But with a signed application, Apple allows a new version of the app from the same signature authority to continue accessing the keychain without nagging the user.

You might think that switching to Developer ID would be as simple as switching from the self-generated certificate to a certificate from Apple. I certainly assumed it would be, and so apparently did The Omni Group when they shipped a version of OmniFocus signed with their Developer ID certificate.

But there’s a catch. As The Omni Group’s own Ken Case pointed out on Twitter, simply substituting the Apple certificate will lead to code signature validation failure on 10.6 and earlier systems:

OmniFocus for Mac v1.10.1 is now available for those who were running into the keychain bug in our Developer ID builds on pre-Lion systems.

— Ken Case (@kcase) March 16, 2012

What is the cause for this failure? I asked Ken and he kindly went into more detail over Twitter. I decided to elaborate on his response and to do some more research of my own. I’m sharing a summary of my findings here for the reference of other developers.

Identity Confirmation

Every item in the keychain has individual settings for how it may be accessed in the future. The desirable Keychain behavior of allowing an app continued access to keychain items painstakingly documented in an Apple tech note, but the gist of it is that after gaining approval to access a keychain item from the user, an app will be allowed to continue accessing that item if it is in fact the same app it claims to be. The thinking here is, if the user indicated to the system that she trusts MarsEdit, then any future application that can prove it is also MarsEdit should inherit that same trust.

An app is determined to be itself by meeting its designated requirement (DR). The DR is the list of policies, as specified within the code signature, for determining the authenticity of the app. The code signing process generates a default DR that is suitable for most applications. You can use the codesign command-line tool to examine the DR of any signed application on your Mac. For brevity, I’m omitting all output here except the DR:

% codesign -d -r- /Applications/MarsEdit.app

designated => identifier "com.red-sweater.marsedit" and
certificate root = H"3a3292a0f7b9b5f5492d956d8f561621fe446e51"

This makes sense. The app should be considered to be “MarsEdit” if it has the right bundle ID, and is properly signed by a certificate whose trust chain leads to a certificate with the listed SHA1 fingerprint. In this case, the certificate listed is the self-generated certificate I made a few years ago for Red Sweater’s apps. Since the certificate is embedded into the app by the code-signing process, the resuilting app is imminently verifiable and will “satisfy its own DR” on any Mac:

% codesign -dv /Applications/MarsEdit.app

/Applications/MarsEdit3.4.4.app: valid on disk
/Applications/MarsEdit3.4.4.app: satisfies its Designated Requirement

I never had a role in choosing the DR for MarsEdit. It was the default policy installed into my app by the codesign utility. I didn’t care about the DR, I just cared that users weren’t nagged about accessing the keychain on successive updates to my app. Thanks to Apple’s sensible defaults, it did in fact “just work.”

A Betrayal Of Trust

When I switched over to Apple’s Developer ID certificate, I continued to assume that Apple’s default signing behaviors would lead to reasonable behavior with regard the Keychain. And, for the most part, it does. Here is the DR for a version of MarsEdit that was signed with Apple’s certificate:

designated => identifier "com.red-sweater.marsedit" and 
anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and 
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35"

Here the requirements take a little more decoding to make sense of. I’m learning as I go here but I’ll try to translate. The first line is pretty obvious, specifying the bundle ID. The other lines proceed to state requirements of the certificates in the signing trust chain. It identifies an anchor, certificate 1, and a leaf, so it expects three certificates in the chain in all. And specifically for these certificates it expects:

1. The root certificate must be an “apple generic” certificate. This is a special certificate designation specifically documented by Apple as being any certificate “for code signed by Apple, including code signed using a signing certificate issued by Apple to other developers.” I take this to mean that the root must either be an Apple certificate or a Developer Developer ID Certification Authority certificate that Apple provided to me for signing my app.
2. Certificate 1 (the certificate in the middle of the trust chain) must have a particular field, which I was able to verify on my Mac exists in the “Developer ID Certification Authority” certificate.
3. The leaf (my private Developer ID certificate) must have a particular field, which I again verified exists in my certificate.
4. The leaf must also have a specific subject value of “493CVA9A35”, which I take to be the unique ID Apple assigned to me for my certificate.

In short, the application must be signed with my specific Developer ID certificate, that certificate must be trusted by a certificate that has a special field indicating it’s a Developer ID authority, and that certificate must be trusted by Apple.

You can examine the certificate trust chain for a signed app with codesign:

% codesign -dvvv MarsEdit.app

Executable=/Volumes/daniel/Desktop/newmars/MarsEdit.app/Contents/MacOS/MarsEdit
Identifier=com.red-sweater.marsedit
Format=bundle with Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=4953 flags=0x0(none) hashes=241+3 location=embedded
Hash type=sha1 size=20
CDHash=f20bbe82230831579e301989faf2956a0142e838
Signature size=4234
Authority=Developer ID Application: Red Sweater Software
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Signed Time=Mar 17, 2012 9:01:47 PM
Info.plist entries=31
Sealed Resources rules=4 files=149
Internal requirements count=1 size=260

I’ve highlighted the certificate chain in green. In this case, the trust chain has my Developer ID certificate as the leaf, the Apple Developer ID authority a step removed, and Apple’s Root CA a step above that. This all meshes nicely with the designated requirement rules we outlined above. So what’s the catch?

The catch is that the intermediate certificate is not likely to actually exist on any customer’s Mac. Because that’s a special certificate provided by Apple to developers, it’s only likely to be installed in the keychains of other developers. So when the Keychain goes to validate the app, it does not meet the designated requirement, so it is not trusted to inherit the access to the keychain.

A little bit that I still don’t completely understand is that apps that are signed in this way are allowed access on 10.7, even if the intermediate certificate is missing. I did just a small amount of research into this, but I think this has something to do with the new “SystemPolicy” file located at /var/db/SystemPolicy. It’s an sqlite3 file that you can freely examine. It’s where the system stores information about all the Gatekeeper signed apps on your Mac, but also where it maintains a list of what I think are high-level security overrides. Examining the SystemPolicy file on my 10.7 machine, I find a couple interesting “authority” elements:

INSERT INTO "authority" VALUES(4,1,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.13] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

INSERT INTO "authority" VALUES(5,2,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.14] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

Since Apple’s security code is open sourced, you can actually poke around at libsecurity_codesigning and try to make some sense of it. I don’t want to get too sucked into this curiosity, but I was able to confirm that the items above specify “authority” policies for running applications, and for installing applications. I was also able to confirm these are included in the source code to libSecurity, so they weren’t added on my system as a result of anything I did.

The naming of those items, “Developer Seed”, is pretty interesting. I speculate that Apple knows that the certificate chain for Developer ID-signed applications will not validate, and so they added these high level overrides in 10.7 so that we can test our signed applications, and so that customers can run them without incident. The only problem is there is no such workaround on 10.6 and earlier systems.

Getting Back To Basics

In order for our Developer ID-signed applications to work as expected on 10.6 or earlier, the designated requirement has to be met. Since the odds of the default DR being met are low, we need to override Apple’s default DR and install a simpler one that self-verifies with rules more along the lines of the way our self-generated certificate did it.

As Ken Case pointed out in his tweets, you can do this by specifying the requirements information explictily on the command line:

% codesign -f --sign "Developer ID" 
  -r='designated => certificate leaf H"xxx" and
  identifier "com.red-sweater.marsedit"' MarsEdit.app

(Split into three lines for readability on my blog. It should all be on one line.)

This will sign the app using your Keychain-installed Developer ID certificate from Apple, but it will specify the given explicit DR. The xxx string is replaced by the SHA1 fingerprint for your Developer ID signing certificate, which you can find by examining the certificate in the Keychain Access app and looking at the very bottom.

Now, when the Keychain validates the designated requirement on either Mac OS X 10.7 or 10.6, it passes the test.

A Proper Solution

I have some concerns about working around the problem using this custom, dumbed down designated requirement. Sure, it’s no “dumber” than the self-generated signature’s DR, but what concerns me is whether Apple’s support for Gatekeeper in 10.8 will make assumptions about an app’s, ahem, Gateworthiness, based on some of those semi-magical certificate field checks in the problematic DR. If I’m correct in my guesses about how those SystemPolicy entries work, then there is already “magical” behavior happening in 10.7 for Developer ID signed apps. Who’s to say there won’t be continued magic going forward?

I’m not sure what the right fix is. Maybe it’s just this, dumbing down the requirements string. But it would be nice if Apple’s default code-signing requirements “just worked”…

Hey, wait. Why does this version of my app built with Xcode 4.3 “just work”? All of my shipping apps are currently built using Xcode 3 on a 10.6 machine. All of the examples above are based on analysis of an app that has been code-signed in that environment. But when I look at a version of my app built Mac OS X 10.7, and with Xcode 4.3, the requirements are a little different:

designated => anchor apple generic and
identifier "com.red-sweater.marsedit" and
(certificate leaf[field.1.2.840.113635.100.6.1.9] /* exists */ or
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and 
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35")

I’ve highlighted in green the part that is different from my old DR. Where it used to require that intermediate Developer ID authority certificate that is missing from most Macs, it now requires either that certificate OR that a field in my Developer ID signing certificate exists. This effectively restores the passing status of the DR on any Mac, whether it has an intermediate installed, or whether or not there are any alleged funky overrides going on in SystemPolicy.

So, my recommended fix either to use Xcode 4.3 to build your app, or mimic Xcode 4.3’s code signing behavior by stealing its codesign arguments (look in the build log after building on 4.3) and applying them verbatim in your build process.

I’ve learned a whole lot about code signing and certificates today. If you got to the bottom of this post, I hope that you have too. In any case, this should give you a good idea of a situation you need to be aware of, and test against: if you are signing your app with Developer ID, and need to run on 10.6 or earlier, make sure the Keychain is still kind to your app.

Update: Now With More Facts

My conclusions above were based on some false assumptions on my part about how designated requirements are synthesized and how the certificate validation process actually works. Based on some valuable comments from Chris Suter and Evan Schoenberg I think it’s fair to say the root problem here is that “default” DR synthesis is actually just a placeholder that allows the host system to insert whatever default it thinks is suitable for the app. This explains a weird thing that I thought may have just been a bug: that the designated requirement showed up differently on 10.6 than on 10.7. It turns out this reflects the fact that if you don’t specify a specific DR, the system will do it’s best to make sense of your app (maybe based on its signatures to some extent?).

So, I’m not 100% fact based yet, but I think the new moral of the story is: if you need to run earlier than 10.7, specify your DR literally because the default DR generated for a Developer ID certificiate will not work perfectly on 10.6 and earlier.

Update: Now With Even More Facts

My friends at Fetch Softworks went even deeper on this issue, discovering that Apple’s own code-signing defaults for Mac App Store submissions offer a useful clue for a better, more generalized designated-requirement for Gatekeeper-signed apps. In a nutshell: you want to key the designated requirement off of your developer ID being listed in the certificate, rather than a precise fingerprint of a certificate.