Core Data Model Merging

May 4th, 2011

I haven’t used Apple’s Core Data framework all that much, but I’m trying to dabble more in it with newer projects where I don’t rely as much on legacy data storage, or am willing to take the hit of migrating from those legacy persistence models.

As a developer, the obvious upside to using Core Data is Apple’s powerful framework manages most of the nitty-gritty details of managing the … well, the data your app uses both in memory and on disk. But giving up precise control over the storage and mangement of this data also becomes maddening when you run into problems: you’re not directly responsible for the management, so you’re less likely to know exactly what is going wrong.

I ran into a problem recently having to do with versioning of core data model schemas, and how good a job the frameworks do or do not do of automatically updating existing data that a customer may have on disk. In recent years Apple has made it increasingly easy to make minor changes to your model schema and having the frameworks handle the migration easily. Apple calls this trivial technique lightweight migration, and when it works well, it works very well. To modify your existing data model, you just add a new “version” of your xcdatamodel file, and add a few options flags when creating your persistent store:

NSDictionary *options =
   [NSDictionary dictionaryWithObjectsAndKeys:
   [NSNumber numberWithBool:YES],
     NSMigratePersistentStoresAutomaticallyOption,
   [NSNumber numberWithBool:YES],
     NSInferMappingModelAutomaticallyOption, nil];

[myPSC addPersistentStoreWithType:NSSQLiteStoreType
   configuration:nil URL:storeUrl
   options:options error:&error];

This actually works quite well for simple addition or removal of entity attributes, but you might be forgiven for assuming its completely freaking broken if you run into a problem I did, having to do with a stale “.mom” file being left in the built application’s bundle in the iOS simulator and on iOS devices.

Essentially, if you’ve been building your project up to now with a single, unversioned xcdatamodel file, and you convert it to a versioned xcdatamodel file, it will start producing a new “.momd” file in your application bundle, and stop producing the older “.mom”. But because of a bug in the way Xcode installs the application binary in the simulator and devices, it will leave existing files from previous builds where they were in the bundle.

If, like many folks, you use the convenient [NSManagedObjectModel mergedModelFromBundles:nil] method to initialize your managed object model, this will, to use a technical term, bite you in the ass. The problem is Core Data looks at the .mom and the .momd files as equally viable, and attempts to merge them into some cohesive whole. When it discovers they both redundantly describe the same entities, it blows a gasket with a runtime error like “Can’t merge models with two different entities named ‘MyEntity'”

I did some googling around and discovered that the simplest solution is to simply delete the iOS app from the device or simulator, ensuring that the next time you install, you get a pristine, clean copy of the app without the troubling “.mom” file. But there’s a huge problem with that: in deleting the app you also delete its associated data files, and thus are left in a position where you aren’t actually testing the ability of Core Data to effectively migrate from your previously versioned data.

On the simulator, it’s no problem to dive in and trash the problematic, stale file. You can find the application binary somewhere in:

[Home] -> Library -> Application Support -> iTunes Simulator

Once you’ve deleted the “.mom” file you can test with your new model and everything hopefully works great.

On the device, it’s not as easy (I think) to get in there and do pinpoint editing of the application contents, but I still wanted to test my changes against the real data on my device. I was also starting to get paranoid about the possibility of an App Store update at some point also causing a stale “.mom” file to be left in an app bundle, and wreaking havoc. So the 100% safe solution to make sure the one-true “.momd” file is consulted when building the Core Data object model, is to point directly at it:

NSString *momdPath = [[NSBundle mainBundle] 
	pathForResource:@"MyModel" ofType:@"momd"];

NSURL *momdURL = [NSURL fileURLWithPath:momdPath];

managedObjectModel = [[NSManagedObjectModel alloc]
	initWithContentsOfURL:momdURL]; 

Hopefully seeding this solution into my blog will make it available for folks searching on similar failures in their Core Data projects.

MarsEdit 3.2.2

April 21st, 2011

MarsEdit 3.2.2, an extremely minimal update to 3.2.1, which nonetheless fixes some importantly frustrating bugs, has been rejected by Apple for alleged violations of their policies that I can’t yet make sense of. Rather than postpone indefinitely the availability of this release while I work out the issues with Apple, I’m making it available immediately to all customers.

MarsEdit 3.2.2

  • Fix an issue where the posts list is sorted in wrong order on launch
  • Fix Tumblr support to allow downloading more than 50 recent posts
  • Fix an issue where passwords were sometimes not accepted correctly with Blogger & Tumblr

I recommend that all direct-purchase customers update at your earliest convenience.

For Mac App Store customers, I recommend that you upgrade to the direct-download version only if you are eager to see fixes for any of the issues listed above. If you have run a Mac App Store release at least once on your Mac, the downloadable version of MarsEdit will run as a fully-unlocked version for you, but will not continue to be updated automatically by the Mac App Store.

I’m sorry for the confusion. I prefer to release updates of MarsEdit simultaneously to both App Store and direct-download customers, but in situations like this I feel it’s important to move on with life while continuing to work with Apple on the issues that are preventing approval.

MarsEdit 3.2.1

April 6th, 2011

MarsEdit 3.2.1 is now available from the MarsEdit home page and from the Mac App Store.

I allowed a few minor, but very annoying, bugs to slip through in MarsEdit 3.2. If you’ve run into confusing “Unknown Error” dialogs, they may have been intended to be simple password prompts or other more meaningful messages. Users who depend on Markdown or other custom text filters being applied to their previews will also find this release more reilable for preserving the selected text filter.

MarsEdit 3.2.1

  • Fix problem with custom preview filters not properly saving on posts
  • Fix an issue that caused “Unknown Error” message instead of prompting for password
  • Add Expression Engine to systems list, superceding obsolete pMachine
  • Update special-case warning detection for Tumblr over-capacity failures

Enjoy!

Baking Blogs With MarsEdit

April 4th, 2011

Brent Simmons has lately been singing the praises of baked, or statically published blogs. He encourages folks to switch from blogging systems that rely on dynamic scripting languages and database storage, in favor of systems that essentially just load plain HTML pages when a blog is visited.

Marco Arment was inspired by Brent’s plea, and according to comments on his Build and Analyze podcast, is strongly considering or is in fact starting development of a new baked blog system. This is especially significant considering that Marco is largely responsible for the development of the Tumblr blogging system, which would appear to be about as far as you can get from a baked blog type of system.

Brent focuses mostly on the performance aspect: dynamic scripting languages and databases are notoriously bad at fielding huge numbers of requests at once. They have to do too much “real work” in order to load a page, so if a blog suddenly gets very popular, all the work becomes too much and the site can simply buckle under the load.

Marco clearly respects the performance benefits of baked blogs, but seems to be even more interested in the control and data independence that are afforded by a simple, text-file-based blogging system. He noted in Build and Analyze that a major motivator for writing the Tumblr backup tool is he doesn’t like the idea of all his valuable writing being stored exclusively in a system that is out of his control.

To Bake or Not to Bake?

The arguments that Brent, Marco and others are making may have caused you to consider “going static.” But many of you are also deeply committed to using MarsEdit for writing and publishing. I’ll try to enumerate many of the solutions that are available to you, while explaining how MarsEdit would fit into that system’s workflow.

Option: Do Nothing

For completeness, let’s acknowledge that you may not need to do anything. If you’re satisfied with a hosted blogging system such as Blogger, WordPress.com, Tumblr, or Squarespace, then you should probably stick with what’s working. All of these services take away the headaches of managing your own blogging software, and due to their enormous scale, are unlikely to suffer performance problems even if you’re linked to by fifty Daring Fireballs at once.

Option: Add Caching

All of the performance benefits of static publishing can be achieved by using software that is smart enough to cache the dynamic content of a blog in temporary, static HTML files. As Marco mentioned, this can be done at a system-wide level with tools such as Varnish, or at a blog-level with plugins such as WP-SuperCache for WordPress. I personally use WordPress and WP-SuperCache to power Red Sweater Blog. Some blog systems, such as Movable Type, use a dynamic database for storing blog content, but regularly produce a statically published version of the site, essentially “caching” the content in a similar manner as this.

Option: Server-Side Baking

Essentially a variation of the caching mechanism described above, except that in completely static systems, the “baked” version of the blog is the canonical, reference version of the content of the blog. There is no database from which it’s derived. It’s the data. If you want to back up your blog, you simply copy the files.

Any server-side-baking blog system must offer some means for submitting posts to the system, where they are stored in their permanent, static format. Some systems may require you to FTP new post files up into the system, or to submit new post content via a web form, before it is baked into the blog. By implementing support for a standard API such as MetaWeblog, these systems can be made to interpret requests from apps like MarsEdit and store the content appropriately in the server-side static blog. Tom Insam recently explored this approach with the Jekyll blogging system, sharing a MetaWeblog implementation he wrote specifically to get it working with MarsEdit.

Option: Client-Side Baking

Of the many “blog systems” that MarsEdit supports, an increasingly terribly named one is Blosxom which was one of the earliest blogging systems, and unsurprisingly, also one of the simplest. Because Blosxom blogs are statically published with a simple one-file-per-post format, MarsEdit uses a local directory on your Mac to “publish” posts, and provides for a custom script to be run after any edits are made. In other words, the “Blosxom” system in MarsEdit could probably just as well be called “Static Publishing” because it writes a post to disk and then lets your script go wild with it afterwards. What you choose to do: rsync it, ftp it, copy it to S3, etc., is all more-or-less limited only by your imagination.

Keep in Touch

My mission as the developer of MarsEdit is to keep it useful for as many people as possible, without destroying the nature of the app itself. This means that I don’t jump through hoops to satisfy every whim of every user, but I do take feedback seriously and try to keep MarsEdit “with the times.” If you’re trying to use MarsEdit with a particular static publishing workflow, and it’s not working for you, let me know where it’s falling short. Maybe it’s something that would be relatively easy to tweak. Sure, this kind of support is mainly for geeks, but I happen to like geeks a lot!