Baking Blogs With MarsEdit
April 4th, 2011Brent Simmons has lately been singing the praises of baked, or statically published blogs. He encourages folks to switch from blogging systems that rely on dynamic scripting languages and database storage, in favor of systems that essentially just load plain HTML pages when a blog is visited.
Marco Arment was inspired by Brent’s plea, and according to comments on his Build and Analyze podcast, is strongly considering or is in fact starting development of a new baked blog system. This is especially significant considering that Marco is largely responsible for the development of the Tumblr blogging system, which would appear to be about as far as you can get from a baked blog type of system.
Brent focuses mostly on the performance aspect: dynamic scripting languages and databases are notoriously bad at fielding huge numbers of requests at once. They have to do too much “real work” in order to load a page, so if a blog suddenly gets very popular, all the work becomes too much and the site can simply buckle under the load.
Marco clearly respects the performance benefits of baked blogs, but seems to be even more interested in the control and data independence that are afforded by a simple, text-file-based blogging system. He noted in Build and Analyze that a major motivator for writing the Tumblr backup tool is he doesn’t like the idea of all his valuable writing being stored exclusively in a system that is out of his control.
To Bake or Not to Bake?
The arguments that Brent, Marco and others are making may have caused you to consider “going static.” But many of you are also deeply committed to using MarsEdit for writing and publishing. I’ll try to enumerate many of the solutions that are available to you, while explaining how MarsEdit would fit into that system’s workflow.
Option: Do Nothing
For completeness, let’s acknowledge that you may not need to do anything. If you’re satisfied with a hosted blogging system such as Blogger, WordPress.com, Tumblr, or Squarespace, then you should probably stick with what’s working. All of these services take away the headaches of managing your own blogging software, and due to their enormous scale, are unlikely to suffer performance problems even if you’re linked to by fifty Daring Fireballs at once.
Option: Add Caching
All of the performance benefits of static publishing can be achieved by using software that is smart enough to cache the dynamic content of a blog in temporary, static HTML files. As Marco mentioned, this can be done at a system-wide level with tools such as Varnish, or at a blog-level with plugins such as WP-SuperCache for WordPress. I personally use WordPress and WP-SuperCache to power Red Sweater Blog. Some blog systems, such as Movable Type, use a dynamic database for storing blog content, but regularly produce a statically published version of the site, essentially “caching” the content in a similar manner as this.
Option: Server-Side Baking
Essentially a variation of the caching mechanism described above, except that in completely static systems, the “baked” version of the blog is the canonical, reference version of the content of the blog. There is no database from which it’s derived. It’s the data. If you want to back up your blog, you simply copy the files.
Any server-side-baking blog system must offer some means for submitting posts to the system, where they are stored in their permanent, static format. Some systems may require you to FTP new post files up into the system, or to submit new post content via a web form, before it is baked into the blog. By implementing support for a standard API such as MetaWeblog, these systems can be made to interpret requests from apps like MarsEdit and store the content appropriately in the server-side static blog. Tom Insam recently explored this approach with the Jekyll blogging system, sharing a MetaWeblog implementation he wrote specifically to get it working with MarsEdit.
Option: Client-Side Baking
Of the many “blog systems” that MarsEdit supports, an increasingly terribly named one is Blosxom which was one of the earliest blogging systems, and unsurprisingly, also one of the simplest. Because Blosxom blogs are statically published with a simple one-file-per-post format, MarsEdit uses a local directory on your Mac to “publish” posts, and provides for a custom script to be run after any edits are made. In other words, the “Blosxom” system in MarsEdit could probably just as well be called “Static Publishing” because it writes a post to disk and then lets your script go wild with it afterwards. What you choose to do: rsync it, ftp it, copy it to S3, etc., is all more-or-less limited only by your imagination.
Keep in Touch
My mission as the developer of MarsEdit is to keep it useful for as many people as possible, without destroying the nature of the app itself. This means that I don’t jump through hoops to satisfy every whim of every user, but I do take feedback seriously and try to keep MarsEdit “with the times.” If you’re trying to use MarsEdit with a particular static publishing workflow, and it’s not working for you, let me know where it’s falling short. Maybe it’s something that would be relatively easy to tweak. Sure, this kind of support is mainly for geeks, but I happen to like geeks a lot!
April 4th, 2011 at 9:47 pm
For Drupal, there’s Boost (http://drupal.org/project/boost), and of course standard server caching mechanisms work well (Varnish, etc.).
Another thing to consider, especially if switching to a static system, is to use Nginx or some other lightweight alternative to Apache. This can help even further if you get fireballed, slashdotted, and red-sweatered at the same time.
April 4th, 2011 at 10:39 pm
Another option is to use a CDN (Content Delivery Network). A CDN is typically made up of caching proxy servers. I configure my Apache instance to add “Cache-Control” headers. My web site DNS entry points to the CDN, so browsers retrieve objects from the CDN instead of from my server. If the CDN does not have a recent copy of an object, it retrieves that object from my server.
The benefit is that I have full control over my server, but my server sees very little of my site traffic. This provides me with great web site performance and incredible scalability.
I use MaxCDN (http://www.maxcdn.com/). I pay them $40 per terabyte of bandwidth. There are other alternatives as well; I am not necessarily claiming MaxCDN to be better than any alternatives.
I have MarsEdit talking directly to my server, not through the CDN. I also jump through some hoops to make sure that my WordPress Admin credentials and cookies never pass through the CDN. This is important.
Full disclosure: I spent about half of my career working for CDN’s. However, I have never worked for MaxCDN, I don’t work for a CDN now, and I currently have no financial interest in any CDN’s.
April 4th, 2011 at 10:41 pm
Unless they have changed stuff significantly since I moved to WordPress, it is possible to use a baked system with MovableType, in fact I think it was the default back when I used it (and I ended up switching to WordPress because of how long it took MovableType to bake the website I think). And again unless something has changed since then I think MovableType worked with MarsEdit as well. Obviously I haven’t stayed up-to-date on MT since switching to WP but it was a pretty good blogging platform when I used it.
April 4th, 2011 at 11:29 pm
This is the second time today I’ve read about WP-SuperCache.
And I did not realize that MarsEdit can also handle Blosxom. Does that only handle the CGI version, or can it also handle the PHP Blosxom, as well?
April 5th, 2011 at 12:05 pm
I’m surprised Marco went with writing his own blog. I did that for years and then after some spam got through into my comments I switched to WordPress. The combo of WordPress and MarsEdit is good enough I can’t imagine ever wanting to go back to my own code again. (Although I do look forward to when pasting graphics is added to MarsEdit) As you noted with a caching plugin you get all the practical benefits of static pages anyway.
April 9th, 2011 at 5:45 pm
Movable Type does indeed do static, in fact, it’s the default, hence the ‘rebuilding’ thing, although you don’t rebuild to post. The DB stores posts and comments, and in all the years bynkii.com has been on Movable Type, (which is all of them), i’ve only had one capacity problem, and that was the commenting system, which by necessity does a lot of DB ops. The solution to that was Disqus, and i’ve not had a problem since.
Movable Type’s setup can be a bit of a pain in the ass, but once it’s running, it’s solid. In all honesty, my main “Interface” to Movable Type is MarsEdit. The only time I need to deal with Movable Type directly is for site changes and some backups.
April 9th, 2011 at 5:45 pm
Oh, and Movable Type has an open source component.
April 15th, 2011 at 10:57 am
I think that caching is the answer to the question. Moving from dynamically generated pages to static web pages is like going from driving a car to riding a horse…no need to take a step back when there are so many technologies that leverage caching and load balancing via CDN to keep requests on heavily visited sites from drowning servers.
May 14th, 2011 at 11:39 pm
“This is the second time today I”™ve read about WP-SuperCache.
And I did not realize that MarsEdit can also handle Blosxom. Does that only handle the CGI version, or can it also handle the PHP Blosxom, as well?”
Be careful of WP-SuperCache. It’s very powerful, but really needs an outside Content Delivery Network ($$$) to work properly and it can conflict with other plugins. If you’re not comfortable with coding, you might want to look for other alternatives.
Greg Andrews
May 15th, 2011 at 4:01 pm
Quite frankly, I blame Greenspun for this insanity. Him and that damn coffee table book on web publishing.
Firstly Gilberto, your comment is offensive, arrogant, and wrong. The fact that you are suggesting caching (static pages) and CDNs (as a caching mechanism for static pages) to fix the schema of DB backed (non static) publishing should have set off bells in your head. If DB backed publishing is so great, then why cache?
Secondly, Greg you are quite wrong about needing a CDN for WP-supercache. Especially considering the plugin has basic (very) support for CDNs.
I said over decade ago that DB backed websites would be a nightmare due to overuse by people who don’t understand them. It’s happened, and now people are suddenly realizing that DBs are not the Holy Grail because CPUs are not “ever increasing.” (BTW, cores added are less expensive that upping speed, it’s economics NOT physics that’s driving this)
There is little to no reason to put post content in a DB. It’s a misuse of DBs. Even comments don’t need to be there.
If you own your web-server, or have internal servers; DBs can make like easier. If you are “blogging” about your pet toad on a shared server, it’s overkill.
WP is the biggest offender in misuse of the DB. Automattic could easily eliminate ½ the fields and use files instead of DB fields to save processor time. They don’t care about optimization; and most likely they don’t understand it. That is why you will never see a WP site running efficiently.