The other day as I walked home from the gym, listening to my iPod Nano, I started to feel jealous of the iPod Video owners. I love my Nano, but it would be cool to be able to watch video on it from time to time. In spite of having a pretty good color screen, I have no access to such a feature (except perhaps with some Linux-on-iPod trick).

Then I got a sick idea. What if I could convert the frames of a video into chapter markers in an iTunes AAC file?

You may be familiar with chapter markers: they allow authors of multi-part AAC audio files to put marker points in the file so that you can skip between different sections of the audio, even though it’s all part of one file. When you play a song or audio file on your iPod that includes these chapter markers, you can also see an associated image for each chapter.

So, I set out to create an AAC file with lots of chapters. Of course the mechanics of this are a bit harder than it sounds. I had to master Apple’s weird “ChapterTool” (the only official Apple software I know of distributed through a .Mac homepage!), came up with a funny perl script to generate the XML input file that ChapterTool requires, and relied heavily on a modified version of David Van Brink’s qt_tools and Pixture Studio’s awesome QuickImageCM.

I hacked on this for several hours today and finally came up with a pretty interesting demo (10MB). The linked file contains a 3.5 minute song with 800+ chapter markers. The result, as viewed in iTunes, is “somewhat video-like!™” I can’t figure out yet whether the slight jerkiness is because iTunes is not used to flipping chapter art so frequently, or because I screwed something up in the time code generation for the XML file.

Though the file behaves fairly well in iTunes, all hell breaks loose when I try to play the file on my Nano. The Nano sort of freezes for a minute before giving up and skipping to the next track. I guess Nano just isn’t ready to deal with that many chapters in a song file! I tried scaling back my experiment quite a bit by reducing the number of chapters to just one per second. While Nano is now able to play the song, it is rather flaky about keeping the displayed image up to date with the chapters as they fly by. Another major drawback to this “solution” is that having all those chapters on the song essentially makes it impossible to skip to the end or beginning of the track while it’s playing.

So while my dream of hacking video onto my Nano via iTunes is not exactly fulfilled, I think too much went into this not to share the (probably good) news of my failure.

Back in my first days at Apple, I was privileged to work with brilliant engineers (and a few idiots, lest I get too nostalgic). One of these brilliant types, who I might go so far as to describe as a mentor, had a mantra he’d throw out when anybody started whining about the difficulty of programming something. Without listening to the details, he’d sigh and give a contemptuous reply: “It’s only code!”

When I arrived at Apple I was one of the whiners. I thought there were at least three types of code in the world: easy, difficult, and magical. I thought of myself at that time as incredibly good at easy code, and perhaps squeaking by with some panache at the difficult stuff. The magical stuff – device drivers, etc. – I simply hoped nobody would ever ask me to do. By the time I left Apple, I had learned that my mentor was essentially correct.

In fact, his mantra was just a bit simplistic. There is easy code, and there is hard code. But there is no magical code. Furthermore, the ways in which people typically classify code into easy and hard categories is completely backwards. Most average programmers tend to think of things like device drivers and operating system scheduling as “hard,” while something like writing a word processing application is “easy.” In fact, the low-level code that most people are so afraid of is often the easiest code imaginable. It’s easy because it does so little. Device drivers often boil down to nothing more than a glorified switch statement – a slack-jawed servant carrying out with precision the demands of a device and its client. These are so easy even PhDs can write them! Of course, many low-level device drivers and services require elegance and brilliance to do well, but in many instances there is simply less need for clever programming at this level.

People confuse familiarity with ease. They don’t know what goes on beneath the surface of their OS’s GUI, so they assume whatever it is must be magic. If anything, these low-level services can be difficult to debug, but they’re often not difficult to code. I’m angering a lot of overpaid, sloppy device-driver engineers right now.

What’s really difficult is writing a word processor, or any other GUI application with a rich feature set and elegant UI mechanics. Writing a GUI application essentially requires defining a protocol for each “object” in the application, and writing “device drivers” that manage the interactions between those objects. So a program like Word is a web of thousands of device-driver like pieces of code, all interoperating without (we hope) knocking unexpectedly into each other. When this is done correctly, the user perceives it as magic. User magic is good!

But bad programmers assume that if it’s easy to use a program, it must also be easy to write the program. This is why we end up with a bazillion terrible “To Do List” type applications. Everybody thought it was easy, and yet even something with such relatively modest requirements far outweighs the difficulty factor of a relatively complex device driver.

So when good programmers come to me and foolishly describe themselves as unqualified for low-level work like OS design or hardware interfaces, I set them straight. You’re not unqualified – you might be overqualified! Now write me a kick-ass To Do List program.

I really love this guy’s way of communicating powerful, knock you out of your rut ideas. I guess these are what he calls “reboot your brain” ideas. I often like to repeat an idea he raised during a radio interview I heard with him in San Francisco. He was being interviewed by Marty Nemko on a local NPR show about careers and the workplace. Marty asked him in particular about the value of “keeping good ideas a secret.”

I was inspired by Kawasaki’s response, which was something along the lines of “there are too many good ideas to know what to do with, and execution is all that matters.” This is a challenging way of looking at the innovation-driven business world, but I think he’s on to something. As he sees is, basically you just have to pick something you really want to see happen and go with it. OK!

I was pleased to learn recently (via Daring Fireball and then reminded by Michael Tsai) that Kawasaki has started a blog. Not only did he start the blog, he’s pumping out articles at a furious pace. Cool!

Kawasaki’s most recent article makes some good points about how to be an “intrapreneur.” That is, a really good, inventive employee of a bigger corporation. While I agree with a lot of what he says – he misses something big when he suggests that intrapreneurs should suffer artificially:

I beg to differ! While regular employees are checking out after at most 6 or 7 hours in that Herman Miller chair, the intrapreneurs often need all-night accommodations for their tired asses. If they must suffer (and really, why should they?), then make them drink Farmer’s Brothers coffee or something.

The worst part? Kawasaki’s final point enjoins the intrapreneur to “start all over again” once they’ve made a successful product and been folded back into the corporate mainstream. At that rate, they’ll be sitting in scoliosis-inducing chairs forever!

From time to time, I get dragged into the Mac OS X Console application. Either it’s because I’m printing debugging output via NSLog and Xcode has suddenly stopped showing it in the “Run” window, or I’ve just started seeing some wiggy stuff happen with my computer and I need to check under the hood.

What starts out as an innocent enough expedition often ends up with me scratching my head about something else I see in the Console log. This is the festering trash heap where every programmer’s worst “it couldn’t really happen” nightmare output goes. Well, there’s also a lot of annoying “forgot to remove the NSLog” type innocuous output there, too. But at any rate, I usually feel a little ashamed that I wasn’t aware earlier that such and such app has been unable to open a window for the past 3 days. I wish I could keep up with this stuff, but I don’t want to be one of those nerds who leaves the Console window open all the time just waiting for junk to happen. I want to live!

So I had an idea. I’ll track this junk the way I track all the other junk. With NetNewsWire! What is syndication for? It’s for packaging data in a format conducive to my computer tracking changes in junk over time. I want to “subscribe” to my Console log.

What I’ve come up with is sort of an embarrassing hack, but it does kind of work:

But wait a minute. If it’s an embarrassing hack, then shouldn’t I keep my mouth shut? Why would I want to go spreading my half-assed solutions across the Internet? Isn’t that bad marketing? Yes, but I have ulterior motives:

1. I sort of want to show it off, anyway.
2. I need your help.

Yes, you! I need your geeky, unixy, sysloggy help. I’ve got this fun little hack, which I’m now prepared to let you download, because you’ve read along so patiently.

What are you downloading? It’s a small shell tool whose only purpose is to translate your Console log into RSS format. You get no options. You just run it, and hope that you like what you see. If you’re using NetNewsWire, you can “subscribe” directly to the executable. So just put the file somewhere on your disk, and then select “New Special Subscription…” from the File menu. You have to tell NetNewsWire that it’s a “Script,” and then you have to tell it that it’s a “Shell Script,” even though it’s a binary executable.

Now update and see your last 30 console “chunks” directly from NetNewsWire.

What’s a chunk? Aha! Yes, this is where you and the “help” thing come into play. One of the problems with syndicating something large and unwieldy is figuring out how to glom together related items. It would be overwhelming to the point of uselessness if every single line in the Console ended up as a separate feed item. So I have to try to be clever. What I’m doing right now roughly follows this logic: starting at the end of the file, move up line by line, collecting lines into a “chunk” until a line “looks like” it belongs to a particular application. Once a particular application is identified, keep moving up the file, until a line that doesn’t identify an application, or that identifies a different application, appears.

Sounds complicated, huh? Yeah, well that was the easy part. That works great when output looks like this:

2006-01-03 12:23:16.462 Xcode[4453] NOTE: Referenced project...[edited]
2006-01-03 12:23:16.562 Xcode[4453] NOTE: Referenced project...[edited]
2006-01-03 12:23:16.643 Xcode[4453] NOTE: Referenced project...[edited]

See – that’s the “dream chunk.” It is so easy to parse. I could even turn the timestamp into a feed item timestamp. It’s just to beautiful. Then you run into examples like these (grouped together for convenience of presentation):

(event handler):Undefined value
Authentication Service: Started
Jan  3 22:30:50 DanielG5 authexec: executing ...[edited]

Bad boys like this pop up all the time. Isn’t somebody in charge here? Argh – different application services are allowing logging to happen that follows different or no conventions. So my happy little “chunk things together” strategy starts to fall apart.

So I drop this funny little hack in your lap in the hopes that you’ll be inspired to try it out, and if you’re just the kind of thing who enjoys this kind of analysis, you’ll help me come up with a systematic approach for “chunking” the Console log. The real problems are when there’s some really well-defined line like the Xcode examples given above, followed by a dozen lines of output that were spewed by that program’s last “well-defined” timestamped log-point.

I suppose I could compromise and just say “all unidentifiable stuff” goes into a chunk of its own. Then you’d end up with random spewage orphaned from its owner, but it would still be temporally near and therefore show up contiguously in the feed.

A new logic for the tool might be:

1. Look at this line, does it “look like it starts with a date” and “have a decimal number in square brackets”? If so, try to find lines above it with the same number in the square brackets. Put all of these lines into a feed item with a date of the bottom-most line.
2. Look at this line, does it look unidentifiable? If so, group it with unidentifiable lines above it until I regain sanity. For consistency, give this feed item the same date as either the chunk below or the chunk above.

I’d be particularly curious to hear the opinions of anybody who’s done this kind of log file parsing before. Heck, one of you is probably going to post a link to something that already does exactly this. That would be beautiful! I’m looking forward to hearing your thoughts.