Back in my first days at Apple, I was privileged to work with brilliant engineers (and a few idiots, lest I get too nostalgic). One of these brilliant types, who I might go so far as to describe as a mentor, had a mantra he’d throw out when anybody started whining about the difficulty of programming something. Without listening to the details, he’d sigh and give a contemptuous reply: “It’s only code!”

When I arrived at Apple I was one of the whiners. I thought there were at least three types of code in the world: easy, difficult, and magical. I thought of myself at that time as incredibly good at easy code, and perhaps squeaking by with some panache at the difficult stuff. The magical stuff – device drivers, etc. – I simply hoped nobody would ever ask me to do. By the time I left Apple, I had learned that my mentor was essentially correct.

In fact, his mantra was just a bit simplistic. There is easy code, and there is hard code. But there is no magical code. Furthermore, the ways in which people typically classify code into easy and hard categories is completely backwards. Most average programmers tend to think of things like device drivers and operating system scheduling as “hard,” while something like writing a word processing application is “easy.” In fact, the low-level code that most people are so afraid of is often the easiest code imaginable. It’s easy because it does so little. Device drivers often boil down to nothing more than a glorified switch statement – a slack-jawed servant carrying out with precision the demands of a device and its client. These are so easy even PhDs can write them! Of course, many low-level device drivers and services require elegance and brilliance to do well, but in many instances there is simply less need for clever programming at this level.

People confuse familiarity with ease. They don’t know what goes on beneath the surface of their OS’s GUI, so they assume whatever it is must be magic. If anything, these low-level services can be difficult to debug, but they’re often not difficult to code. I’m angering a lot of overpaid, sloppy device-driver engineers right now.

What’s really difficult is writing a word processor, or any other GUI application with a rich feature set and elegant UI mechanics. Writing a GUI application essentially requires defining a protocol for each “object” in the application, and writing “device drivers” that manage the interactions between those objects. So a program like Word is a web of thousands of device-driver like pieces of code, all interoperating without (we hope) knocking unexpectedly into each other. When this is done correctly, the user perceives it as magic. User magic is good!

But bad programmers assume that if it’s easy to use a program, it must also be easy to write the program. This is why we end up with a bazillion terrible “To Do List” type applications. Everybody thought it was easy, and yet even something with such relatively modest requirements far outweighs the difficulty factor of a relatively complex device driver.

So when good programmers come to me and foolishly describe themselves as unqualified for low-level work like OS design or hardware interfaces, I set them straight. You’re not unqualified – you might be overqualified! Now write me a kick-ass To Do List program.

I really love this guy’s way of communicating powerful, knock you out of your rut ideas. I guess these are what he calls “reboot your brain” ideas. I often like to repeat an idea he raised during a radio interview I heard with him in San Francisco. He was being interviewed by Marty Nemko on a local NPR show about careers and the workplace. Marty asked him in particular about the value of “keeping good ideas a secret.”

I was inspired by Kawasaki’s response, which was something along the lines of “there are too many good ideas to know what to do with, and execution is all that matters.” This is a challenging way of looking at the innovation-driven business world, but I think he’s on to something. As he sees is, basically you just have to pick something you really want to see happen and go with it. OK!

I was pleased to learn recently (via Daring Fireball and then reminded by Michael Tsai) that Kawasaki has started a blog. Not only did he start the blog, he’s pumping out articles at a furious pace. Cool!

Kawasaki’s most recent article makes some good points about how to be an “intrapreneur.” That is, a really good, inventive employee of a bigger corporation. While I agree with a lot of what he says – he misses something big when he suggests that intrapreneurs should suffer artificially:

I beg to differ! While regular employees are checking out after at most 6 or 7 hours in that Herman Miller chair, the intrapreneurs often need all-night accommodations for their tired asses. If they must suffer (and really, why should they?), then make them drink Farmer’s Brothers coffee or something.

The worst part? Kawasaki’s final point enjoins the intrapreneur to “start all over again” once they’ve made a successful product and been folded back into the corporate mainstream. At that rate, they’ll be sitting in scoliosis-inducing chairs forever!

From time to time, I get dragged into the Mac OS X Console application. Either it’s because I’m printing debugging output via NSLog and Xcode has suddenly stopped showing it in the “Run” window, or I’ve just started seeing some wiggy stuff happen with my computer and I need to check under the hood.

What starts out as an innocent enough expedition often ends up with me scratching my head about something else I see in the Console log. This is the festering trash heap where every programmer’s worst “it couldn’t really happen” nightmare output goes. Well, there’s also a lot of annoying “forgot to remove the NSLog” type innocuous output there, too. But at any rate, I usually feel a little ashamed that I wasn’t aware earlier that such and such app has been unable to open a window for the past 3 days. I wish I could keep up with this stuff, but I don’t want to be one of those nerds who leaves the Console window open all the time just waiting for junk to happen. I want to live!

So I had an idea. I’ll track this junk the way I track all the other junk. With NetNewsWire! What is syndication for? It’s for packaging data in a format conducive to my computer tracking changes in junk over time. I want to “subscribe” to my Console log.

What I’ve come up with is sort of an embarrassing hack, but it does kind of work:

But wait a minute. If it’s an embarrassing hack, then shouldn’t I keep my mouth shut? Why would I want to go spreading my half-assed solutions across the Internet? Isn’t that bad marketing? Yes, but I have ulterior motives:

1. I sort of want to show it off, anyway.
2. I need your help.

Yes, you! I need your geeky, unixy, sysloggy help. I’ve got this fun little hack, which I’m now prepared to let you download, because you’ve read along so patiently.

What are you downloading? It’s a small shell tool whose only purpose is to translate your Console log into RSS format. You get no options. You just run it, and hope that you like what you see. If you’re using NetNewsWire, you can “subscribe” directly to the executable. So just put the file somewhere on your disk, and then select “New Special Subscription…” from the File menu. You have to tell NetNewsWire that it’s a “Script,” and then you have to tell it that it’s a “Shell Script,” even though it’s a binary executable.

Now update and see your last 30 console “chunks” directly from NetNewsWire.

What’s a chunk? Aha! Yes, this is where you and the “help” thing come into play. One of the problems with syndicating something large and unwieldy is figuring out how to glom together related items. It would be overwhelming to the point of uselessness if every single line in the Console ended up as a separate feed item. So I have to try to be clever. What I’m doing right now roughly follows this logic: starting at the end of the file, move up line by line, collecting lines into a “chunk” until a line “looks like” it belongs to a particular application. Once a particular application is identified, keep moving up the file, until a line that doesn’t identify an application, or that identifies a different application, appears.

Sounds complicated, huh? Yeah, well that was the easy part. That works great when output looks like this:

2006-01-03 12:23:16.462 Xcode[4453] NOTE: Referenced project...[edited]
2006-01-03 12:23:16.562 Xcode[4453] NOTE: Referenced project...[edited]
2006-01-03 12:23:16.643 Xcode[4453] NOTE: Referenced project...[edited]

See – that’s the “dream chunk.” It is so easy to parse. I could even turn the timestamp into a feed item timestamp. It’s just to beautiful. Then you run into examples like these (grouped together for convenience of presentation):

(event handler):Undefined value
Authentication Service: Started
Jan  3 22:30:50 DanielG5 authexec: executing ...[edited]

Bad boys like this pop up all the time. Isn’t somebody in charge here? Argh – different application services are allowing logging to happen that follows different or no conventions. So my happy little “chunk things together” strategy starts to fall apart.

So I drop this funny little hack in your lap in the hopes that you’ll be inspired to try it out, and if you’re just the kind of thing who enjoys this kind of analysis, you’ll help me come up with a systematic approach for “chunking” the Console log. The real problems are when there’s some really well-defined line like the Xcode examples given above, followed by a dozen lines of output that were spewed by that program’s last “well-defined” timestamped log-point.

I suppose I could compromise and just say “all unidentifiable stuff” goes into a chunk of its own. Then you’d end up with random spewage orphaned from its owner, but it would still be temporally near and therefore show up contiguously in the feed.

A new logic for the tool might be:

1. Look at this line, does it “look like it starts with a date” and “have a decimal number in square brackets”? If so, try to find lines above it with the same number in the square brackets. Put all of these lines into a feed item with a date of the bottom-most line.
2. Look at this line, does it look unidentifiable? If so, group it with unidentifiable lines above it until I regain sanity. For consistency, give this feed item the same date as either the chunk below or the chunk above.

I’d be particularly curious to hear the opinions of anybody who’s done this kind of log file parsing before. Heck, one of you is probably going to post a link to something that already does exactly this. That would be beautiful! I’m looking forward to hearing your thoughts.

Anybody who has dabbled with CoreAudio programming on the Mac is probably familiar with the indispensable MTCoreAudio framework, by Michael Thornburgh. The framework makes it easy for Cocoa programmers to harness the power of CoreAudio’s HAL (Hardware Abstraction Layer) by wrapping some of the major components in Objective-C wrapper objects.

What has Thornburgh been up to lately? MTCoreAudio releases haven’t come as fast and furious as they did in the early days, so he must be keeping busy. It turns out he co-founded a company called amicima (pronounced like Italian – think “da Vinci”), whose mission it is to develop robust p2p infrastructure protocols for a variety of possible uses. Blah blah blah, right? Hasn’t everybody developed a p2p protocol suite? What’s left to develop? I thought so, too, but amicima’s technology offers some compelling features.

The company has just released a Mac OS X version of their flagship technology demo: amiciPhone. This program doesn’t make a great visual splash – it’s not going to win any beauty awards. It’s a “technology demo” in the best and worst senses of phrase, highlighting a robust infrastructure with minimal attention to the application’s user interface. Still, Thornburgh’s solid Cocoa-based interface provides access to the voice, file transfer, and text chat features of the program. And while he’s no visual designer, he couldn’t resist the temptation to add some Mac-only niceties. He sent me a PDF file through the program and, before agreeing to accept it, I was treated to a graphical preview of the document’s first page. That’s cool! amiciPhone is “a lot like Skype” when judged purely by its feature set. But the application isn’t out to dethrone the existing VOIP giants – it’s merely a demo of what the company’s underlying technologies might be used for. (Not that somebody else couldn’t use the technology to write a Skype-dethroner!)

Some things I noticed during my long-distance chat/demo with Thornburgh:

• Voice connection survives change of IP address. This is pretty amazing. Thornburgh opened up a voice chat with me on his PowerBook. He then proceeded to pull the plug on the ethernet, causing a few seconds timeout while his computer came to terms with the backup Airport connection. When his connection came back online, he was chatting away again. He plugged his computer back into ethernet and, because of the Mac’s much quicker switch to the hardwired connection, his IP address (confirmed with tcpdump) switched seamlessly without any interruption in voice quality.
• Slick congestion control. I’ll have to take Thornburgh’s word for it, but apparently most VOIP programs suck when it comes to managing voice and file transfer at the same time. I don’t have anything to compare with (unless you count the numerous times I’ve failed altogether when trying to transfer a file with iChat), but I was impressed when Thornburgh’s voice continued uninterrupted while a giant QuickTime movie started transferring from his house in California to mine in Massachusetts.
• High audio quality. This is a testament to CoreAudio, MTCoreAudio, and speex, the open source audio codec they’re using for the demo. I launched amiciPhone on my Mac with a MOTU 828 audio interface attached, and within seconds I was hearing Thornburgh’s voice through the speakers on either side of my office. The demo allows essentially any CoreAudio output or input device to be selected, so you can use a specific device for amiciPhone without changing your default sound settings.

The fact that this was all happening over a secure AES-128 connection with open source components got me wondering if it wouldn’t be too difficult for an application like Adium to incorporate parts of it, maybe facilitating a “voice chat” add-on separate from whatever protocol is being used for the text-chat functionality. I don’t know if the Adium team already has a plan to support voice, but since Thornburgh’s company is licensing the technology in both GPL and commercial forms, it could be an appealing option for both open source and well-funded software teams.

Of possible interest to Cocoa developers is something called MObj, which betrays Thornburgh’s passion for Objective-C style dynamic programming. This component of the amicima suite is licensed under a generous BSD license, and promises a number of Foundation’s core features in a cross-platform, ANSI-C format. I’ll have to keep that in mind next time I’m forced to write code away from the comforts of Cocoa.

I’m looking forward to seeing what technologies amicima comes up with in the future, and more importantly what creative developers out there might end up doing with them. In the mean time, maybe some kind soul will donate a UI overhaul to amiciPhone – or at least an icon!