(Note: The title of this entry is an homage to Aldous Huxley’s classic Doors of Perception.)

Most Mac users and many developers don’t realize how extensively windows are used in the composition of the overall UI experience on Mac OS X. In the years since Mac OS X was first released, the basic “Quartz Window” has been increasingly used as the workhorse element in providing the glitzy UI features we take for granted. An average user looking at their desktop, with all Finder windows closed, would answer “none” to the question “how many windows are open?” But there are dozens if not hundreds of windows being displayed in part or full on a typical Mac user’s screen. They just don’t look like windows.

A fun and easy way to experiment with this phenomenon is to put the Mac’s built-in screen capture facility into “window grabber” mode.

1. Press Cmd-Shift-4 (a shortcut whose history can be traced to FKEY resources, lightly documented in the old-ish Inside Macintosh books).
2. Unless you’ve configured otherwise, press the space bar once to switch to “window grabber mode.” You’ll see the mouse cursor change into a clunky (definitely film, not digital) camera icon.
3. Without clicking the mouse, simply move this camera icon around your Mac’s UI, watching as it highlights the various windows that it sees.

Move the cursor up to the menu bar and observe that most of the menu bar is a single window. While Spotlight’s icon gets its own window, all of Apple’s menu extras share a single window (within which you can cmd-drag such extras around to sort them). Any applications you’ve installed that use the NSStatusItem API will appear to the left of Apple’s menu extras, and each of these status items you will see also comes with a window.

The desktop? It’s a window. Every icon on the desktop? Windows. (Thanks to Will Koffel for pointing this out at the last Boston CocoaHeads).

With the Dock visible, cruise over to just touch its edge. Notice how it lights up? It’s a window. Every single icon in the dock is also a window, and if you bring up a dock item’s contextual menu, you’ll see that it’s a window, too. Not just contextual menus – click the menu bar and then invoke the camera icon. All menus are windows. And all submenus are windows.

While in screen capture mode, press Cmd-Tab to bring up the task switcher UI. It’s harder to observe here because the two modal tasks are competing with each other, but the task switcher UI is a window, as are each of the application icons included within it. Don’t believe me? Hold the screen capture camera over an item as you cmd-tab to it and release it. Screen grabber artifacts don’t lie!

A Top-Down Approach

When you get tired of the interactive method of browsing for the ever-present window object’s dominance of your computer, you might enjoy double-checking your work with the Quartz Debug application, installed as a default part of the Xcode tools. In my experience, even developers who have used Quartz Debug extensively are often unaware of the “Window List” feature hidden away in its “Tools” menu. There is much more than the simple “flash updates” type functionality you find in Quartz Debug’s main window. Check out the menus!

With the Window List open you can see every single Quartz window being employed by the system to deliver your smashing UI experience. This is great for simply gasping at the impressive speed with which all of these windows are (usually) being managed. It’s also a nice way to sneak up on surprising uses of windows in areas where you might not have thought to apply the screen capture trick. This window list includes all windows, whether they are currently visible or not. But if the window is visible, then selecting it in the Quartz Debug window list causes it to pulsate in a very unignorable way. Even better: the pulsing continues while Quartz Debug is in the background, so select a window in Quartz Debug and then switch to the owning application and try to figure out where the heck it is.

My curiosity was peaked just now when I noticed Omni Outliner Pro was sporting nine windows to accommodate my single open document. Two of them are somewhat obvious: I’ve got a document window and a drawer coming out of that window. Drawers are fairly commonly known to be windows. I’ve also got a single (or so I think) utility/inspector window:

(Update: While the UI above is overall very slick, notice how the jpg conversion reveals the shadow artifacts that result from sneaking several independent windows together. While this isn’t so noticeable in real-time, it’s an interesting flaw, and another example of why it’s tricky to get away with abusing the windowing system.)

This nifty little piece of UI work consists of a list of inspectors, each of which can be expanded or collapsed as the user sees fit. This is a familiar paradigm probably made popular by the use of just such a technique in the Finder’s “Get Info” window. I switched to OmniOutliner and turned on screen capture mode. Sure enough, each of those collapsible sections is its own, independent window! Well shoot howdy! I wondered what could have motivated the developers to do this. It’s true, dynamically growing and shrinking the subviews of a window and adjusting the overall window size at the same time isn’t exactly a newbie task, but I wouldn’t guess keeping the frames of several independent windows in sync and animating at the same time is either.

It became clear when I started playing around with this “Inpector Window” more carefully. I’m a pretty new user to OmniOutliner, and I don’t read many manuals. So it came as a complete surprise when I clicked and dragged one of these sections and effortlessly “tore it away” from the rest of the items. Each of the items is its own window because it truly is a window in a user sense, not just an implementation sense! As you drag these sections away from each other they take with them any sections attached beneath them. You can just as easily drag them back together and watch them snap into place. At any given time only the top-most section of a group of connected items contains a close box. This is just brilliant. I really must applaud the developers of Omni Outliner for this ingenious implementation, which provides a straight-forward single window metaphor for beginners while allowing power users to easily rearrange and compose their inspectors in a more suitable form. This is “window abuse” at its very best!

Windows, Damn Windows, and Broken Windows

Back in the old days (on the Mac), a window was a window was a window. The notion of a window to a user was the same as it was to a developer: a draggable chunk of screen real-estate, usually rectangular, usually containing a title bar, a close box, and a zoom box. A developer could go out of her way to design a window that defied the logical expectations of the user, but it would be a waste to do so, because the system (Mac Toolbox) would carry all the baggage of presuming that the window was a “real” window. You know, a “user window.”

As time has passed, and especially with the iterations of Mac OS X and the establishment of Quartz as the “base unit” of UI design on the Mac, the definition of “window” has been divided such that it means different things to different groups of people. To users, it still means “a moveable chunk of my screen with a title, a close box, etc.” To Carbon and Cocoa developers it generally means “the top-level container in which my ‘views’ are implanted.” While to Quartz developers it probably means little more than “a programatically moveable, layerable claim on screen real estate.” This lightweight Quartz interpretation of the concept makes for a much more reusable (and abusable) raw material.

At any given moment in time, the technical definition of a window may be the same or different from the user definition of a window. While they were more or less in sync in the past, we now have at least three distinct varieties:

1. Windows – For the sake of sanity, let’s reserve the unqualified word for the highest-level concept of a window. This is the “user’s window.” You can talk about these objects in your documentation. Users know how to navigate with their mouse to the target of your description. Example: “Click the red bubble in the upper-left corner of the browser window to close the web-page document.”
2. Damn Windows – Only a nerd could love these. They make you say, “Damn! That’s a window?” Unfortunately, you can’t call them windows to a user because they will have no idea what you are talking about. Damn windows are the bulk of what I’ve been discussing in this article: surprising uses of the low-level window object. For instance, you can’t ask a user to “move the mouse pointer to the menus window at the top of your screen, hover above the ‘File’ item and scroll down the File window to the ‘Close’ item.” In the case of OmniOutliner’s magic inspector. The five damn windows must be collectively referred to as a window, even though the developers know very well it’s not a single window! They have to bite their tongues and agree that it’s a window. The user may be asked to separate the coordinated group of damn windows into five separate, free-standing windows. Only then may they individually be referred to by the unqualified term.
3. Broken Windows – These usually start out being classified as damn windows, but soon you learn that they violate some contract between any two of the system, developer, and user. While damn windows are good things – they provide enhanced user experience through creative use of the basic window object – broken windows are disastrous because they lead one party in the dance of window interactions to be deceived, and therefore to treat the window inappropriately.

Broken windows occur when the system or a developer attempts to use a (user level) window to achieve a tricky end, without taping up all the loose ends.

NSStatusItem’s Broken Window

An example of a broken window, and the motivation for this blog entry, is the NSWindow employed by NSStatusItem to display an application’s “menu item icon” in the upper-right corner of the Mac OS X UI. This implementation crosses the line from being a damn window to a broken window by assuming the convenience of NSWindow without responsibly managing the impact that such a decision has on unsuspecting developers and users. NSWindow is a high-level developer’s API. Although an NSWindow ultimately gets displayed by a low-level Quartz window, there are certain high-level expectations that an NSWindow in a Cocoa application must conform to.

When a developer installs an NSStatusItem into the system menu bar, they do so by either deferring to the system’s default icon view, or by installing their own custom NSView subclass to handle user interaction. This is where things must have gotten tricky for the Apple engineers. To give an application access to a piece of real estate in the upper-right corner of the screen, they needed to give it a low-level window. This means Quartz will have to cordon off a bit of space and allocate a window object. This will show up when you hover your mouse over any status items installed in your menu bar. But to facilitate easy adoption by Cocoa developers, a higher-level UI interface was needed, and NSView is the natural choice for such a thing. But NSView can’t be installed in a low-level Quartz window object. NSViews must, as a rule, be installed in other views. The exception to this rule is where the “top level view” for any particular group of UI gets connected to the lower-level window API. This conection happens by way of an NSWindow object. The NSWindow’s “content view” contains any number of NSViews, each of which may contain other NSViews. But without NSWindow, you’ve got no NSView.

So the Apple developers decided to sneak an NSWindow into the equation. While the API advertises the capacity for a developer to install an NSView in the menu bar, it says nothing of the ramifications this will have on the application’s window list, or on whether the installed status item will be vulnerable to strange attacks by virtue of its rather untruthful “status” as a full-fledged NSWindow.

As I said before, the difference between a damn window and a broken window has everything to do with maintaining contracts. If the user’s and developer’s vision of the state of “windows” in an application is never violated, then it’s a damn window. If a contract is broken, well … it’s a broken window.

By assuming the responsibilities of a Cocoa window, NSStatusItem breaks an important user contract. Cocoa windows mean something to the user. For example, AppleScript exports meaningful information about an application to the user space through System Events’ “windows” property. A user can ask for a particular window by index, where a lower index represents a “more frontmost” window. It’s entirely reasonable for users to expect that the focused document of an application can be referred to like this:

tell application "Adium"
	get window 1
end 

I’m running Adium right now, with a single chat window open and activated. Adium is configured to show its NSStatusItem-based window in the menu bar. The above script, instead of returning the focused chat window, returns the sneaky little NSStatusItem window! So a user who relies on the functionality of AppleScript now has to know that “for Cocoa applications with NSStatusItems installed, the secret status item window is window 1.” That’s total BS. This window is broken.

NSStatusItem also breaks a developer contract. It would be one thing to grimace and accept that control over a status item’s window might fall into the hands of the user, but once there, it must be bullet-proofed against harmful manipulation. I don’t really care how NSStatusItem implements its functionality, so long as it doesn’t destabilize my application.

By hiding the NSWindow implementation from me (the developer) but exposing it to the user (through AppleScript), the status item sets developers up to “play the fool” on behalf of any user script that ends up manipulating the wrong window. See, these fake little NSWindows are capable of doing the darned’est things. Apparently Apple didn’t realize just how much manipulation might be attempted through AppleScript. If I run the following script:

tell application "Adium"
	set zoomed of window 1 to true
end 

I end up with this little beauty of a UI artifact:

I’m pretty sure the Adium developers had no idea they had to guard against such disturbing UI manipulation. I’m using Adium as a convenient demonstration that this affects any scriptable Cocoa app with an NSStatusItem, but obviously my biggest concerns lie in my own products. So I follow the above script up with a similar tweak:

tell application "FastScripts"
	set visible of window 1 to false
end 

Which yields an altogether comically destroyed status bar:

To Apple’s credit, after just a little bit of mouse movement, the status items seem to regain their posture. Visually, at least. In the case of setting an application’s status bar window’s visibility to false, it ironically fixes the visibility but somehow in the process causes the corresponding NSView to be unclickable. In order to resume clickability of my FastScripts icon, I have to go back to AppleScript and manually set the visibility back to true. C’est la vie!

I suppose some of you might be thinking that these violations are minor because they only affect a relatively advanced group of users (AppleScript authors), but the fact that these flaws are exposed through AppleScript doesn’t mean they won’t also rear their ugly heads elsewhere. The problem is a careless abuse of the technology of windows objects to achieve goals that are contrary to the user’s vision of what a window is. Every time you choose to do this, you must consider carefully what the impact will be on the developers and users who rely on your software. In this case, AppleScript reveals at least two obvious problems with the NSStatusItem implementation. Let’s hope there aren’t more to be discovered!

This entry was posted to AppleScript, Cocoa, Programming, Usability. Comments are closed. Thanks for reading!