We Need a Hero
September 27th, 2006You’re a good developer. You’ve tackled lots of tough problems they’ve thrown your way. You are respected by your peers. You’ve done a great job with your life! And then there’s that nagging in the back of your head. You can’t kill the guilt of that glaring omission – the horror of horrors. The place where your code just doesn’t stack up.
You’ve got sucky AppleScript support.
Don’t beat yourself up. Apple is to blame for this, and they should step up and start taking responsibility. Adding AppleScript support to an application, even with the pleasure/pain of Cocoa scripting, is as much of a rite of passage for developers as learning to write AppleScripts is for users. That is, it’s a nearly universally agreed upon royal pain in the ass. And that’s wrong.
Why is it so painful? With Cocoa Scripting it’s common to hear that adding support is almost as simple as flipping a switch, and it sort of can be. But it’s also true that skydiving is as easy as finding something tall to jump off of. Now how do we go about ensuring a safe landing?
I talk to a lot of developers through mailing lists, IRC, and even live in person. I’ve never met one who was completely confident about adding scripting support to an app. Even the smartest Mac programmers I’ve ever met tend to confess that they “just tweaked things until they magically worked, then stopped touching it.”
This mirrors my own experience. So let me vent some things that I think are wrong with Apple’s handling of “the AppleScript situation.”
- Inadequate Documentation. The Cocoa Scripting Guide is pretty frickin’ good but it falls short on some frustrating levels. The step-by-step is encouraging, but it fails to mention key points like “for some features, an sdef just totally won’t work, and you’ll want to resort to using old-fashioned script suites or even an ‘aete’ resource.”
Worse are the misleading details about indexed array operators. I ran into this with FlexTime. I need the scripter to be able to manipulate the list of activities associated with a given document. My code-level name for these objects is “TimedEvent.” The documentation suggests that methods of this form should do the trick:
- (void)insertObject:(TimedEvent *)newEvent inTimedEventsAtIndex:(int)index
But in practice, I had to google my way through blood, sweat and tears to discover that this form was required for scripting:
- (void) insertInTimedEvents:(TimedEvent*)newEvent atIndex:(int)index
Ugh! Kill! Kill!
- Buggy Examples. Like any good programmer, you’re looking to stand on the backs of giants. In the case of scripting, you’re going to want a good “scripting definition” file, consisting of all the standard Cocoa-supplied handlers, and any custom goodies you add on yourself. So that “standard handlers” stuff should be easy to copy and paste, right? If you look for such a starting point, you’ll find several options. And they all seem to be buggy. It’s not easy for me to advise a developer starting with AppleScript whether they should use Skeleton.sdef, NSCoreSuite.sdef, or some other less scientific approach. The documentation is equally vague:
If your application supports any of the commands, classes, or other scriptability information defined in the Standard and Text suites, as most do, you should copy the suite elements for those suites into your sdef from an existing sdef file, such as Sketch.sdef.
It doesn’t matter which path you take, you’ll end up banging your head bloody against the keyboard as you try to decipher endless, frustratingly nuanced bugs. Script results will come back with confusing chevron values instead of class names:
«class » «constant ****all » of script item "Text Input - 2 Btns"
Confusing console log messages will appear, complaining that methods like “scriptingAnyDescriptor” are not implemented in your objects. You’ll wonder aloud what the hell that method is for, and even ask around on mailing lists and search google, only to find the plaintive cries of other frustrated developers.
There should be no “hunt-and-gather” phase in adding Scripting support to an application. Apple should include a perfectly functioning standard sdef as part of the Xcode templates for all application targets. If they want good scripting support, they need to throw us a bone.
- Spotty e-Mail Support. There’s a mailing list dedicated to allegedly solving the problems of frustrated scripting support implementors. The problem is, nobody from Apple reliably steps up to answer the most vexing questions posed there. Inevitably the tough questions either go unanswered, or are eventually acknowledged by Dustin Voss or Bill Cheeseman. The two most reliable sources for AppleScript implementation support are non-Apple employees who usually have to painfully admit they only figured it out after hours of scientific experimentation. When Dustin or Bill doesn’t chime in, occasionally an Apple representative does, but it’s highly unpredictable, and more often than not silence reigns. This sucks!
OK, I know, it’s not their “job” at Apple to monitor and reply to issues on the free mailing lists. But maybe it should be. The fact is, if there was anybody on that list replying to questions with the accuracy and authority of somebody like Eric Schlegel on Carbon-Dev, then AppleScript support would be a lot better. We’re treading water, here!
Other groups at Apple also provide stellar support through their lists. Core Audio leaps to mind. USB, Quartz Composer, Cocoa, and Networking also have real experts – the developers themselves – providing quick and friendly support to developers on a daily basis. Most of these groups in fact have 2 or more team members actively tackling problems on the lists. And when something is undocumented or broken, they admit it and we move on. I’m sure there are others. At Apple, helping developers is the rule rather than the exception, and I’m grateful for that.
Thank you, Eric! Thank you Jeff Moore, Bill Stewart, Fernando Urbina, Barry Twycross, David Ferguson, Pierre-Olivier Latour, Chris Kane, Douglas Davidson, Ali Ozer, Becky Willrich, and Quinn! I’m sure there are dozens more that don’t spring to mind or whose lists I don’t read. You’re all heroes, and we appreciate it. Truly.
But we need a Cocoa Scripting hero.
Contained By What?
One of the biggest hurdles in adding Cocoa Scripting support to an application seems to be coming to terms with AppleScript’s “containment hierarchy.” It requires that any scripted object be able to fully specify its location in the containment hierarchy. For instance, in MarsEdit this text paragraph I’m typing right now needs to know that it’s the:
See, that’s how AppleScript refers to things. And Cocoa Scripting exposes objects directly to AppleScript, so that scripters can manipulate them directly. Long story short, is your Cocoa objects need to know how to tell AppleScript where they live. The way this happens is by way of the “objectSpecifier” method, which most of your scriptable objects will end up needing to implement.
This was a big mental block to me, because I’m not used to “back-referencing” from my objects to their containers. Maybe it’s my application design naiveté, but I find it awkward that in FlexTime for instance, my “cue action” object should know that it’s owned by a particular activity in a particular document. This just seems clunky to me. But as far as I know we’re stuck with it.
But just because the object exposes the objectSpecifier method, doesn’t mean it has to be in charge of it. I’m more comfortable in general with the containing object being responsible for claiming and disavowing ownership of a given object. So I came up with a generic solution that works pretty well for me.
RSContainableObject (Free, MIT License) is a generic NSObject subclass that can have its owning object and relevant key set on it. Then, when somebody asks it for its objectSpecifier, it uses that information to provide the required directions. The actual specification is accomplished by asking the containing object to reckon the object relative to itself, so the parent must itself respond to “objectSpecifier,” either because it’s also an RSContainableObject, or because it implements the method itself.
This approach works for me because I discovered that the vast majority of scriptable objects in my applications are in fact direct subclasses of NSObject. Those that are not, like my NSDocument subclass, inherit the objectSpecifier magic directly from Cocoa. So my typical set up is an NSDocument subclass that contains trees of various RSContainableObject subclasses, which all know how to reckon themselves recursively back up the chain in terms of their container.
These custom model objects end up with declarations that look something like this:
@interface TimedEvent : RSContainableObject <NSCopying>
Then when such a model object is set as a property or added as an element of a containing object, the container takes responsibility by setting the pertinent ownership terms on the object:
// Associate the object with us [newEvent setObjectContainer:self containedByKeyName:@"timedEvents" asToManyRelationship:YES];
The above example is for a “to many” relationship, meaning it’s a member of a list of items. But the same event object could be associated as a property relationship with a similar type of pattern. Here I pretend there’s an attribute of this object called “best damn timer” with an associated Cocoa method of “bestDamnTimer”:
// Disassociate from the old object [mBestDamnTimer setObjectContainer:nil containedByKeyName:nil asToManyRelationship:NO]; // Retain & Release [mBestDamnTimer release]; mBestDamnTimer = [newTimer retain]; // Associate [newTimer setObjectContainer:self containedByKeyName:@"bestDamnTimer" asToManyRelationship:NO];
By factoring the messy “objectSpecifier” code into one place for a wide variety of use scenarios, I avoid having to engage too often in the (for me, at least) bug-prone ritual of writing that method from scratch.
You might observe that a problem with this approach is that objects can’t be owned by more than one object. That’s true, but that’s also a basic limitation of AppleScript. Objects have “one true specifier” even if there are multiple legal specifiers that lead to it. For instance “default timer of document 1” might resolve to the same object as “timer 3 of application ‘ClockThing'”, but the RSContainableObject relationship should only be between the application and the object. A given object at one time can have only one “canonical” object specifier, and that is the relationship that this class affords.
Hope this helps…. but we still need a hero!
Update: A couple readers have pointed out that I might as well link to some good resources for AppleScript debugging and design:
TN 2106 – Scripting Interface Guidelines. This has a good discussion of the containment hierarchy, what it means for your application, and how it relates to the world-view of the scripter. Thanks, Erik Wrenholt.
TN 2124 – Mac OS X Debugging Magic. This talks about general techniques for debugging AppleEvents and also reminds us of the Cocoa Scripting debugging “defaults” setting NSScriptingDebugLogLevel. I always forget about that and maybe I wouldn’t be quite so tightly wound up about AppleScript if I remembered it! Thanks, Jonathan Wight.
September 27th, 2006 at 8:10 am
Oh my god, I’m going to have that song in my head all day… “He’s gotta be strong, and he’s gotta be fast, and he’s gotta be fresh from the fight…”
September 27th, 2006 at 8:34 am
I’m thinking of abandoning AppleScript support completely for my app and just going with Python based scripting, as described here: http://weblog.bignerdranch.com/?p=19#more-19
Has anyone out there tried anything similar?
Maybe Apple should just abandon AppleScript and its funky syntax and embrace Python instead. :-)
September 27th, 2006 at 9:56 am
Thanks for offering this to the community! AppleScript support indeed seems like one of those “magic” areas where incantations and bat wings might serve just as well as Google. I had thought that Automator would inspire more applications to be usefully scriptable but I don’t know that it has. Hopefully Leopard will provide us with a richer Automator and more people will find out about it.
September 27th, 2006 at 10:07 am
If by “You”™ve got sucky AppleScript support.” you mean no AppleScript support you got me and my app. :-(
I do want to add it, as it’s the road to integration for a lot of people, but every-time in the past when I’ve tried to even write AppleScript my head asplode. I sat in on some AppleScript sessions at WWDC 2005 but it wasn’t clear what I needed to do. I think AppleScript has two ways to implement it now.
Hopefully I can get some help with it from a local guy who is an AppleScript god.
September 27th, 2006 at 10:48 am
Don’t forget the absolutely brilliant sdef editor:
http://www.shadowlab.org/Software/software.php?sign=SdEd
It’s literally saved me hours whilst working on adding scripting to Event Maker.
September 27th, 2006 at 11:29 am
Thanks for the code, it will come in handy. I completely agree something needs to be done though I cynically think we need an executor not a hero. It is a far worse mess than Java-Cocoa, the two worlds just don’t intersect enough.
Joe: I’ve had a lot of luck with Javascript. http://will.thimbleby.net/script/
September 27th, 2006 at 12:11 pm
Daniel, couldn’t this line be in RSContainableObject code, as part of the dealloc method? I suppose that even if it was, one would still have it in its own code too, “just to be sure”.
[mBestDamnTimer setObjectContainer:nil
containedByKeyName:nil
asToManyRelationship:NO];
Another comment: this whole hierarchy things reminds me of the CoreData architecture a lot, with automatic handling of inverse relationships, so I wonder if there are potential cool tricks to get automatic Applescript support with CoreData (and maybe I should just RTFM!).
September 27th, 2006 at 12:24 pm
Hi Charles: I don’t think it’s necessary to go out of the way to set things back to nil in the object’s dealloc itself. I mainly set it to nil from the container object when “disassociating” mainly to prevent any confusion if that object goes on to live a happy life outside of the containment hierarchy.
But your comment did point out to me that I have potential leak! I’ve added a dealloc method to RSContainableObject so I can be sure to dispose of the mContainedByKeyName string object. The container object is never retained so it doesn’t need to be released. I updated the downloadable archive to include this new dealloc method.
As for CoreData, I think there is some “scripting for free” mojo in there, but I am also woefully behind in learning the ins and outs of it.
September 27th, 2006 at 1:05 pm
Daniel: all you said makes perfect sense. I was thinking more about reducing the amount of code to write and factoring it out. I am glad my otherwise somewhat fallacious comment had a beneficical side effect!
September 27th, 2006 at 2:51 pm
I’m with you. Understanding the scripting tutorial is one of the toughest thing. It’s more of a trial and error thing.
What helped me was…
1) The script tutorial from Matt Neuburg. It provides a nice intro.
2) I learned scripting support step by step. I played around a while with simple properties. Then I added 1-N relationships and after 4 hours of trial and error a 1-1 relationship :). The trick was to add a hidden 1-N relationship and a property to access the single reference. After that I started to add commands.
3) looking at the script definitions of other apps could help too (unfortunately most of them do not use the new sdef format)
4) Automated tests. Testing the scripting interface can be tricky. I have a lot of scripts that test different parts and combinations of my scripting interface. You can add them to the build process in XCode, too. If you use the Script Editor do not forget to restart the editor after you changed the scripting definition (that cost me another hour or so).
5) As mentioned above. Use the wonderful sdef editor. Apple should put a link to this tool in their tutorials – but even the applescript engineer at WWDC did not know the tool.
As I mentioned when I wrote about Objective-C 2.0 and missing annotations. I hope that someone comes up with some kind of XDoclet for Objective-C to add scripting support with ease.
September 27th, 2006 at 4:18 pm
I guess ‘Amen’ would be a good reply to that…
The combination of sub-par (even for Apple) documentation and general obscurity makes implementing AppleScript support a royal pain if you need to go further than the most basic basics.
The variety of scripting definition file formats doesn’t help, as doesn’t the fat that you can generate both the traditional scriptSuite and sdef files from your fancy new sdef file and including all of them can magically improve your application’s script support.
All of this is particularly painful because it seems that Cocoa is quite close to magically providing good support for scripting out of the box. It’s just that it fails to fully do so and that figuring out what to do in those cases is rather tricky.
I wrote about my pet peeves with AppleScript in Cocoa a while ago. And there are a number of comments which I found rather helpful at the time on that.
September 27th, 2006 at 10:28 pm
Just a quick post as I change planes here in Germany…
Thank you for your critique. I assure you that these issues will and are being discussed, and we will continue to incorporate your feedback and requests as we move AppleScript forward.
Many programmers aren’t aware of the incredible rate of adoption of AppleScript amongst our customers and developers. AppleScript usage and adoption continues to increase, both inside and outside of Apple. Thousands of businesses, many major corporations, build their workflows on AppleScript and find it to be an irreplacable component of how they do buisness and a critical adavantage over their competitiors. Even consumers use AppleScript scripts written and posted by others (like Doug’s Scripts for iTunes) to solve workflow issues they encounter. AppleScript usage is growing amongst all users.
In terms of Apple, AppleScript usage in growing. As an example, the recent announcement of Aperture 1.5 highlighted its AppleScript and Automator support. This integration has proved to be invaluable for professional photographers looking for ways to deal with the avalanche of content that comes with the digital media age. Aperture’s AppleScript support has provided them with the means to be even more productive. You will continue to see more AppleScript integration in Mac OS X and Apple’s applications.
In regards to Automator, AppleScript usage has increased since Automator was released. Even though Automator is not based on AppleScript (actions are written in Xcode and can incorporate any language or framework supported by Mac OS X, of which AppleScript is one), it has raised awareness about the importance of automation and so correspondingly of AppleScript.
It’s been my personal observation that the majority of script writers aren’t programers and have no desire to be “traditional” programers. They like the “relaxed” syntax of AppleScript. That doesn’t mean that the “gotchas” of AppleScript are acceptable — indeed, in each release of the Mac OS X, we strive to streamline and advance the language. Leopard will certainly be no exception in that regard. We want to make AppleScritp good for developers as well as consumers.
Again, thank you for your feedback (ouch). We wil use it as we continue to move AppleScript forward. ;-)
Caio!
Sal Soghoian
AppleScript/Automator Product Manager
September 27th, 2006 at 7:54 pm
I think AppleScript can only be boosted by diehards. For the rest of us, scripting has been solved by several alternatives. It would be useful to begin a grassroots effort to ignore AppleScript and begin making a scripting architecture that works.
It might be DOM-based. It might support multiple scripting language bindings (Python, Tcl, Perl, javaScript…) and compiled languages. It -might- even have an OSA bridge to deal with old style (AppleScript) scripted apps. It will certainly have a test system, documentation, and solid developer support, otherwise it’s a non-starter. It’d be great if Apple embraced it, but ultimately that may not be necessary.
What alternatives have been tried? In the 90s people embedded Tcl, for example, in apps, or added plug-ins to Tcl. What else has been tried on Mac OS?
September 28th, 2006 at 2:11 am
>BTW, “My programmers aren”™t aware” should read “Many programmers aren”™t aware”. Sorry about that.
Strangely, I thought the first version was the right one. From where I seat, I don’t see AppleScript usage increasing because:
1. It’s painful to write AppleScripts Scripts. By removing the ability to record scripts from Finder usage, Apple just killed the best way to see how a script can be written. Since Apple does not support AppleScript as much as it did in Mac OS 8/9, I don’t see the point in using AppleScript. Automator is the ultimate proof that AppleScript is being moved to the Trash. AppleScript should have been put int the same coffin as Mac OS 9.
2. it’s painful to write Cocoa application supporting AppleScripts. AppleScript support in Cocoa is like the TextEditor demo in Cocoa. Sure, it’s easy to do easy things, but as soon as you start trying to do more complex things, AppleScript support is poor (e.g. moving items within arrays is buggy, dealing with 1-N and N-N or 1-1 relationship is a nightmare come true, etc.).
September 28th, 2006 at 6:35 am
Sal: it’s a real honor that you read and chose to comment on my blog post. Thanks for taking the time to do that (for those who don’t know, Sal is basically “Mr. AppleScript” at Apple, and has been for a number of years).
As with any kind of frustrated vent, I run the risk of hurting people’s feelings. So I really hope you don’t take too much of attack personally. I’d like for you, and anybody else at Apple who might have influence, to view it as more of a symptomatic description of the problem than a specific indictment.
If we can agree that the value of AppleScript support in 3rd party apps will increase proporitonally to Apple’s increased support of developers, then I’m a happy man.
On the positive notes, that AppleScript adoption is growing and that Apple is working hard on improving the proposition across the board: great news and I’m looking forward to it.
Thanks again, I really appreciate getting your feedback to my thoughts.
September 28th, 2006 at 6:37 am
Many of the comments are taking my rant as an opportunity to sort of summarily judge AppleScript as useless or ready for the recycling bin. I have to say that in spite of AppleScript’s flaws, it’s really the best things we’ve got going for building extensability right into our apps. And the fact is that many people confuse the language (what the users write with) with the support (generic AppleEvent handling that may serve AppleScript or any number of other languages).
So it’s one thing to suggest that another language might be an improvement over AppleScript, but in any case we’re going to need a system for exposing the model of our applications and allowing script languages to interact with them. I’d rather learn how to harness Apple’s Cocoa Scripting than write my own.
September 28th, 2006 at 7:34 am
What about Suite Modeler? This thing saved me big time when I first put applescript support into my scope controller app. The examples provided were great as well (I have since contributed to them).
http://homepage.mac.com/donbriggs/
September 28th, 2006 at 7:42 am
While we’re on the subject, how about something to make Applescript enabled Cocoa applications more easily recordable? The only way I know of doing this is a Carbon call. Pretty scary when someone googled this subject and I came up as the expert on this on the web (very scary!).
For users starting from scratch, nothing beats turning on the Record button in Script editor, seeing how doing stuff in an app generates a script, and then tailoring that script to do exactly what you want.
If you want users to use and see all that Applescript can do, a blank page is a pretty tough way to begin. Make it easy for them to get started.
Sadly, recordable applescript apps have really gone by the wayside in the past few years. There are some notable and excellent exceptions, such as Pro Fit (which is a Carbon app).
September 28th, 2006 at 10:43 am
This would explain why scripting support is not great in a lot of applications, including many of Apple’s own (indeed, Apple is one of the worst offenders in this regard. Ever try to script Safari, for example?).
Of course, AppleScript is itself a mostly undocumented language. If it weren’t for Matt Nueberg and his “Definitive Guide”, I would still be entirely helpless in the face of AppleScript (and I would never have been able to write FlexLog, either :). I’ve wanted to learn for as long as I’ve used a Mac (going on 13 years now), and have never found success due to lack of documentation.
September 29th, 2006 at 6:59 am
I echo Ryan’s sentiments. I’m heartened that Sal sees AppleScript usage going up, and wonder where the statistics come from. Are they coming from corporate and higher end developers? In oter words, people whose jobs are so well funded they can afford the time to make it work?
I’m not a Cocoa developer, just a wannabe. I’m no slouch, however, as I’ve been a professional software engineer for 30 years, and coding since 1972. I am not sure which is the source of the unpredictable and unusable nature of AppleScript, the language and it’s fuzzy “relaxed syntax” or the underlying logic- and data-access model. Daniel, you’re an optimist and it’s totally in character for you to look at the bright side, offer workarounds and then say “AppleScript is the best thing going…”. I’m not satistifed with good enough for some, so I have to add “…and that’s not saying much.”
I will add that I’ve written three complete scripts: two using Automator to copy files to a hardwired folder; another iTunes script to rename book tracks made by editing a similar script. If that’s what qualifies me as a successful AS user, then I’ll shut up. But I expected more from a system scripting language. Witness the VBScript I wrote for Visual Studio which converts a method by adding thread-safe protection code to it. I couldn’t do that in AppleScript if I worked at it until I died. And I’m not known as a Microsoft booster.
September 29th, 2006 at 2:38 pm
“Strangely, I thought the first version was the right one. From where I seat, I don”™t see AppleScript usage increasing because:
1. It”™s painful to write AppleScripts Scripts. By removing the ability to record scripts from Finder usage, Apple just killed the best way to see how a script can be written. Since Apple does not support AppleScript as much as it did in Mac OS 8/9, I don”™t see the point in using AppleScript. Automator is the ultimate proof that AppleScript is being moved to the Trash. AppleScript should have been put int the same coffin as Mac OS 9.”
Um, having been doing WAY too much AppleScript work for far too long, I realy have to disagree with a lot of that. Actually, all of it. The automator comment is silly, since AppleScript is the prime language for Action development.
First, Apple’s AppleScript support, particularly in the Pro applications has been *growing*. Aperture, Soundtrack, etc.
(Again, my offer still holds. Make FCP and DVDSP fully scriptable, and I’ll eat a deep-fried sock freshly pulled from the Pro App VP at Apple at either Macworld Expo or the WWDC. Live. No ketchup. Make them recordable, and the sock for the deep-frying and the eating can be from a source of their choosing. ”
Apple Remote Desktop, while not having as good a dictionary as it could have, has a solid one, and it’s made my use of it as a sysadmin far easier.
The vast improvement in Folder Actions over 8/9 is another example.
As far as recordability, I always viewed it as a trap, because what it was showing you was a scripted version of the user input, which is not always the way you want people to learn. For anything but the most simple, single-window operations, it created really horrible syntax, even by the worst of AppleScript standards.
There’s also something else to point out. If your application has a solid Scripting dictionary, then you get more than just AppleScript. You get OSA support, which allows people, today, to use Perl, Python, JavaScript, and i *think* Ruby, instead of Javascript for your application. (I’m really not sure about the Ruby one.)
You don’t have to do any more work for those langauges than you do for AppleScript.
I DO completely agree that Apple’s documentation and support for AppleScript is still way too neglected. (You think it’s bad on the cocoa end? Try using AppleScript Studio aka “The Environment of Random Bugs with varying lifespans and causes”. Dat’s some ugly spicy stuff there)
There’s also a ton of workflow and actual work done using AppleScript that isn’t going to change anytime soon. The people using AppleScript internally to their company are not going to use Python just because (application) doesn’t like AppleScript. More likely, they’ll find a new application. Even then, language – specific scripting implementations cause more problems than they solve. OSA, far more in OS X than before, lets you avoid that entire issue.
Devs can do what they want with their code, but dumping AppleScript is probably not a great option in the long run.
September 29th, 2006 at 2:53 pm
OK, I’m back from Photokino in Germany — a bit lagged — but alive ;-)
Thank you all for your comments — positive or negative — they are useful and are taken in the spirit of constructive critisism.
1) Regarding documentation:
I just did a quick search on Amazon for AppleScript and the first four links are to high quality books from Matt Neuburg, Adam Goldstein, and Hanaan Rosenthal — all excellent sources of AppleScript knowledge.
For detailed step-by-step topic instruction, Macscripter has been posting very powerful series on various AppleScript topics. In addition, they host a forum, and a n extensive repository of scripts and scripting tools.
Scripting Matters have been conducting sold-out hands-on AppleScript training classes in the US and Europe. Ray Robertson and Shane Stanley are incredible instructors and their sessions have become nust-attend events.
Den Waldie has just released his 2006 AppleScript Training CD which is a thorough step-by-step examination of all things AppleScript.
For the fifth year, Macworld will offer a two-day hand-on AppleScript training conference taught my me. The classes have always been full so sign up soon.
That’s just a quick smattering of some of the training available for AppleScript.
2) AppleScript is impossilbe (difficult) to write.
As I mentioned in my last post, this comment usually comes from programmers who expect AppleScript to be like the languages they already know. It’s not and never will be. It was designed for motivated Macintosh users who wanted a way to control their computers and the applications that run on them. And this approach has been succesful. Although the AppleScript language does have its quirks, I’ve met countless “non-programmers” who have taken to it successfully to make their jobs easier.
However, for the “traditional” programmer, AppleScript has challenges. It’s “loose” changable syntax, non-uniform object-model support, and differing qualites of application dictionaries, can be daunting for those trained to write code in a specified uniformed typed manner. [warning: bad anology approaching] Kinda like the change from Mac OS 9 to Mac OS X — “What is this Home folder, why can’t I just put my files where I want?” You may have to get rid of preconceptions about how AppleScript “should” be and just explore. You will be surprised at the things you can do with AppleScript.
For example, the “every… whose” clause:
tell application “Finder” to duplicate every document file of the entire contents of folder “Documents” of home whose name contains “Smith Project” and name extension is “doc” to the disk named “Backup”
To someone used to a strict language syntax, this may appear confusing. It may help to understand that every tell statement (what you see here) usually contains two things: a reference to the object or objects to be manipulated, and the action to be performed. If you read the script again, you’ll see that those two items are there.
You can’t expect the AppleScript language to be Object-C or Perl or Python, any more than you could expect Chinese to be English, or French. Yes, it’s fair to expect the AppleScript language to be consistent and more uniform in its design (we’re working on those issues), but its approach — an English-like high level language — is fundamentally different from other languages. And that’s OK.
For those who want to access OSA scripting looks via other languages like Python and JavaScript, there are free tools to do so, so you can control iTunes in just the same way as AppleScript does.
Again, thank you all for your comments!
Sal
September 29th, 2006 at 3:18 pm
I was just thinking… What is the Shell equivalent for the AppleScript example in my post?
tell application “Finder” to duplicate every document file of the entire contents of folder “Documents” of home whose name contains “Smith Project” and name extension is “doc” to the disk named “Backup”
Something like:
find ~/Documents -name ‘*Smith Project*.doc’ | cp /Volumes/Backup
September 29th, 2006 at 3:52 pm
I dugg this story, and was surprised not to see it previously dugg. The URL is here. As I said on my blog, nothing in the development process made me feel as stupid as trying to add AppleScript support.”
September 29th, 2006 at 4:32 pm
Sal – again I appreciate getting your comments here. But all of the advice you’re offering is from the point of view of AppleScript authors, not implementors. The resources for AppleScript authoring could be better, but as you point out there’s a lot of good stuff out there. Do you have any pearls of wisdom for the implementation end of things? And out of curiosity, does any of the implementation end of things fall under your supervision at Apple, or is it left to the individual technology groups (Cocoa, Carbon, Apps, etc) ?
September 29th, 2006 at 6:36 pm
Sal’s certainly right in that there are some excellent books that the end user can purchase. But the pain of Applescript becomes a truly *Royal* pain when you try extending your knowledge to, say, an application that Matt Neuberg hasn’t covered. You open the dictionary, stare at the objects, scratch around a little, scratch some more, maybe make a tiny bit of progress, come up against something else that doesn’t work as you think it should, scratch head again, wonder whether you’re being particularly dense, pick a few more scabs, and finally begin to wonder whether the feature advertised in the dictionary actually works. Because there’s no standard scripting implementation you’re constantly rubbing up against this nagging feeling that maybe the scripting support is, well, sort-of-support. Bits of it work, but does everything? And this process is a long way from being fun.
For the couple of years that I bashed my head against Applescript there were persistent calls among users for examples to be included within the scripting dictionary (or in some other easily accessible location). This would stop a bit of the scratch/peck/google/beg/sigh/scream cycle that accompanied almost every attempt I would make to script an ostensibly scriptable application. As far as I can tell you’re still entirely reliant on scant-to-none documentation from application developers, on hanging around mailing lists, googling, much trial and many errors… not unlike how Daniel describes the situation for developers in the piece above.
I do try and approach Applescript from a fresh perspective every so often: someone posts a problem and I think that it should be a natural fit for the capabilities promised by Applescript-able applications. Occasionally I can ferret out enough info to crack the cryptic crossword that appears in the script editor –and a lot of the hints that help in this process have come from Sal– but way too often the whole script dances a little, then trips over a problem with syntax for a half hour or so. That’s enough to ensure that the Script Editor quickly gets commanded to Q.
September 29th, 2006 at 7:05 pm
Having worked with AppleScript from both the implementation and authoring sides I certainly agree that it is more mysterious than I would like. Nevertheless it is a very powerful tool and one of the reasons why the Mac version of our software (Panorama) is much more powerful than the Windows version.
“You open the dictionary, stare at the objects, scratch around a little, scratch some more, maybe make a tiny bit of progress” — my experience with AppleScript used to be very similar to this, but not any more. When I discovered Script Debugger from Late Night Software my experience with AppleScript changed completely. The debugging aspect of this program is cool, but what really is great is it’s ability to graphically explore the object model of any scriptable application. Instead of guessing and poking around in the dark this tool let you cut right to the chase even with an application you’ve never scripted before. It costs a couple of hundred dollars but for me it paid for itself the first day and has repaid it’s cost many times over since. In fact, sometimes now I almost consider writing AppleScript fun. Definitely the frustration factor is dialed way down.
September 29th, 2006 at 7:49 pm
“Sal – again I appreciate getting your comments here. But all of the advice you”™re offering is from the point of view of AppleScript authors, not implementors.”
Bingo.
Apps that implement useful AppleScript dictionaries are too few and far between. One of the best ways for Apple to help AppleScript end-users is to deal with Daniel’s original complaints so more developers could add robust dictionaries to their apps.
September 29th, 2006 at 8:53 pm
Amen my brother.
As a long time AppleScripter (since 1994), when I started to produce my own apps, of course I wanted to offer robust AppleScript support. I now have several apps and only one has “real” AppleScript support (http://www.jonsphonetool.com/) and that was a complete kluge to add. Painful on every level.
From both a user’s (scripter) and developer’s (Cocoa) perspective, Apple has always let AppleScript languish. I think it is one of the most, if not the most important technology Apple has. It enables you to string together various apps and technologies to produce incredibly robust solutions (although they might not run very quickly, ahem).
Just try to find information on AppleScript from Apple’s main site (for consumers, not developers). When you go to http://www.apple.com/applescript/ , you’ll be forwarded to a feature page for Mac OS X that tells you to go to the AppleScript website for more information. The link? http://www.apple.com/applescript/ that takes you back to the feature page you’re already on! If you go to the Resources page, and click on the link to Essential Sub-Routines (http://www.apple.com/macosx/features/guidebook/sbrt/index.html), you’ll get a 404. It has been like this for months (maybe years).
I love AppleScript (my business is built on it) but it is the neglected child that Apple desperately needs to lavish some attention upon — for both developers and users.
September 29th, 2006 at 9:04 pm
This thread is going two ways; first that AppleScript is hard for new, non-programmer users. Second, which I understood to be Daniel’s original point, adding AppleScript support to applications is difficult and poorly documented.
Regarding the difficulty of learning, AppleScript was a gateway drug for me. I first dabbled in AppleScript a dozen years ago for a fine art installation and it led me to many other languages. The beauty and frustation of AppleScript lies in its ability to scale from naive and simple scripts to robust object-based designs. Over the past several years the architectural direction of the language seems to be nudging serious authors towards Cocoa object-based models. The ability to create true AppleScript-Cocoa hybrids in AppleScript Studio is a fantastic learning opportunity to move working concepts to Cocoa.
About application support, someone should get the Omni folks to comment since they seem to be quite good at it. As an example, OmniGraffle can copy an entire document as an AppleScript construct. The first time I found that it blew my mind and left me cursing the hours I’d spent feeling my way around the dictionary and available scripts.
But I completely agree that Apple needs to be the leader here and I hope Sal is right about what’s coming. All Apple applications should ship with robust scripting support and clearly documented examples. More Apple people implementing AppleScript support will lead to better examples, more streamlined development and more voices for explaining and instructing.
September 29th, 2006 at 11:53 pm
Indeed, that is wrong. (BTW, the incorrect documentation is here and here.)
The question in my mind is which is wrong: the Foundation AppleScript-support code (which I hope really is using KVC), or the documentation. I prefer the syntax given by the documentation, so I’m hoping that the code will be updated to use that in preference to the syntax that it currently supports.
Filed a bug yet?
September 29th, 2006 at 11:59 pm
Having worked in a large number of corporations, I can probably guess what the problem is. Hypothetically, let’s say lots of resources were afforded to Applescript during development and launch.
However, over the years, those resources have been diverted to other tasks. Now there’s just a small maintenance group and not nearly enough manpower to revamp how application Applescript support is done, nor enough to produce new documentation.
The group works hard to do the best they can, but even small changes take a long time for a small group.
Am I close Sal? If so, is there anything developers can do to help champion your cause to management? On the other hand, if I’m totally off the mark (or you simply can comment on such matters), ignore me. :)
September 30th, 2006 at 1:05 am
Addressing some of your comments:
The old AppleScript website will return, but probably in a slightly different form. Until then, there is developer.apple.com/applescript/ which hosts a variety of useful articles and resources.
In regard to our plans, I’m limited what I can say in a public forum other than making it easier for developers to add AppleScript support to their applications is at the top of the list.
As to the health of AppleScript… I can assure you that AppleScript is very healthy and far from being a “maintenace group.” Many OS services and applications, including those of our largest developers, depend upon a robust and thriving AppleScript implementation. AppleScript’s importance has grown.
And finally, as to the scope of my job and influence, and how the internals of Apple work… sometimes, it’s best not to know how the sausage is made. ;-)
September 30th, 2006 at 2:10 am
Sal –
I’ve not ***yet*** tried to add scripting to my app’s (they are all for internal use; and all my extensibility has been done by loading a Cocoa bundle and exposing header files for my objects)… but in looking at what it would take, I get the feeling that it would really help if there were some sort of “verbose” or “trace” mode for OSA/Applescript, that listed (perhaps to stderr?) ***exactly*** what was happening, ***exactly*** what transforms/type-coersions/casts were taking place, and ***exactly*** where the system was looking for ***exactly*** what.
However, I ***have*** done quite a bit of scripting from the user side of things (I even once wrote a database clustering/replication system with Applescripts embedded in a set of Filemaker Pro databases – inter-server communication was performed by ginning up a HTTP post, and using the TCP OSAX!), and have a few thoughts:
>> 2) AppleScript is impossilbe (difficult) to write.
One of the reasons why working in C or Cocoa is so much easier for me (now, granted; C, Cocoa, and Applescript all do diffrent things, but the overall user experience can be compared) is because I can step thru my code and see the results of an operation in the debugger. The built-in scripting system does not have this functionality (or, at least, it does not have this in any reasonably obvious place – I never found it in what ships with the system).
So, let me reiterate the ***need*** for some way of interactively exploring the result of an operation. You should be able to issue a fragment to the system, and see ***exactly*** what you get back; and what operations you can do on it. Oh, and this should be available kinda like Spotlite or Quicksilver, for doing “quick & dirty” yet complex operations.
Also – the debugger should ***not*** be a 3rd party app. Either write your (Apple’s) own and bundle it with the system, or go buy out Script Debugger (or better yet – buy it out and hire the guys who wrote it).
The other reason why I find Cocoa so much easier to work in is the rich documentation built right into the editing environment. Perhaps some means of including links to HTML documentation pages in the scripting dictionarys along with a browser in the script editor would be helpful – then, as you are browsing the dictionary tree, you could view the documentation for an app’s (or, all app’s, for a combined tree) scriptable bits & doo-dads (to use a technical term :-) right in your editing enviroment. And to support all of this, rigorous discipline would need to be required of us developers to actually write good documentation and link it in with the dictionaries. Note that this is seperate from (and in addition to) good books.
September 30th, 2006 at 3:22 am
My heros:
Laurent Sansonetti
http://chopine.be/lrz/diary/2006-09-15_RubyCocoa-and-libffi.html
Tim Burks
http://www.rubycocoa.com/mastering-cocoa-with-ruby
September 30th, 2006 at 6:07 am
Sal,
Thanks for taking the time to comment on Daniel’s blog post here. For me personally, while I find AppleScript invaluable for controlling my applications (because there’s no other way to do it), trying to write scripts is a constant headache, even after months and years of experience writing dozens of complex scripts utilizing many different applications.
The problem is that AppleScript’s “malleable”, “flexible” language design, where the same keywords have different meanings in different scopes, where sometimes the same code will work completely differently if split onto two separate lines, where refactoring into helper functions is impossible because we can’t reliably pass objects and functions from Applications’ scripting dictionaries into our own functions and classes (handlers and script objects is I guess what I should call them), where different apps with the same function, like browsers, end up exposing absolutely completely different scripting dictionaries, because of lack of Apple leadership, etc. etc. make writing scripts a mostly trial and error process.
Extending those scripts later on is a total pain, because seemingly insignificant changes can cause everything to break catastrophically, with cryptic error messages like “can’t make object of type «class foo» into spam”, which regular users (at least those I’ve tried to advise) can’t figure out for the life of them.
And I’m really not trying to make this sound like a rant, or an attack, so please don’t take it that way. I have no doubt that you’ve met many non-programmers who have used AppleScript to make their lives easier. No one here is disputing that–I’ve helped many such users with their scripts–but I think your analysis that regular users find AppleScript much easier to learn than experienced programmers do is deeply flawed.
The problem with AppleScript is not that its syntax is different from other languages, but that it is so fragile and context-dependent. Because there are 10 ways to write any single line of AppleScript code, only 6 of which work in any particular situation, with the other 4 failing seemingly without reason, it is very difficult for anyone, programmers and regular users alike, to master the language. Write something that looks like it should work using “natural English”, get it to look quite similar to posted examples, and it will fail almost every time. AppleScript’s syntax must be learned just as much as any other programming language, and to pretend that making it English-like makes things easier to get right is fantasy. But that’s not the gripe. The gripe is that even once you’ve learned the syntax, it’s still a struggle almost every single time to get trivial scripts to work correctly.
By way of comparison, I started learning Python 3-4 years ago, having never been a programmer before that. After 2 days of reading through tutorials, I successfully wrote a 250-line program, in which the only bugs were misspellings. After fixing those, it ran, exactly as I intended it to. In AppleScript, I can’t even imagine a 250-line script being bug free without numerous iterations of a “test, bang head against wall, cry softly into pillow, test again” cycle.
We all want to love AppleScript, we really do. And it really does help me do things I can’t do in any other way, every day. But I would sure jump for joy if I could use Python or Ruby or Lua or Scheme or, or… to script my Applications.
-Jacob Rus
September 30th, 2006 at 8:41 am
Again, thanks for the great feedback. I hear what you’re saying. My comment about programers vs. non-programers perhaps is overly generalized. I meant that novices are more accepting of the quirks and work around them since they have no prior frame of reference.
Regarding Python: I’m curious, has anyone here tried the Python OSA component at:
http://www.python.org/download/mac/
Would that address your issues if it, or something similar, was included in the OS?
BTW, there’s a great page at http://aurelio.net/doc/as4pp.html translating Python and AppleScript constructs
September 30th, 2006 at 10:08 am
Certain people in this forum would be better suited with F-Script. F-script is a smalltalk based language and use the full power of OO. And of course you can use and embedded the software in Cocoa.
September 30th, 2006 at 11:04 am
I added AppleScript to my Core Data application.
I absolutely could not have done it without sample code from another developer, who graciously provided it to me.
It’s not just a matter of groking the language, I think that developers shouldn’t have to understand AppleScript in order to hook up AppleScript.
Put another way, I’ll give you some additional perspective.
I’m primarily a WebObjects developer. In WebObjects, I have to deal with two concepts.
The first is SQL. Writing SQL can be challenging for most developers because it is not a procedural language. It is a set-description-language. When you write SQL queries, you’re really describe the set of results you want. The database turns that set description into a mathematical construct, topologically morphs it to fit its best guess at the fastest way to build the set.
For those of you non SQL types, here’s an example:
select cdiv.title,prod.productName,ltd.licenseType,count(*) as cnt
from WSCompanyDiviSionProduct prod, WSLicense lice, LicenseTemplateData ltd, WSCompanyDivision cdiv
where
lice.creationDate >= TIMESTAMP ‘2006-01-01 00:00:00’ AND
lice.divisionProductId = prod.divisionProductId AND
lice.templateDataId = ltd.templateDataId
and prod.divisionid=cdiv.divisionid
group by cdiv.title, prod.productName, ltd.licenseType
If you’re not familiar with SQL, that’s going to be a bit confusing. If you’re not a SQL god, you’re going to think that those AND clauses are like IFs in Java/C. They’re not, an AND in SQL really means “set intersection”.
Why am I blathering on about SQL? Because Sal was talking about the whose in Applescript. Its the same kind of thing.
The other thing I spend a lot of time with is KVC or key value coding. I actually think of this as “data driven design”, because I did a stint as a video game programmer. In a video game, you code 10-20 mechanics, and everything after that is level-design; that is a data file that tells the arrangement of the mechanics. Sonic the Hedgehog only really has 10 mechanics, a spring, rings, etc.
So KVC is really, really, cool, because you can walk very complex object heirarchies with ease:
[email protected]
Give’s you the name of the to do that is due next. You can put that into Interface Builder as the value of a text field, and you-are-done.
Again, very like AppleScript.
So I understand what AppleScript is trying to do. I’m even using core data, and KVC.
Yet understanding how to implement it in my Frictionless program was nearly impossible. Do you know there’s not a single AppleScript->CoreData example program? The one I got from someone else even had to use an undocumented call in order to work!
For that matter, there are hints in the Apple docs that AppleScript lets you all kinds of cool stuff with KVC and ObjC. Yet I haven’t the slightest how to use it.
So its not that the language is obtuse or geared towards users. SQL is much worse then Applescript, and I can use that pretty well. In fact, I use C,Java, Perl, Python, SQL, C++, and Objective-C all the time.
It’s that its not clear to us as developers what sort of plumbing we need to install in order to get AppleScript to work.
I think this problem could actually be best solved with SAMPLE CODE. I’ve noticed that there are two types of developers:
1. Developers who prefer Sample Code.
2. Developers who prefer docs.
I’m a sample code kind of guy, because I’ve found that docs tend to gloss over the details, but Sample code has to work.
I’ve also noticed that any Apple technology with lame sample code doesn’t get adopted very fast.
So Sal, if you want TEN TIMES the acceptance rate of AppleScript, publish some sample code that:
1. Fully publishes a complex core-data model.
2. Demostrates KVC on the non-core-data objects.
For instance, I learned accidentally that for AppleScript you have to do a bunch of
stuff in your accessor methods.
3. Has good comments with background.
Pierce
September 30th, 2006 at 12:04 pm
One of the success stories for OmniOutliner 3 was that we added tons of automated system tests using AppleScript with a little Ruby driver. This solves several problems, 1) it makes it easier to write tests, so they get written, 2) the AppleScript support itself gets tested and 3) the AppleScript support becomes very extensive since otherwise there are parts of the core functionality that can’t get tested. We use all this for testing correctness, wall-clock performance and memory usage. As a side benefit, it becomes easy to write Automator actions, though we aren’t seeing a huge demand for these from our users.
Still, we have a love/hate relationship with AppleScript. It’s obviously a huge benefit for OmniOutliner to have great scripting support since it allows it to act as a platform for visionary folks like Ethan of kGTD fame. But at the same time, we joke around the office that AppleScript is a “read-only language”. If you see a script, you can read it and probably figure out what it does, but trying to figure out *how* to do some operation can be really frustrating.
On top of that, the error/checking & reporting facilities in the script suite parsers and the Cocoa Scripting interface really need help. They have gotten much better over the years, but there is still what I call the “No” problem. We’ll add some feature to an app and try to test it via AppleScript and all we get back is a blank failure. It could be that we’ve introduced some typo into our sdef, or that we’ve missed some KVC method, or any number of other things. Internally we have a tool for tracing all the ObjC message sends within an application. I primarily use this when working on scripting support to help solve the “No” problem — it isn’t ideal, but at least I can see what the scripting support was trying to do and then make some guesses about why it failed. It seems like a warning sign when you have to reverse engineer stuff to track down trivial errors. I try to log Radars in these cases, but I’m sure I’ve forgotten a few.
There are some other time sucking holes — for a long time marking something read-only had no effect. So rather than being able to use normal KVC methods ‘foo’ and ‘setFoo:’, we have started using ‘scriptFoo’ and only implementing the getter (this may or may not be fixed in the latest builds, I’ve been stuck in 10.3 compatibility mode for a while). Another gigantic whole is ‘move’. In OO3 I went to a lot of effort to make sure the user couldn’t destroy their document trivially by accidentally writing a script that does ‘move first column to end of rows of document’. ‘move’, ‘set’ and any command with a destination location specifier need to have automatic type checking based on the metadata in the script suite. I find it pretty annoying that in every scripting KVC method I have to add type checking to make sure that the user didn’t type something silly like ‘set color of SomeObject to 3’. In the case of CoreData, AppleScript seems to have no notion of unordered relationships and CoreData has no notion or ordered ones. I haven’t decided how I’m going to solve that issue yet, but I’m guessing it’ll be ugly (hints welcome! :)
I’ve started on adding scripting to OmniFocus, even at this early stage, so that we’ll get this deep level of scripting support by having it drive our tests. But, of course, I ran into the ‘make new’ problem NSManagedObject problem that never got answered on the applescript-implementors list, except by 3rd parties using undocumented API (logged a couple Radars related to that).
Again, we are getting a lot of benefit from AppleScript, but I’d agree that it takes way more effort to implement than it should.
September 30th, 2006 at 12:13 pm
I’m not a developer by any means but I can code and I learned to code by learning javascript in the late nineties. Why Javascript? The web of course. I just started cutting up other peoples code without even understanding what it was and I was off to the races. I got better at it and later mastered it. I could never get the same results with applescript even though I tried and tried and tried.
As more web sites used javascript in more clever ways you could see that the entire web dev community was getting better and the scripts were better. The number of decent scripters rose dramatically. Flash dropped Actionscript 1 for javascript and things really took off for Flash. Adobe added javascript support to their apps and I happily used that knowledge to write all kinds of useful scripts for Illustrator and photoshop. Scripts that I use on a daily basis. Scripts that I cannot write with applescript.
Then OS X created web-kit and Safari and then Widgets and I thought for a moment that decent scripting was coming to OS X and it would be done with the user-centered Apple philosophy that I love so much. And I’m still waiting…
I’m sure I’m going to get comments suggesting I get OSA Javascript and be happy with it. But I want Apple to take it to the next level. I want more than four books with examples. I want Apple to leverage the millions of scripters and sites and hundreds of javascript books.
I really hope Apple re-evaluates Applescript and realizes the natural language syntax isn’t working. It was a nice though a decade ago but it’s 2006 and the web has shown the way. Just compare the growth of javascript users with Applescript users and realize that there is an untapped resource here.
September 30th, 2006 at 2:12 pm
Some thing everyone is missing so far, even Sal to some extent.
It’s actually possible, using the Open Scripting Architecture to do everything in AppleScript via Python/Perl. Someone has written an OSA accessor for both Perl and Python.
So there is AppleScript the language, which builds on OSA, to provide scripting.
As Cocoa developers, we need to provide scripting, not AppleScript…
The catch: 99% of the scripts out there are written in AppleScript, and all of the OSA plugins need examples.
Perhaps if someone who is good at AppleScript could tell the rest of us how to use those plugins, it would help the rest of us get it.
Here is a link to get y’all started:
http://appscript.sourceforge.net/
Here’s some sample code:
get paragraph 1 of document 1 of application “TextEdit” (AppleScript)
app(‘TextEdit’).documents[1].paragraphs[1].get() (Python)
As an OO programmer, this tells me that OSA is going to ask me for a “documents” collection.
Then its going to ask me for item 1 of documents.
It’s going to ask that for the paragraphs collection.
It’s going to ask the collection for item 1 of that.
October 1st, 2006 at 12:54 am
Sal wrote: “Ray Robertson and Shane Stanley are incredible instructors and their sessions have become nust-attend events.”
That should be “nuts attend”, of course…
October 1st, 2006 at 11:13 am
Also, in the vein of AppScript, is http://rubyforge.org/projects/rubyosa/ — this a project that I find very exciting. We are using the dated RubyAEOSA library now, but I’m looking forward to the new RubyOSA package.
While I think it will be very productive for me personally to use RubyOSA, I’m not sure I think that random users will use it — hopefully, though :)
October 1st, 2006 at 12:24 pm
I’ve not made my programs AppleScriptable (mainly because no-one ever asked to), but I do have some experience creating AppleScripts. Alas, each time I turn to AppleScript to get a job done, I end up extremely frustrated. I don’t think this is caused by the design of the language itself, rather the problems I run into are due to a lack of documentation and inconsistent ‘frameworks’.
When Sal Soghoian states that AppleScript is still very much a priority within Apple, that is sort of comforting. On the other hand, how can this be when at the same time most recent official AppleScript Language Guide available on apple.com is from 1999 (nineteen-ninety-nine) and covers AppleScript 1.3.7? I find these facts hard to reconcile. It also makes me wonder if there exists an up-to-date description / specification of modern day AppleScript at all… In practice, finding information on AppleScript facilities involves a lot of hunting and gathering on Google. Providing information about AS seems to have become mostly a ‘third party opportunity’.
Another thing inherent to AppleScript (and maybe to application scripting systems in general) is that you need to learn a new ‘framework’ (dictionary) for each program you want to script. Worse, each application uses its own terminology common concepts, and the organization of dictionaries can be all over the map. Applying what you learned while scripting app A to your next project scripting app B is very difficult.
Lastly, I find the information provided in dictionaries to be extremely limited. Especially since most of the time the dictionary is all you’ve got to go on. There are no extra comments or explanations, there is no example code. If you look at the dictionary entry for a command it often doesn’t specify the accepted input for the command or the possible errors conditions. It is even not uncommon to find dictionary entries for properties or commands that aren’t implemented at all (or yet). All this turns AppleScripting into a trial and error exercise, much more so than programming in any other language.
As a result I tend to avoid AppleScript as much as possible.
October 1st, 2006 at 12:27 pm
Sal, I think the problem with programmers finding AppleScript frustrating goes beyond the syntax.
While the verbosity, lack of things like the ability to sort a list, match a string against a regular expression, etc. can be irritating, these are things easy to grasp.
What”™s not easy to grasp is the AppleScript gotchas which are inconsistent and I think partly due to bugs:
For example if I do something like
tell app "Adium" to return first chat whose chat name is "Allan Odgaard"(assuming there is such) then I get nothing back. But if instead ofreturnItellthis (void) result tosend message, a message is sent. So I can tell this reference things to do directly, but I can”™t let a variable store the reference. Often though the opposite has been the case, i.e. that I had to store the reference in a variable to further query it.I have seen these things over and over again, but I have never understood them, and that is adding to the frustration, and makes for an awful trial-and-error environment. For the records, I do understand the more well known gotchas, such as naming a variable
trackin an iTunes context etc.The scripting dictionaries, with their flat view of all exposed classes of a given application can also be daunting, and applications often dictate a less than usual workflow because of the extreme reliance on OO. For example to create a folder you instantiate a folder object (with properties) and provide it with the result of a query (more or less) to let it know where this folder should be located on disk.
These “workflows” are sometimes hard to discover on your own, which means it is easy to get stuck.
I have frequently been in the situation where I still wasn”™t sure what the proper workflow was, got an idea, then had to struggle for five minutes just to get my new approach to build, and then figure out that this was also a dead end.
So, would it help to include AppScript or another OSA component with the OS? Probably some, if it doesn”™t feel too awkward to use the scripting dictionaries from the alternate language, but it is IMHO only solving a little part of the problem.
October 1st, 2006 at 6:50 pm
I think that’s actually an Adium bug. (For those who don’t know, I’m an Adium developer.) We haven’t really expended any effort on getting AS working properly in Adium. I’ve made a sdef, but we need to pound on the Obj-C implementation behind it before Adium works more properly.
October 2nd, 2006 at 12:39 am
>> I think that”™s actually an Adium bug.
…Which is clear as mud to the poor scripter because a) you can’t see ***exactly*** what’s there; and b) even if you could, getting a refrence and doing an action directly are two diffrent things despite the fact that conceptually one must be able to refer to somethign to tell it to do something (er, at least I think they are – I need to read up on this more). And of course, the reason that the locality of operation changes between the two is because AppleScript (and OSA in general) is based on IAC, and not a plug-in archetecture…
October 2nd, 2006 at 1:46 am
Sal wrote:
I’d go with something like:
find ~/Documents -type f -name "*Smith Project*.doc" -exec cp {} /Volumes/Backup \;…although you won’t get the Finder warning you about files with the same name that way. :)
October 2nd, 2006 at 1:49 am
Or if you wanted something closer to the original syntax:
cp `find ~/Documents -type f -name "*Smith Project*.doc"` /Volumes/BackupI’m not sure how the shell (or
cp) handles it if the number of files is large, though.October 2nd, 2006 at 8:19 am
Oh my, I can only imagine the uproar from the prepress industry if Applescript was abandoned for something else. The number of man hours that have been put into Applescript workflows within the industry is simply astonishing. I have personal experience of this both as an application developer looking to add Applescript support to applications and as a user trying to script Applescript workflows. I wholehartedly agree that Applescript needs an overhaul but backwards compatibility is a must.
October 2nd, 2006 at 1:51 pm
Hi:
As a non-programmer bridging the design and database worlds in a production environment, I NEED APPLESCRIPT! Why? Because I don’t think with arbitrary symbols or in dot syntax. I am a logical fellow, I think in English, and I don’t have a degree in Computer Science or Mathematics.
But I do know what I want, and I know how I want it to happen. Applescript is the only thing that lets me automate my processes easily, quickly, and flexibly.
Thanks,
guy
October 6th, 2006 at 10:35 am
Peter Hosey: At first I was certain that it was an Adium bug, but I later ran into a very similar problem with another application where exactly the same workaround fixed it.
October 11th, 2006 at 1:01 pm
Yes, AppleScript implementation is too hard.
Since pre-history (CodeWarrior PowerPlant) it has always been so.
When OS X was first released it came with a cool program called Interface Builder. Wow, I could wire up a GUI in no time! I told Apple, this is the way you should do AppleScript implementation. Let me wire up a Scripting UI the way I can wire up a Graphical UI.
Well, we’ve since gotten bindings, Core Data wiring of the model, KVC (all great stuff), but still no way to wire up AppleScript support. (I don’t doubt that the problem is hard but solving it could mean an explosion of AppleScript support by developers.)
Suggestions to Apple:
– offer an IB-style way to implement AppleScript.
– offer extensive sample code illustrating anything that can’t be wired up.
– offer great debugging for the implementer and for the scripter (even if it lifts the curtain a little on opaque AppleEvent structures).
– continue to encourage English-like termininology that is as consistent across applications as possible.
– make Cocoa applications automatically recordable with respect to moving, closing and minimizing windows, etc. This means taking some of the recordability implementation for the Finder and moving it into NSApplication or some other Cocoa implementation class. Then we can get base recordability for free and implementers need only make their specialized events recordable.
– fix bugs.
Suggestions to implementers:
– always include example scripts. I stick mine right in the Help documentation that the user can pull down from the Help menu.
– make your terminology as English like as possible and as consistent with other apps as possible.
– make your custom AppleEvents recordable (without trying to do what should be Apple’s job of making moving windows, etc. recordable.) I gave a simple example of how to do that here: http://lists.apple.com/archives/applescript-implementors/2006/Mar/msg00024.html where I construct the AppleEvent in Cocoa and use a single Carbon call to send it. Another example can be found here: http://lists.apple.com/archives/applescript-implementors/2004/Nov/msg00014.html
– communicate with scripters and find out what they really want. The whole point of AppleScript implementation is for the scripter. We must listen.
Finally, I want to send huge thanks to all who contribute to the AppleScript implementers mailing list. The Apple engineers who contribute there are enormously helpful and do so on their own time! If they haven’t been around as much lately, we can only hope that they are very busy and hard at work making AppleScript better.
October 15th, 2006 at 2:22 pm
Very interesting discussion here. How wonderful to see Sal responding to it so positively.
I am a scripter rather than an ‘implementor’ but I have worked closely with various developers as they struggled to implement AppleScript in their apps. It has always been a world of pain.
Here are a few points I’d like to raise:
AppleScript implementation needs to be a ‘no-brainer’, and preferably something which happens ‘automagically’ in Xcode (at least for Cocoa apps).
Apple should lead the way with its own AppleScript implementations. There are a few good ones (e.g. QuickTime Player), some really bizarre ones (TextEdit) and plenty of quirky, abandoned monsters (e.g. AppleWorks).
Far too many important (and expensive) Apple apps (e.g. pro audio/video) have no AppleScript support at all. This is a very poor show. The pros that need this stuff as much as the amateurs!
Some of Apple’s own scriptable apps go directly against the design guidelines described in the famous TN2106. (e.g. iTunes). This should be fixed, even if it breaks older scripts. I think at least we should hear some good excuses for why the iTunes dictionary has both ‘updatePodcast’ and ‘updateAllPodcasts’. (And what’s with the ‘camel case’? What a howler!).
Applescript is deliberately designed as a ‘little’ language (i.e. it’s mostly just the core syntax, with the API provided by app developers), but there are a small handful of half-implemented ‘core’ features that for over a decade have shown tremendous promise but never been fleshed out.
I am thinking (for example) of the ability to use the filter reference form (‘whose’ clauses) on scripters’ own lists, records and objects, or the ability to work with multiple text item delimiters. These things would be a tremendous boost to both scripters and implementors and while such features may not sell any additional iMacs, they might shift a few more high-end machines, and besides surely not rocket science. These features aren’t just scripter fantasies, they are tantalizingly half-implemented. Something about spoiling the ship for a ha’pe’th of tar.
And Sal, or anyone at Apple, can we PLEASE get the Cocoa ‘default’ scripting implementation tidied up and properly documented. (Mostly the ‘Text Suite’) It promises much but in most cases it’s really not doing what anyone wants. Some developers even have the gall to say their app is ‘AppleScriptable’ when in fact it merely has the ‘default’ cocoa dictionary, which is often not even connected up to anything. Apple could take steps to make this less ‘acceptable’.
I respect the need to keep details of unreleased products but we still need to be reminded that ‘something is happening’. That means decent, up-to-date docs, example projects which compile and run ‘out of the box’, and active participation from Apple Engineers on the mailing lists and other online fora. Thank you, Sal, for participating here. It means a great deal more than you may imagine!
And when are we going to get AppleScript on the iPod? ;)
October 16th, 2006 at 8:10 am
And now there also is a Ruby/Apple-event bridge by Hamish Sanderson:
http://rb-appscript.rubyforge.org/
It’s already used by the safariwatir project, btw!
October 23rd, 2006 at 5:25 pm
Sal wrote:
Regarding Python: I”™m curious, has anyone here tried the Python OSA component at:
http://www.python.org/download/mac/
Would that address your issues if it, or something similar, was included in the OS?
##############################
############################
Yes!! Please!!!
I saw this statement and I jumped for joy.
Something like appscript and Pyobjc that you guys would fold into the os and mantain, not instead, but in addition to AS. That would provide the best of both worlds for everyone, AS for the novice and for those expert users with legacy needs and Python for those of us who prefer a more traditional programming aproach . But documentation and examples are paramount. How about getting that in 10.5 ?
Best regards,
Steven Quinones-Colon
January 25th, 2007 at 11:36 am
I “cut my teeth” in programming using AppleScript to automate prepress workflows for the newspaper industry, and have since added perl and am currently adding Cocoa/Obj-C to my toolset. While agreeing with the frustration levels many in this post have experienced in writing AppleScript, and understanding the headache it must be to add AS support to Cocoa applications (I have yet to cross that bridge), I see AppleScript as a continued necessity for good Mac apps. Why? Because it is an end-user, doesn’t-have-to-know-computer-programming, language for “the rest of us.” If I had not had AppleScript as a gentler learning curve into a broader world, I don’t think I would have bothered trying. And I know of many others whose experience was similar. The truth is, abstracting the complexity of controlling applications to the end-user level necessitates a language that doesn’t require a complete shift in syntax rules from what the average Joe already knows — i.e., English.
I do agree that AppleScript needs some work. Better list manipulation, a regular-expression syntax, stronger support in the finder, more recordability, and a LOT more speed are things that I would give my right arm for. As for standardization, I think much of the problem is the result of third-party developers not putting in the effort to compare their applications’ AS behavior with existing applications that have rich, intuitive scripting dictionaries. It would help if Apple did some work on standardizing and documenting the way dictionaries should be written, but in a somewhat obscure way standards do exist. Back when I started, QuarkXPress was considered a standard bearer in scriptability (although InDesign has far surpassed it), and there are many apps today (thanks, OmniGroup, Adobe, and others!) that other developers should be looking to in order to see ways most AppleScripters expect their precious language to be implemented. Perhaps we need an Indie AS Implementation guide similar to the Indie HIG.
I have two suggestions, from an end-user’s perspective, to developers when contemplating adding AppleScript support:
1) Expose as much of the functionality of your application as possible to the OSA engine. Please don’t leave needed functions “in the dark” from an AS point of view. A good bad example of this is the more recent iterations of QuarkXPress (I’m thinking of 6.5) where they added a wonderful PDF Export ability but did not make that function scriptable, forcing us AppleScripters to continue to rely on printing or exporting as EPS, and then passing the resulting file into Distiller. And anyone who has ever tried scripting QuarkXPress printing can attest that it is wildly unreliable at best.
2) Use terminology in your dictionary that mimics the same concepts reflected in the GUI. In other words, write your AppleScript syntax in such a way that if I understand how to do a particular process from the GUI, I can easily map that knowledge to the identical feature in AppleScript. This does not mean scripting button clicks and palette openings, but if I want to, say, create a 2″ by 2″ text box 1″ from the side of the page and 3/4″ from the top, I shouldn’t have to learn an obtuse reference form to do so. AppleScript implementation should be as elegant and well-thought-out as the GUI.
Lastly, I should note that I strongly disagree with posters suggesting this or that framework for adding support for specific scripting languages to applications. The LAST thing we need is for every developer to decide what scripting language their application is going to use and essentially isolate it from the rest of the scripted-application world. One of the most powerful features of the OSA is its ability to pass instructions from one application to the next. Most AppleScript workflows I and others have created use at LEAST two applications. Even if you prefer to use a different scripting language yourself, please take the time to implement the OSA in your app. This way, using an OSA component, anyone can write their scripts in whatever language they prefer. And us AppleScripters can continue on in our blissful, English-like world.
March 13th, 2007 at 5:42 pm
I just logged in at http://developer.apple.com and as of 3-5-2007 they have a few new sample AppleScript apps with code, complete with detailed(?) readmes. I haven’t sat down with any yet, but they look promising… at least for an AppleScript newbie like me.