It’s the beginning of the month, which means it’s invoice time for me. I very rarely send out paper invoices. These days most companies are hip enough to have addresses like AccountsPayable@<company>.com. So my low-volume invoicing mechanism involves a Pages templates document, and a little manual copy and paste from my web-based hour-tracking system.

One company I work for makes things “easier” by requiring that I submit invoices through a nasty web invoicing system. Just about everything about this site is wrong. To add insult to injury, they charge me, the “supplier” for the privilege of stumbling my way through their ridiculous invoicing system. To be fair I haven’t run out of “free credits” that I got when I first signed up, but I’m really not looking forward to the day when I have to pay for this abuse.

Among the ridiculous things I must endure every month as I fill out the invoice data, is the selection of “units” for my “hourly work” line item. It’s always “hours,” just like every other professional consulting service is almost always “hours.” But hours aren’t special to this company – they’re just another item smack-dab in the middle of the world’s most comprehensive list of invoicing units, and I have to scroll down to the middle of it every time:

Please Select From our Ridiculously Stupid List of Possible Units1,000 10 10 Metres 10 x 0.5ml 10 x 0.6ml 10 x 0.75ml 10 x 1ml 10,000 10-Pack 100 100 Metres 100-Pack 1000L 100g 100g 100L 100ml 10L 10ml 10ml 125Kg 1Kg 1L 2.5L 200L 20ml 250g 250ml 25g 25Kg 25L 25ml 25ml 4 Week 400L IBC 4L 4x48L 5 x 10ml 5 x 5ml 50 Pound Bag 50-Pack 500g 500ml 50Kg 5g 5L 5ml 6 Pack 9 x 17L 9 x 20L 9x15L 9x20L Acre Assembly Bag Bale Ball Barrel (US) (petroleum etc.) Barrel – Imperial Base Basket Batch Board Feet Bottle Box Box of 12 Box of 6 Bucket Bundle Bushel Can Canister Carload Carton Case Catchweight Centigram Centilitre Centimetre Coil Container Copies Crate Cubic centimetre Cubic decimetre Cubic Feet Cubic foot Cubic inch Cubic Inches Cubic metre Cubic metre (net) Cubic millimetre Cubic yard Cycle Cylinder Day Decilitre Decimetre Degree Dollars (US) Double Centre Dozen Drum Each Fifteen kg drum Fiftyfive gallon (US) drum Fluid ounce (UK) Fluid ounce (US) Foot Gallon (UK) Gallon (US) Gram Gross Hour Hucom Hundred Hundred cubic feet Hundred cubic metre Hundred feet Hundred kilogram Hundred Litres Hundred Pounds IBC Inch Inches Inner Item Items Jumbo Kilogram Kilometer Kilometre Kilos per Drum (wet) Kilowatt Kilowatt Hour Kilowatt Hour Kit Kit of 3 Large Length License Link Litre Load Load Lot Lump Sum Mega Watt Metre Metre cubed Mile (statute mile) Milligram milliliter Millimetre Million Minute Minutes Month Ounce Ounce Outer Overtime Hours Pack Packet Pages Pair Pallet (lift) Pallet/unit load Percent Perf. unit Piece Pint (UK) Pint (US) Pound Pounds Quart Quarter Quarter Hour Quire Rack Ream Reel Ring Role Roll Second Service Set Sheet SITA Skip Spool Square centimetre Square foot Square inch Square kilometre Square metre Square yard Synthetic Latex Dry Kilos Synthetic Latex Dry Tonnes Synthetic Latex Wet Kilos Synthetic Latex Wet Tonnes Tank Therms Thousand Thousand Dosages Thousand Feet Thousand Sq. Metre Tin Ton Ton (UK) or longton (US) Tonne (metric ton) Truckload Truckload Tube Twenty-Four Unit Vial Wallet Week Wholesaler Wraps Yard Year

I know – it’s funny! Like – Brazil funny. The first time, and then you start to loathe invoices. This should be a happy time, when you think “I’m going to get paid.” Instead I think, “I wonder if I’ll survive the invoicing system this month.” Maybe one month I’ll just charge them for “27 50-Pound Bags of Mac Consulting” and see what happens.

Other fun aspects of this system include the classic “works only if you click the Submit button” bug. Hitting return submits, but submits with bogus data. This allows me to regularly observe the classless manner in which the system conveys SQL database errors to me! “Oh! ’16:Can not init Service’, that explains it!”

This is a perfect follow-up to my Verizon post. It’s another example where a company must be making hundreds of thousands – no millions – of dollars peddling crap. On the bright side, when you swim in a sea of useless and buggy software, it’s easy to feel good about putting an honest day’s work into quality programming, and working with people who share these values.

Since moving to the Northeast I’ve been a customer of Verizon for my telephone and internet service. Being a customer of Verizon is basically the same as being a customer of SBC, as I used to be in San Francisco. Things basically work most of the time, but any time you need to change anything, install anything, pay anything, get anything fixed, etc., you better hold your breath and pray for the best.

Among the sad offerings at Verizon is its infuriating web page. Back in San Francisco I remember being upgraded to “SBC Yahoo!,” and that when I switched to the Yahoo-branded service, at least the web interface for things like billing and services was a bit better.

So when I got a letter in the mail from Verizon offering a “free upgrade” to Verizon Yahoo!, I thought I had nothing to lose.

Grr … it makes me download and install custom branded software. Apparently this is part of the “upgrade.” If you want to play in the Yahoo home page, you have to let it run a scary installer-type program that connects back to the Internet and displays clunky HTML status while it has its way with your Mail and Safari settings.

I should have backed up everything, but I was feeling lazy I guess. Knowing the Telco companies, I should never be lazy. I let it do its thing all over my computer, and when it finally relaunched Safari: surprise! no more bookmarks. And when I say “no” bookmarks, I mean it. It didn’t even stuff a bunch of Verizon marketing bookmarks in there to pretty up the browser. Just a big gaping hole where my bookmarks (in the bookmark bar and menu) used to be.

Of course, it also replaced my home page setting in Safari. To this! Yes, the garbage you see on your screen is what I see on mine, too. Lucky us – living the high life with Yahoo and Verizon.

I was relieved to learn that whoever programmed this little monster was at least concerned enough about their work to make a backup of the existing Bookmarks file. Not that this is acceptable for the 99% of users who won’t freaking know to look in ~/Library/Safari for the lost Bookmarks. But for anybody unfortunate enough to go through this upgrade and suffer my fate, maybe you’ll get “lucky” like I did and find your bookmarks thus preserved.

You can imagine that by this point I was pretty scared to relaunch Mail.app. Another “favor” the Verizon installer did was to reconfigure my Verizon email account in Mail so that it would go through a yahoo branded SMTP/POP server. I could easily imagine this programmer clobbering all of my accounts, assuming that the user “surely only uses Verizon email.”

I’m happy to say that my pessimism in this regard was ill-founded. Whether by skill or dumb-luck, the Verizon installer managed not to obliterate my Mail settings.

Now I’m just hoping it didn’t do anything else that I haven’t noticed yet. Keeping my fingers crossed.

Update By looking carefully through the “installer” program, I was able to find quite a bit of information. I know the name of the person who wrote this code, which I will omit to spare him the public humiliation. I know the name of the company, which I will omit to spare myself from legal harassment (though we should be suing them!), and I know the exact cause of the bug.

The installer is based on a collection of sloppy AppleScripts, and the author was kind enough to leave the offending AppleScript’s source code intact in the shipping image. I know the answer to my question above: was it skill or dumb-luck that caused my Bookmarks to be backed up? It was definitely dumb-luck. In fact, it was a sloppy piece of programming that led to that remnant being left around.

The Verizon installer attempts to inject its bookmarks into the user’s Bookmarks.plist file by:

1. Making a copy of the existing Bookmarks.plist file.
2. Erasing the original Bookmarks.plist.
3. Iterating through the original bookmarks one line at a time, echoing the line back out to the original Bookmarks.plist.
4. Sneaking its own items in when it feels the time is right.

So what happens when one little thing goes wrong in this nasty mess? You lose your Bookmarks.

This whole thing involves about a hundred bazillion dangerous, presumptuous “do shell script” calls. Instead of using AppleScript’s built-in file access routines, the author has chosen to use shell scripts and the “echo” command to perform all of his writes. For all of his reads, he uses a combination of the UNIX “cat” command with the “sed” and “tail” command. For each line of the original Bookmarks line to be read, he cats the *entire* file and pipes the result through sed and then tail. Seeing this kind of stuff makes me wonder whether half of my hard drive was deleted and I just haven’t found out yet.

So what caused the script to fail, and my bookmarks to disappear? The following line of AppleScript code:

do shell script "cd;/bin/echo '" & theLine & "' >> Library/Safari/Bookmarks.plist"

By assuming that “theLine” will never contain any characters that cause unexpected, perhaps dangerous side-effects to occur, the author has made his disastrous error. Among my Bookmarks is a javascript “bookmarklet” which contains, as part of its “URL” a lot of interesting characters. It’s a JavaScript! The author of the script probably felt pretty comfortable about putting single quotes around “theLine.” By doing so he essentially protected himself from lots of the hairy problems that could arise from special characters like ampersand and tilde popping into the command line. But what about the single quotes in my Bookmark? As soon as the first single quote from the Javascript bookmarklet is hit, the party’s over. As the rest of the string is processed, it’s a crap shoot whether we’re in “safe” mode or not. When he hits the next single quote in my Bookmarklet, *phew* we’re safe again. But soon there’s another one and we’re in danger. It so happens that during one of these unsafe stretches, the script runs into an ampersand. What does the shell do when it encounters an ampersand? It forks the process out and continues accepting commands. For instance, go the Terminal and type:

ls & ls

See how ls gets called twice – sort of concurrently? So every time this piece of work from Verizon gets to an unprotected ampersand in my Bookmark file, it ends the current command and starts up another one. This is what the AppleScript failure looks like when I narrow it down to the offending line:

sh: line 1: amp: command not found
sh: line 1: amp: command not found
sh: line 1: amp: command not found
sh: line 1: amp: command not found
sh: line 1: +(new%20Date).getTime();var%20request=
new%20XMLHttpRequest();request.open(POST,path,true);
request.setRequestHeader(Method,POST%20+path+%20HTTP/1.1);
request.setRequestHeader(Content-Type,application/x-www-
form-urlencoded);request.send(post);};: No such file or directory

It’s damn lucky that “amp” isn’t an alias for “format my hard drive.” Damn lucky for me, and damn lucky for this careless Verizon consultant. Hopefully every other person in the world this happens to will be able to share my relatively lucky escape.

I would try to inform Verizon about this, but to be honest I don’t think I could get through to them if I tried. If anybody in a position to be embarrassed by this happens to be reading, maybe you can help get this brought to the right person’s attention.

Update 2 It’s a lucky coincidence that the data being parsed in this case is fairly well-limited to what Safari will allow you to put in a bookmark. Since Safari converts all spaces in the bookmark (even if it’s inside a javascript block) into an escaped space character, there are no actual spaces in the bookmark text by the time Verizon’s installer gets to it. This is very lucky, because it means a bookmark like “javascript:alert(‘&quot;rm -rf ~; echo ‘)” won’t actually delete your entire home directory. Instead, the shell says something like “command not found: rm%20-rf%20~”.

This lucky break for us doesn’t make the installer any less frightening – it still kills your bookmarks, or at best, just corrupts all your bookmarklets. And I would be surprised if this is the only bug in the installer – I certainly advise any Mac user to avoid running this at all costs. If you do need to run it, I would do a full backup of your computer before doing so.

Perhaps my New-Years resolution has been to tie up loose ends in the personal coding projects department. Having recently finished an update to FastScripts, and putting Clarion 2.0 in the “beta cooker” for a little while, I turned my attentions to a little piece of work I use specifically for practicing yoga, but which should have practical applications across a wide range of timed physical activities.

FlexTime makes it easy to put together an ordered “routine” of separate activities, each of which may last for an arbitrary length of time and possess an arbitrary number of periodic cues. While FlexTime runs through the routine, you are cued by “activity start” and “periodic repeat” cue sounds, each customizable for the particular activity (currently limited to standard alert sounds). For example, I find it useful in yoga to hear a certain sound for the start of a pose and then for the various minor changes within that pose.

FlexTime 1.0 Beta is a very simple application with a few rough edges, but I hope you’ll find it useful for something. If you have feedback about how it could be made more appealing, please let me know.

I anticipate enhancing the complexity of the “routine” concept as well as the complexity of individual “activities” within the routine. Also high on the list of priorities is a feature to “export” the routine as a plain audio file so it could be taken to the gym on your iPod, for instance.

The download archive contains two example documents. If you have other ideas for novel uses, please put together an example document and send it to me!

One of the aspects of being a professional software developer that is a real grind is the tedious, careful double-checking we must do in order to reduce the odds that our work will lead to some major metropolitan area bursting into flames.

Double-checking means diffing.

Diffing is hard. Even with somewhat friendly tools at our disposal, it’s a tedious, annoying pain in the butt. It doesn’t help that diff tools don’t seem to have improved much in the past 10 years, and are still obnoxiously finicky when it comes to issues like line-endings, encodings, etc.

But the worst part of diff tasks, as with most things, is simply getting started. I am particularly averse to working on things that are hard to start. While diffing from Xcode can be really easy just hit a keyboard shortcut to see a comparison with the last checked-in copy, for instance), diffing arbitrary files in the Finder usually follows a pattern more like this:

1. Launch a diff-comparison tool like BBEdit or FileMerge.
2. Navigate to File A in the Finder. Drag it to the diff-tool.
3. Navigate to File B in the Finder. Drag it to the diff-tool.
4. Tell the diff-tool you’re ready to compare!

Well, that’s too many steps for me. So I wrote an AppleScript to apply some rather straightforward logic to the selected items in the Finder, and open your favorite diff-tool for you with the comparison process started. The logic of the script is simple enough:

1. Are there two items selected in front window? Compare them.
2. Is there exactly one item selected in the front window and one item selected in the second-to-front window? Compare them.

Here, “compare them” means pull an Xcode-like move and launch the “diff opener tool” of your choice. I’ve included in the script the paths to two “secret tools” used by Xcode for opening with BBEdit and FileMerge. If you want to use a different tool you’ll have to find a way of invoking it directly with two full paths, or write a helper tool that can do the trick for you.

This simple script makes a great Cmd-Opt-C shortcut in the Finder. It’s the same shortcut I use for “compare with latest” in Xcode. Now when I see two files in the Finder that I want to compare, I just select them both and trigger the script. Wham!

Feel free to download the script: Compare Files. Let me know if you have any suggestions for improving it.