Out Of My Access Control
October 30th, 2007I’ve been testing Leopard for many months, but not until updating to the final version did I get bit by a strange side-effect of a new default behavior in 10.5, relating to Access Control Lists (ACLs).
My habit for migrating to new testing releases of Leopard, and ultimately to the final release, has been to keep my old home directory, but to point my new operating system at it by defining a custom home directory location. This works great, especially since I can install lots of handy things like system preference panes once in my home directory and have them at my disposal forever without reinstalling them.
I also have a habit of keeping a lot of my testing resources for MarsEdit and other products set up in my home directory. As soon as I upgrade to a new system, I fine-tune a couple settings and I’m off and running again with my “usual development environment.”
As you might imagine, one of the important resources to testing MarsEdit is a local Apache web server configured to serve a variety of test blogs. It makes it a lot easier for me to test when users report a problem against a particular system, if I can just hook up to my own private replica. I don’t even need to have an internet connection, because the blog server and client are all running right on my Mac. It’s pretty super!
Until It Grinds To A Halt
I noticed after upgrading to the final release of Leopard that none of my test servers were working as expected. Navigating to a test blog via the browser yielded a mysterious permissions error. What on earth could it be? Apache worked perfectly before, and now is failing with a cryptic error. The Apache error log just repeats this mysterious failure:
Symbolic link not allowed or link target not accessible: /Library/WebServer/Documents
I’ll admit this message is not completely out of left field, as I do use a symbolic link to identify my web server folder. I run a script after updating the OS, to put many common facilities “back into order.” Among other things it wipes out the usual directory for “Web Sharing” and replaces it with a link to my testing stuff, stored inside the Documents folder of my home directory. This works brilliantly because I’m assured the stuff I care about is safely stowed in my home directory, and changes I make will get saved even if I wipe out the OS.
So why is this link suddenly failing on Leopard? It’s perplexing. It worked in Tiger, and it worked in all of my testing up until the final release of Leopard. I thought the link must be bad. No. Perhaps I’ve screwed up my Apache config? No, it’s exactly the same. I don’t know if I should be angry or embarrassed about the fact that it took me 2 hours of debugging to figure out the root cause of the problem, but I’ll spare you all the gory details and cut to the chase.
Leopard Alters Your Home Directory
Leopard 10.5 seems to have gone out of its way to alter the permissions of key directories in my home directory, adding an explicit “can’t delete” rule to Documents and several other of the special folders. I spotted a clue when I did an ls from the terminal, and noticed an extra “+” after the permissions, which I learned means it has an Access Control List associated with it. I then learned about the “-e” option to examine the ACL settings explicitly, for example “ls -led Documents” yields:
drwxr-xr-x+ 103 daniel 501 3502 Oct 26 17:23 Documents 0: group:everyone deny delete
See that line? It means nobody can delete the folder. The worst thing? This particular ACL setting makes no visible impact on the permissions settings that are editable via the Finder. It just says “Everyone: Read Only.” Apparently the rule to disallow deletion is not supported by the UI (except to the extent that if you try to trash one of these altered folders, you will be refused). So if you don’t figure out how to use ls with the appropriate flags, you’ll never notice the change. And if you don’t learn how to use “chmod” with the appropriate flags, you’ll also never be able to remove the item from the ACL.
I removed the ACL rule from the Documents folder (chmod -N Documents), so that its permissions matched “the good old days.” I went to reload my server in the browser and voila, problem solved. I’m still not sure exactly why this caused a problem, but that’s the least of my concerns. The fact that it was caused by Leopard changing the permissions of directories in my home directory makes me a teeny bit annoyed. I like to think of the items in my home directory as belonging to me — as being somewhat sacred and under my control.
Take Home Message
Ah, well. I suppose these special folders have always carried signs of Apple’s ownership. After all, they get magical custom icons, so maybe I should choose another less territorial area for my important files. But this does make a sort of tangential example of how things can go wrong because of the most subtle of changes. When we developers whine and moan about not having access to the release OS in time to test, this is exactly the kind of thing we are worried about. We never really know what we’re dealing with until the final release is before us, so we’re naturally nervous until we can sit down and test it.
Fortunately in this case the issue doesn’t affect any of my products, so far as I can tell. It only ground my testing and development environment to a halt for a few hours. But for other developers, who knows? If you’re running into mysteriously permissions-related problems, it might be worth a look at those access control lists.
October 30th, 2007 at 7:14 am
That is pretty interesting. Odd that a “can”™t delete” permission would cause Apache to balk.
It occurs to me that what 10.5 did to you, basically, was repairing your permissions. And it causes something to break. Hope all the folk who recommend repairing permissions as the first step for debugging any problem take note.
October 30th, 2007 at 8:00 am
When I installed the WWDC build of Leopard back in June something similar happened…my existing ACLs were overwritten with that 0: group:everyone deny delete thing in almost all of my user directories…including ~/Library/Preferences, and somehow that had an inherit thing set so that nothing could be deleted from Preferences…and my hard disk soon filled up with zillions of tiny files that Leopard apparently writes and deletes to the prefs directory…but it couldn’t delete them.
Short version is I’ve warily decided to install Leopard (tomorrow) on a nice shiny new drive and carefully set things up again from scratch, so I’m working from a comfortable known state, permissions-wise.
Then, I’ll screw everything up.
October 30th, 2007 at 8:26 am
I had a bit of a different permissions problem crop up, and on two different machines. I did a clean install on both and then used the Setup Assistant migration tool (not Migration Assistant after boot) to migrate user data off of a bootable backup. Many user files, including lots of preferences like for the dock, got migrated as owned by root. Needless to say, some havoc was caused and I ended up having to
chown -R
my home folders. I’m not sure if it’s related to ACLs (seems doubtful) but just in case someone else sees the same…October 30th, 2007 at 11:07 am
On our test machine, which has been running previous Leopard seed builds since WWDC 2006 and now has the final GM build, there is no ACL on the ~/Documents folder. We missed a build or two late in the process, but that makes it hard for me to support that Leopard did this all on its own, at least in the GM version.
October 30th, 2007 at 11:11 am
Matt —
I’ve done 4 installs over the last 4 days. 3 erase / installs and one migrate. All got the new ACL treatment. These were from retail discs…. At this point, I would say that Leopard is most certainly doing this…
October 30th, 2007 at 11:12 am
Matt – thanks for the feedback – I was worried I might be overstating the case, but I had gotten some corroboration from friends after I first discovered it. Just to clarify – you are checking for the ACL by “ls -led” on the affected directory?
Rich – thanks for your comments – I am glad to hear that at least if Matt is right that it’s not happening across the board, it is still happening to quite a few people.
October 30th, 2007 at 1:29 pm
I’ve seen this too, as I like to symlink certain parts of my home account between OS releases (Music, Movies, Pictures). All have ACL rules on them, and have done for a few seeds now.
This is doing an erase and install, and having a new blank user though. Slightly different to the process you guys are discussing/
October 30th, 2007 at 3:23 pm
Possibly, if it is an inbuilt thing, it’s so that people
(a) can’t delete their Documents folder by accident (people can do dumb things, after all)
(b) malware can’t do rm -rf ~/Documents (and, who knows, rm -rf ~/ – have you looked to see if this “can’t delete” ACL applies to the whole of the home folder?)
So it might be Apple being a beneficient backstop, rather than trying to futz up your Apache. Though I’m sure an upgrade install would tear up all sorts of symlinks etc if you had MySQL installed and PHP in Apache etc.
October 30th, 2007 at 3:26 pm
Charles – I don’t think Apple’s trying to futz anything up. It’s just this is the kind of thing where unintended consequences are really hard to track down. I wanted to mainly help publicize in case others run into the problem.
October 30th, 2007 at 4:28 pm
Does anyone have a link on how to set up Virtual Hosts on Leopard?
I haven’t been able to figure it out now that Netinfo is gone.
Thanks.
October 30th, 2007 at 6:03 pm
Never mind. I figured it out.
October 30th, 2007 at 6:04 pm
Daniel – I didn’t mean to suggest that Apple was futzing things either. More that this is a protective measure; barely anyone would come across it. And those who did – like you – would eventually be able to figure it out. QED.
October 30th, 2007 at 10:33 pm
Charles – I just now did a /bin/ls -led /Users/myusername… and the home folder does indeed have the no delete ACL.
October 31st, 2007 at 7:47 am
@Robert – which would seem to confirm my suggestion – it’s a well-buried piece of anti-idiot/malware protection.
I’m still on Tiger, so have no ACL, though e is given as an option for ls. Does nothing, though.
October 31st, 2007 at 5:28 pm
Hi,
Is there a difference when people do upgrade, archive their system and do a new install?
Someone did notice something?