Lion’s Whole-Disk Encryption
July 20th, 2011One of my favorite new features in Lion is a completely revamped “FileVault”, Apple’s brand-name for encryption techonologies that protect the data on your disk from eavesdroppers, should the disk be lost or stolen.
In Mac OS X 10.6 and earlier, FileVault was a feature that only affected your home directory. In OS X Lion, it applies encryption at a very low-level, encrypting an entire volume of your disk at a time, and keeping it encrypted as you use it.
I was able to enable FileVault for my boot volume with relative ease, using the Security & Privacy preference pane in System Preferences. However, the UI for this is pretty limited, and notably, it only allows you to protect the computer’s startup disk.
The way I have my Mac configured, most of my sensitive data is not on the startup volume, but is instead on a second partition called “Data” where I keep my home directory, media files, etc. Apple’s Disk Utility allows you to erase and reformat a volume as encrypted from scratch, but what if you want to migrate a volume in-place, the way the system does the boot volume? You’re not completely out of luck.
OS X Lion ships with a low-level technology called “core storage,” which is used to facilitate a wide variety of disk-maintenance functionality, including whole-disk encryption. To get a quick look at what core storage supports, type “diskutil cs” at the Terminal command line. For a more in-depth look, type “man diskutil” and search for the core storage command documentation.
Important: This is the part of the blog article where I warn you to be very careful before proceeding. The diskutil command is capable of doing incredibly destructive things to your disk and to your data, so you should feel confident before doing anything that you have a 100% reliable backup of your data.
To convert an arbitrary volume to Lion’s whole-disk encryption, you use diskutil’s core storage “convert” command, and provide a passphrase. For example, if you have a volume called “Data” attached to your Mac, you would run something like this from the command line:
% diskutil cs convert /Volumes/Data -passphrase '[yourPasswordHere]'
Warning: at least one person has run into an issue where the passphrase was set to something unexpected because of characters such as ! and $ being interpreted by the shell before being passed to the tool. One more good reason to make sure you have a backup before messing with any of this stuff.
The “diskutil cs convert” command kicks off a conversion process similar to what the System Preferences panel does when allowing you to convert your main startup volume to core storage with encryption. At any time during the conversion, you can use the diskutil command again to see status of your volumes, whether they are encrypted, not encrypted, or in-progress while converting.
% diskutil cs list
You’ll see a bunch of information, but search carefully for the named volume (e.g. “Data”) that you just started the conversion process on. You’ll find a line starting with something like:
Size (Converted):
This shows you what the progress in the conversion is. From time to time, check this manually, to see how far along things have progressed.
Caveats
In addition to the major admonition above to backup your data carefully, you should also know that after you have converted a volume, it seems to be in a sort of provisionally encrypted state where it’s still being treated by the running OS as a “native volume” although it’s been converted and is ready to be treated as a “core storage” volume. I have to confess I don’t really understand it 100%, but it seemed like a really good idea to me to restart as soon as possible after the conversion is complete.
But before you restart, bear in mind that there appears to be a bug in the login process that will prevent a user whose home directory is on an encrypted (“locked”) secondary volume from being able to log in. It seems that whatever logic Apple applies to unlock volumes at login time is not applied early enough to allow the actual login to occur. This means that if you converted your secondary volume like I did, and it contains your home directory, you won’t be able to login.
For this reason, make sure that you have a valid account to log in to whose home directory is located on the main startup volume. In my experience, the process of logging in to this main-volume account will prompt the system to ask for the secondary volume’s password in order to unlock it. Once the secondary volume is unlocked, you can log out and log back in to your regular account, with the home directory on the secondary volume.
This bug is pretty annoying. Hopefully this is something that Apple will get fixed soon, and it may be for bugs like this that they haven’t enabled full-disk encryption as a full-fledged user-facing feature of the operating system. In the mean time, if encrypting your data is important to you, I hope these instructions and caveats will serve you well.
Update: Not surprisingly, this topic is covered in some detail in John Siracusa’s Lion review.
July 20th, 2011 at 2:56 pm
You do not have to specify a password on the command line. You can just end it with -passphrase
and it’ll prompt you for a password in a slightly more secure manner.
July 20th, 2011 at 6:01 pm
Thanks for the excellent article! This is exactly what I was looking for. :)
July 21st, 2011 at 1:46 am
It looks like you can’t encrypt software RAID volumes?
gravity:~$ diskutil info /dev/disk5|grep ‘Mount Point’
Mount Point: /Volumes/Raid0
gravity:~$ diskutil info /dev/disk5|tail -7
This disk is a RAID Set. RAID Set Information:
Set Name: Raid0
RAID Set UUID: 0C33C1B2-8681-48DE-B234-047A174FA982
Level Type: Stripe
Status: Online
Chunk Count: 61026666
gravity:~$ diskutil cs convert /Volumes/Raid0 -passphrase
Started CoreStorage operation on disk5 Raid0
Error: -69710: You can’t convert an AppleRAID volume to Core Storage
Rats! Any idea if there is a way around this limitation? Thanks!
July 21st, 2011 at 4:54 am
Thanks for the awesome write up Dan! Your blog continues to be at the top of my favorites list. You gave me high hope that I too could encrypt my secondary drive with my home dir. But then @Andy had to crash those hopes with his RAID comment. ;-) Oh well, thanks Andy!
July 21st, 2011 at 8:27 am
Thanks for this blog post. It helped me get set up. I created a ‘temp’ account that I use to log in before I choose my primary account. It seems as though the user you use to log in at the boot screen is attempting to auto-login, so you get the error message before you get to the usual user-selection login window in Lion. It seems like the usual option for ‘autologin’ is no longer available, as I was hoping to auto-login my non-admin ‘temp’ user and have it immediately log back out. Oh well, I’ll see what kind of solution I can cobble together…
July 21st, 2011 at 12:29 pm
Thanks for the tip about the login issue. I’ve replaced /Users with a symlink to a secondary partition for years, probably for the same time-to-make-the-donuts-I-mean-install-the-build-of-the-day reasons during our time at Apple you do.
July 21st, 2011 at 1:41 pm
After I had started my TM disk encryption, I thought I’d best turn TM off so it won’t start doing backup during the encryption, but when I got the TM settings open, I noticed it actually shows you the progress of the encryption!
July 21st, 2011 at 4:17 pm
Hmmm. Mine starts and then craps out with the following:
Started CoreStorage operation on disk1s3 BacklinAir
Resizing disk to fit Core Storage headers
Creating Core Storage Logical Volume Group
A problem occurred; undoing all changes
Reclaiming space formerly used by Core Storage metadata
Error: -69853: Boot file not written
Any ideas? It’s a backup drive, so shouldn’t have a boot file…though I am not disk genius.
July 22nd, 2011 at 4:27 am
Does any of you know if enabling full-volume encryption will disable the new TRIM support? Because in Linux, using cryptsetup will disable TRIM because it doesn’t pass the command down to the drive in order to prevent information disclosure about which blocks are used or not.
July 23rd, 2011 at 7:09 pm
I’ve created a system LaunchDaemon and shell script that allows me to mount a second encrypted drive containing my user directory during system startup.
You can find it and install instructions at:
https://github.com/jridgewell/unlock
Best of luck!
July 24th, 2011 at 4:04 am
@justin
you should extend your script for access the mac keychain from shell
1. the password for the encrypted volume should be stored in the system keychain
2. at login the system keychain is already unlocked and you can access the password for the encrypted volume
example
http://blog.macromates.com/2006/keychain-access-from-shell/
July 24th, 2011 at 4:15 am
*add*
if you volume name is “Daten” and you have stored the password in System keychain
the shell cmd would be:
security find -gl Daten “/Library/Keychains/System.keychain”
output:
keychain: “/Library/Keychains/System.keychain”
class: “genp”
attributes:
0x00000007 =”Daten”
0x00000008 =
“acct”=”82886EEB-A1C5-4389-A532-487AB4CA5C9C”
“cdat”=0x32303131303732343037333430315A00 “20110724073401Z00”
“crtr”=
“cusi”=
“desc”=”Encrypted Volume Password”
“gena”=
“icmt”=
“invi”=
“mdat”=0x32303131303732343038303634395A00 “20110724080649Z00”
“nega”=
“prot”=
“scrp”=
“svce”=”Daten”
“type”=
password: “TOPSECRED”
July 24th, 2011 at 5:35 am
@Matt:
I also get the error -69853 and then the process stops.
July 24th, 2011 at 6:11 am
Edit to my last post: This only seems to happen with bootable back up drives made with SuperDuper or Carbon Copy. With non-bootable drives, it works just fine.
July 24th, 2011 at 7:32 pm
@ leonsio
Good catch, I didn’t realize the system keychain was unlocked during startup. The scripts been updated to use the keychain now.
July 25th, 2011 at 11:36 am
Thank you so much for this guide! Now, I finally have an encrypted TM-backup. I never found the GUI to re-format and encrypt, but the command line worked just fine – and doing it that way also makes me feel like a fully fledged hacker, an added bonus! ;)
July 26th, 2011 at 8:24 am
@Andy: HOWTO enable encryption on a RAID, see my blog.
In other new I also run into this error:
Error: -69853: Boot file not written
July 28th, 2011 at 2:17 am
Hi hope this bug is fixed soon. My Home directory is in a drive located at /Volumes/Home.
July 31st, 2011 at 3:37 pm
To Wolfgang:
Do you know a way to make the disk non-bootable?
Thanks – Andy
August 18th, 2011 at 2:10 pm
I have a external disk with 2 partitions (one for Time Machine, and the other for SuperDuper!). Is it possible to encrypt the entire disk? If not, how can I encrypt both partitions? Thanks in advance.
August 18th, 2011 at 2:14 pm
@Neel, encryption is done on a per-partition basis. You would just encrypt each partition as described here. If the steps here aren’t clear enough it’s probably not a good idea to pursue this process. It would be safer to make complete backups of the partitions, and reformat the partitions as encrypted before copying the data back on to them.
August 18th, 2011 at 2:51 pm
@Daniel, thanks for the quick reply. I’ve used terminal before and feel fine with using the instructions in this article. If I did want to use Disk Utility tho, I only see the option of “Mac OS Extended (Journaled, Encrypted)” in the Erase tab, not in the Partition tab. How to use DU for this task? Also, I’m assuming that means I will have to enter 2 passwords each time I connect my drive? Thanks.
August 18th, 2011 at 3:23 pm
@Neel – I am not sure why it wouldn’t show up under partition, but if you see it under Erase, you could just partition the volume first and then erase each partition using the desired encrypted format.
August 18th, 2011 at 4:07 pm
@Daniel, will give it a try. Thanks again.
August 18th, 2011 at 4:51 pm
Hey there Daniel, great tip. Just to confirm: were you running the conversion on the fly while logged in with the same account which had the home folder on the drive you were converting? Cheers.
August 18th, 2011 at 4:55 pm
@roland – Yes, I was. However, I don’t know if this is advised or not. I had suitable backups in place before I did the conversion.
August 18th, 2011 at 5:04 pm
@Daniel – thanks. Appreciate the write up, will be a great help.
October 29th, 2011 at 12:57 pm
Last time I tried to use the data encryption included with a cheap external hard drive I ended up losing all the data – I’ll have to try it your way next time. >_<
November 5th, 2011 at 2:51 pm
Hey guys, I’m desperate. I followed all these steps carefully and now I can’t access my secondary disk that I just encrypted. The password I set is not accepted. It is not that the password is wrong, something happened during the encryption that makes the disk now completely unreachable.
some minutes after executing the command: “% diskutil cs convert /Volumes/Data -passphrase [yourPasswordHere] “. I rebooted the machine and now I think that because of this, the encryption process got corrupted and now there is no way to fix this.
In this disk I had very important stuff that now it looks like it is lost forever. Ye, I backed this stuff up but as I’m a bit retarded, by mistake I left the back up (sparse image in the same disk thinking it was another one.
the question is. What could I do? Is there anything that I could try?
To revert the process, to change the password, to re-encrypt on top?
Any help would be highly appreciated.
Thanks in advance.
November 6th, 2011 at 12:29 am
Hi Albert – sorry to hear about your conundrum. I’m afraid I don’t know what the disaster recovery options are, in detail. Hopefully somebody else will have ideas and chime in.
It would be worth at least checking whether:
diskutil cs list
Returns any information about your “lost” volume. Maybe it will show that it’s still in progress of being converted, and you just need to wait it out since you restarted while it was still converting.
Daniel
November 6th, 2011 at 7:29 am
Thanks Daniel.
This is the output:
+– Logical Volume Group 71245A9D-16A7-4223-A190-298004FCBB1E
=========================================================
Name: DESK-MAC-DAT
Sequence: 1
Free Space: 0 B (0 B)
|
+- Logical Volume Family E00D40CF-9796-44C1-8B2A-BFE7CDB14695
———————————————————-
Sequence: 6
Encryption Status: Locked
Encryption Type: AES-XTS
Encryption Context: Present
Conversion Status: Converting
Has Encrypted Extents: Yes
Conversion Direction: forward
|
+-> Logical Volume EA8ABC17-974A-4439-BF42-AE48F0107F51
—————————————————
Disk: -none-
Status: Locked
Sequence: 4
Size (Total): 973503901696 B (973.5 GB)
Size (Converted): -none-
Revertible: Yes (unlock and decryption required)
LV Name: DESK-MAC-DAT
Content Hint: Apple_HFS
I think the the problem is that I rebooted without having finished the process. I thought it was like the Standard Filevault process in Settings. That once rebooted, it would have continued. But it looks like not, that if you reboot in the middle or the process, then you get a partially encrypted disk that can’t be accessed or decrypted.
November 6th, 2011 at 2:09 pm
Albert, I’m not really sure what your best bet is here. It would probably be a good idea first of all to NOW do a full bit-for-bit duplication of the volume, before you try any other recovery techniques. Then I would probably continue work on the duplicate drive, just to leave the original in pristine shape in case you need to bring in experts.
One of the things I’d try on a safe backup of the volume would be to ask diskutil to “decrypt” the volume that is currently showing up as “Converting”. Maybe that will jog it to give up on the process and convert whatever portion is already converted back to normal?
Daniel
November 6th, 2011 at 5:06 pm
Thanks Daniel for your help but I’m afraid I won’t get it. At least after having checked other DATA Sources I have, at the end I only lost my Music Library and my pictures from this summer.
These are the outputs of the Diskutil command when trying to unlock or revert the corrupted volume:
MACPRO:~ admin$ diskutil coreStorage unlockVolume EA8ABC17-974A-4439-BF42-AE48F0107F51
Passphrase:
Started CoreStorage operation
Error: -69749: Unable to unlock the Core Storage volume
MACPRO:~ admin$ diskutil coreStorage revert EA8ABC17-974A-4439-BF42-AE48F0107F51The given UUID is a not a CoreStorage Logical Volume UUID
MACPRO:~ admin$ diskutil coreStorage unlockVolume E00D40CF-9796-44C1-8B2A-BFE7CDB14695
E00D40CF-9796-44C1-8B2A-BFE7CDB14695 is not a CoreStorage Logical Volume UUID
MACPRO:~ admin$ diskutil coreStorage unlockVolume E00D40CF-9796-44C1-8B2A-BFE7CDB14695
E00D40CF-9796-44C1-8B2A-BFE7CDB14695 is not a CoreStorage Logical Volume UUID
It looks like it is definitively corrupted.
Thanks anyway Daniel :)
November 6th, 2011 at 5:12 pm
What a bummer, Albert. Sorry about your summer photos! Sounds like that is the biggest loss here.
Daniel
November 8th, 2011 at 6:39 am
I got back all my stuff.
Just to let everybody know in case that this happens to other people.
I think that the reason that this encryption failed was because I interrupted to encryption process in the middle.
I thought that this was going to be like in the Filevault 2 encryption in Settings, that if you reboot the machine, as soon as you are logged in again, it simply continues.
For any reason, the encryption failed and it didn’t continue and was stacked in the middle of the encryption process. This was the reason of my problem of not being able to decrypt my hard-drive after login, and also what made possible that I’ve been able to get all my stuff back.
For the operating system this was an encrypted hard-drive. The password didn’t work because the encryption process didn’t finnish so for some reason, to put the right decryption password didn’t work.
Then I used Data rescue 3, as the hard-drive was not completely encrypted, for this application was still possible to reach all the not yet encrypted data. After scanning the hard-drive for several hours, Data Rescue 3 found all the stuff there with the right folder hierarchy and from there I was able to get back all my stuff.
Just in case this could help anybody in a similar situation.