ZFS Comes to OS X Courtesy of Apple's Former Chief ZFS Architect
Apple's flirtation with ZFS, the file system backed by Sun, has been well-documented with rumors and speculation dating back to 2006. The interest was fueled in mid-2007 by a claim by Sun CEO Jonathan Schwartz that ZFS would supplant HFS+ as the file system for Mac OS X Leopard. While ZFS did not materialize in Leopard, Apple clearly continued to work on the project, with claims of some support appearing in marketing materials for Mac OS X Server Snow Leopard.
Optimism surrounding that development was apparently premature, however, as all mentions of the file system later disappeared from Apple's site amid claims that licensing issues had led the company to scrap the project. By late 2009, Apple had shuttered even its open source ZFS project site, indicating that it had abandoned all work on the project.
In March of last year, Ars Technica posted a lengthy profile of Don Brady, the 20-year Apple engineer who had helped transition to the HFS+ file system for Mac OS X and also headed up Apple's internal ZFS team from 2006 until it was disbanded in 2009. As the report notes, ZFS offers a number of advantages over HFS+, including a 128-bit structure to address large file and volume sizes, as well as features for simplifying data management and increasing data integrity such as copy-on-write technology to preserve the file system structure should errors or failures occur while new data blocks are being written.
The company will soon be releasing its more advanced products, including the $39.95 Gold Edition with such features as Time Machine-like rotating data snapshots and data redundancy. A forthcoming Platinum Edition carrying additional features such as RAIDZ support, data deduplication, and an advanced management utility is set to launch this spring, with pricing yet to be announced. Finally, the company is also working on a Developer Edition that will offer a combination of GUI and command line interface tools for complete system control.
Ten's Complement is not the only group to still be working on bringing ZFS to Mac OS X, as the MacZFS open source project continues to build on the foundation laid by Apple and Sun. But Ten's Complement seems to be the first to bring ZFS to the Mac in a commercial package to facilitate implementation and maintenance of the file system for users looking at options for securing the integrity of their data.
Optimism surrounding that development was apparently premature, however, as all mentions of the file system later disappeared from Apple's site amid claims that licensing issues had led the company to scrap the project. By late 2009, Apple had shuttered even its open source ZFS project site, indicating that it had abandoned all work on the project.
In March of last year, Ars Technica posted a lengthy profile of Don Brady, the 20-year Apple engineer who had helped transition to the HFS+ file system for Mac OS X and also headed up Apple's internal ZFS team from 2006 until it was disbanded in 2009. As the report notes, ZFS offers a number of advantages over HFS+, including a 128-bit structure to address large file and volume sizes, as well as features for simplifying data management and increasing data integrity such as copy-on-write technology to preserve the file system structure should errors or failures occur while new data blocks are being written.
At one time, there was a lot of hope that Apple might transition from HFS+ to ZFS as a more modern replacement. In particular, ZFS's automatic snapshot feature was believed to be the perfect fit for Apple's Time Machine backup tool. When Snow Leopard was first announced in 2008, it was slated to have full read-write ZFS support, at least in the server version.
But when WWDC 2009 rolled around, all mention of ZFS support was scrubbed from Apple's website. A patent infringement lawsuit was still pending between NetApp and Sun, with NetApp claiming that it held patents on copy-on-write. Furthermore, Sun's CDDL open source license was also believed to make ZFS incompatible with Mac OS X, and that Apple couldn't reach suitable license terms with Sun.
Z E V O's Silver Edition brings you sophisticated ZFS storage technology in an easy-to-use solution.
It's ideal for those wanting modern, reliable storage that is simple to set up, validate, and manage. Our Z E V O Storage Setup Assistant makes setting up a single disk for Z E V O a snap. In a few simple steps you'll be up and running.
The company will soon be releasing its more advanced products, including the $39.95 Gold Edition with such features as Time Machine-like rotating data snapshots and data redundancy. A forthcoming Platinum Edition carrying additional features such as RAIDZ support, data deduplication, and an advanced management utility is set to launch this spring, with pricing yet to be announced. Finally, the company is also working on a Developer Edition that will offer a combination of GUI and command line interface tools for complete system control.
Ten's Complement is not the only group to still be working on bringing ZFS to Mac OS X, as the MacZFS open source project continues to build on the foundation laid by Apple and Sun. But Ten's Complement seems to be the first to bring ZFS to the Mac in a commercial package to facilitate implementation and maintenance of the file system for users looking at options for securing the integrity of their data.
Top Rated Comments
(View all)
3 weeks ago
I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.
It's not just Ars, it's John Siracusa. He's written 20+ page OS X reviews since the beta. He knows his stuff.
HFS+ is held together with duct tape and a prayer at this point. I hope Apple has something brewing, because I'm not sure how much longer they can keep slapping more stuff on top.
If you've read Siracusa's review, he suggests that Apple's work on Core Storage (the biggest change in the OS X FS to date) might be an indication of the direction they're going. A few years ago they were hiring filesystem engineers, so they're definitely brewing something.
3 weeks ago
Love that he continued to work on it.
However, given the "silver, gold and platinum" rubbish, maybe they should have made this for Windows. One version for all, even if the average user wouldn't use those features.
I appreciate the amount of effort and time, but fragmented releases like that suck. Especially if you had a "silver" edition and suddenly wish you had the "platinum" edition, you'd probably have to reformat.
However, given the "silver, gold and platinum" rubbish, maybe they should have made this for Windows. One version for all, even if the average user wouldn't use those features.
I appreciate the amount of effort and time, but fragmented releases like that suck. Especially if you had a "silver" edition and suddenly wish you had the "platinum" edition, you'd probably have to reformat.
3 weeks ago
I remember those days of testing ZFS in OSX. We couldn't seem to fix the problem where an attached USB drive formatted as ZFS would go to sleep and BAM ! kernel panic. Back then ZFS needed to be constantly connected (R/W) to any partition that was formatted ZFS. This is why the team at Apple gave up on it.
Since my NDA with that group no longer exists I can say that now :)
It looks like he's fixed the problem.
Since my NDA with that group no longer exists I can say that now :)
It looks like he's fixed the problem.
3 weeks ago
Nice to see you guys covering Zevo. Ten's Complements has a great solutino but there's one issue that I really hope they've found a solution for. Time Machine drives apart from being a backup are also used as disaster recovery. If you store your backup on an FS that the OS does not by default know how to read, you will have a hell of a time restoring from it in the unlikely event of a disaster recovery. You would need some way to slipsteam the ZFS driver into the Lion recovery partition and then every time an OS update wipes out the hack you'd have to recreate it. The only other option would be the creation of a boot disk of some sort for the use of disaster recovery. I'd imagine that the first case breaks Apple's EULA and I am really hoping that Ten's Compliments has been working on creating a book disk of some sort. To be honest, for most OS X users, any backup other than a disaster recovery protection is a waste of time money and space.
Another problem is the fact that ZFS has absolutely no driver to Windows which is not quite the case with HFS but then again writing a Windows drivers is a bit out of the scope of Ten's Compliments.
Oh and as to why ZFS is something you need to care about here is a why:
HFS+ had gotten incrementally better over the years and I'd even go so far as to suggest that it's a marginally better FS than NTFS. However, the age of the FS is being hidden by the OS and in fact there are tons of hidden meta files that you never get to see. The problem with HFS+ and pretty much everything else out there is that they don't check for bit level file consistency. HFS+ however depends on the assumption that there are no bit level errors in your meta data in order to let you access your files. And when errors occur you are under a very very real danger of loosing your files. Apple would have you believe that it is impossible to ever loose files and then they'll go back and tie their pants by saying that you should always backup your files. Well here's what I know from personal experience. I had a set of data that I've been moving between OS upgrades and computer changes and hardware upgrades for the better part of 7 years. Yes, I do clean my preference files whenever it gets really messy. One day, after some sort of a combination of a hardware failure followed by a failed attempt of a software to save changes to a file, my whole documents folder literally disappeared. It was still there in terms of bits but HFS+ could not read it or repair the drive. Had that been ZFS, the error would have been caught long before occurring an ideally I would have known that the drive was failing with my data being written to the non failing parts of the disk (it was not a mechanical failure).
Anyway I lost some files and restored most but the point is even the best efforts of Apple can't quite cover up the fact that the FS design is a bit creaky 20+ years after it's creation.
As to Apple picking up ZFS once again, all is not at all lost. If I may point out, nowadays, Oracle does not have a single person from the Sun ZFS team working for them anymore. Stupid management decisions made it unacceptable for all the guys to stay there. ZFS improvements however, are still being contributed by those same guys. In fact the open source version of ZFS is currently the one that is ahead in terms of features and development efforts. There is still the issue of licence compatibility which I am not so sure if it really is an issue but my point is that if Apple ever chooses to reevaluate ZFS' suitability for OS X there is a very good chance of things turning out much different than they did last time.
Oh and a while back ArsTechnica did a very good piece of HFS+ and why a replacement is needed. Look it up for more details if you wish. It is factually correct.
Another problem is the fact that ZFS has absolutely no driver to Windows which is not quite the case with HFS but then again writing a Windows drivers is a bit out of the scope of Ten's Compliments.
Oh and as to why ZFS is something you need to care about here is a why:
HFS+ had gotten incrementally better over the years and I'd even go so far as to suggest that it's a marginally better FS than NTFS. However, the age of the FS is being hidden by the OS and in fact there are tons of hidden meta files that you never get to see. The problem with HFS+ and pretty much everything else out there is that they don't check for bit level file consistency. HFS+ however depends on the assumption that there are no bit level errors in your meta data in order to let you access your files. And when errors occur you are under a very very real danger of loosing your files. Apple would have you believe that it is impossible to ever loose files and then they'll go back and tie their pants by saying that you should always backup your files. Well here's what I know from personal experience. I had a set of data that I've been moving between OS upgrades and computer changes and hardware upgrades for the better part of 7 years. Yes, I do clean my preference files whenever it gets really messy. One day, after some sort of a combination of a hardware failure followed by a failed attempt of a software to save changes to a file, my whole documents folder literally disappeared. It was still there in terms of bits but HFS+ could not read it or repair the drive. Had that been ZFS, the error would have been caught long before occurring an ideally I would have known that the drive was failing with my data being written to the non failing parts of the disk (it was not a mechanical failure).
Anyway I lost some files and restored most but the point is even the best efforts of Apple can't quite cover up the fact that the FS design is a bit creaky 20+ years after it's creation.
As to Apple picking up ZFS once again, all is not at all lost. If I may point out, nowadays, Oracle does not have a single person from the Sun ZFS team working for them anymore. Stupid management decisions made it unacceptable for all the guys to stay there. ZFS improvements however, are still being contributed by those same guys. In fact the open source version of ZFS is currently the one that is ahead in terms of features and development efforts. There is still the issue of licence compatibility which I am not so sure if it really is an issue but my point is that if Apple ever chooses to reevaluate ZFS' suitability for OS X there is a very good chance of things turning out much different than they did last time.
Oh and a while back ArsTechnica did a very good piece of HFS+ and why a replacement is needed. Look it up for more details if you wish. It is factually correct.
3 weeks ago
We need a decent file system. I've experienced corruption with hfs+ but nothing as bad as with fat32 or ext2.
3 weeks ago
What exactly is falling apart with HFS+? I use ZFS on a media server with raidz1 across 6 disks. It's great for that sort of thing, but for the average user with 1 or 2 HFS+ disks in an iMac, what are they missing?
it corrupts easily, for one.
for anyone who hasn't read this: http://arstechnica.com/apple/reviews/2011/07/mac-os-x-10-7.ars/12#file-system
3 weeks ago
Logical fallacy: argument from ignorance.
Pre-journaling, it happened every single time the system was shut down improperly. Journaling has helped a LOT, but it still doesn't *completely* mitigate the issue of corrupt directory structures. BUT journaling is implemented as a hack on top of the existing HFS+. It really is an after-thought, and it works pretty well for a hack.
Ditto.
Very true post.
There's a reason DiskWarrior is still good to have around. But it's true journaling helps a lot. Still, while it's quite seldom it has happened that I've had to run DiskWarrior to save a broken HFS+ filesystem, even with journaling enabled. I work at a place with about 150 Macs.
I would NOT trust ZFS for mission critical data until it has been proven to be completely non data destructive.
You mean this implementation of ZFS for Mac OS X?
ZFS has been around for some time and it has been and is running on mission critical data just fine.
3 weeks ago
I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.
I've seen it, and it's a horrible experience. It can happen on any disk, the question is whether or not the file system can detect it soon enough that the data can be repaired, and before a corrupted file is backed up overwriting a good one. With HFS+ the corruption won't be detected until you try and copy an unreadable file, or if a file is corrupted but still readable it won't even detect then, you'd have to actually find the corruption yourself.
[ Read All Comments ]
Even as Apple is preparing to open its first Dutch retail store in Amsterdam on March 3, the company is moving closer to expanding its international reach even further as it has updated its Swedish...
Following reports yesterday that Apple would open its first Dutch retail store in Amsterdam on March 3, the company has confirmed that date today with emails to customers and a new dedicated store...
The Charlotte Observer reports that a total of 25 iPhones valued at over $16,000 have been reported stolen from Apple's Northlake Mall retail store in Charlotte, North Carolina. While...
German site iFun.de reports [Google translation] that has it has received information from a source "to be taken seriously" claiming that the iPad 3 will debut there on Friday, March 23.
...
AFP reports that Proview Technology has expressed a willingness to work toward a settlement with Apple in the "iPad" trademark dispute in China, even as the company continues to press forward...
