Why do geeks have lust for ZFS?

“Windows’ current file system can store files up to 4GB in
size. HFS , the current file system that Apple uses, can store files up to 4GB in size. ”

It should be pointed out that this statement is not accurate. Both Windows and OS X support file sizes up to 16 Tebibytes, with a theoretical limit of 16 Exbibytes. Perhaps not as high as ZFS, but significantly greater than the limit given above.

It seems that I’ve overlooked that detail, Eric. Thanks for the comment.

It seems that I’ve confused NTFS with FAT32. NTFS is 16 TiB, while FAT32 is 4GB.

I’m updating the article to reflect this.

[...] When considering the present needs of most home users, Sun’s ZFS is no doubt overkill. Developed for Sun’s powerful Solaris operating environment, ZFS brings truly advanced filesystem technologies to the masses. For some background with respect to the benefits and capabilities that ZFS offers, I’d have to recommend an article I read today that briefly introduces ZFS. With recent versions of Mac OS X reportedly offering some degree of ZFS support, it will likely see increasing use on what can be considered typical home computers. [...]

Great article! I thoroughly enjoyed it, as I didn’t know much about ZFS before. Hopefully we’ll begin seeing ZFS on Linux soon. I’d love to try it out!

“ZFS supports intelligent drive mirroring

By having to wait for two drives to write the same information, RAID1 lowers the drive’s performance. On the contrary, ZFS’s mirroring actually speeds up the reading performance of the hard drive “

You talk about mirroring slowing down writing then say “on the contrary” it speeds up reading… the slow writing is still an issue.
This slow writing and faster reading occurs on almost all RAID1 implementations though, it’s not specific to the ZFS RAID1.

What’s cool is that the mirroring is built into the OS.

BTW, what happens when you want to remove a drive? Like in your example, you want to remove the 300GB drive.

- rmjb

The file system is phenominal. I want to see this be released.

By the way, how many hours (days?) of Pr0n could be held by 1 zb?

wow. raid 1 redundancy at the click of a button in OSX! tho not a replacement for proper backups & archives, this sounds like an ideal desktop backup system, no?

ZFS uses advanced hashing algorithms
We may not realize this, but the way that most common file systems are made is very unreliable for data integrity. As hard drives get bigger, more error-correction is used, and there is more possibility for data corruption which can lead to file loss and several other problems. In fact, today’s drives rely a lot on error-correction mechanisms daily. Sectors are lost all the time, and drives which are bigger than 250GB are doing data correction constantly. ZFS is the ideal file system for the bigger and bigger hard drives that we get to deal with

I didn’t find anything on a brief glance at the page you cite… but this section is entirely unsourced

[...] http://tech.zamwi.com/2007/01/16/why-do-geeks-have-lust-for-zfs/ [...]

BTW, what happens when you want to remove a drive? Like in your example, you want to remove the 300GB drive.

I believe that you can remove it in ZFS, but you’ll need to re-sync it when it will be back in.

I didn’t find anything on a brief glance at the page you cite… but this section is entirely unsourced

They don’t quite list this on the ZFS page itself, but they talk about it in their demonstrational videos. They show diagrams of how the filesystem verifies the integrity of the files by checking the checksums.

Tron, it’s already been implemented by Sun in their operating system, Solaris, which, unlike OS X, is open source, and, if that wasn’t enough, they are willing to post you a copy completely free of charge (dear me, that was alot of commas!).

“By the way, how many hours (days?) of Pr0n could be held by 1 zb?”

Years, my friend. Years.

Welcome diggers, thank you for your support…please check out the brand new zamwi.com forum here.

Assuming the 19.6 Gb used to store a 2h movie like Serenity in HD DVD format, one year of HD pr0n would take up 85,848 Gb. That means one zettabyte would store 13 million years of HiDef pr0n. You’ll want to buy some lube.

What seems the most appealing to me is that it is open source meaning that if Apple, Linux, and even Windows adopted it, it could really open doors for having multiple operating systems on a drive and they would all have read/write abilities out of the box right? Or maybe I’m dreaming.

I hope that you’ve enjoyed this in-dept look at ZFS

This phrase, “in depth,” I do not think it means what you think it means.

[...] the following is a copy of the article from http://tech.zamwi.com/, it explains about the new ZFS file system in apples OSX leopard and all the hype to it. Just thought i’d share with you guys… this a gud read for geek [...]

If you encoded your pr0n at around 100kBps (~1000kbps) which is reasonable quality for divx, and ignoring all overheads, you could potentially store in excess of 317 years of non-stop pron goodness on a single 1 zb hard drive.

Shizgiac.

[...] Read more [...]

[...] For a detailed description of ZFS and its features have a look at the Zamwi.com article. Permalink [...]

Kenny P – why would you want to do that though?

I wouldn’t want a Windows OS going anywhere NEAR my important data files. It’s the whole reason I replaced Windows.

Isn’t a zettabyte a billion terabytes? tera, peta, exa, zetta?

Time Machine is not ZFS
Time Machine is not ZFS
Time Machine is not ZFS

http://arstechnica.com/staff/fatbits.ars/2006/8/15/4995

Hardware RAID controllers can increase read speed and have been able to for a long time now.

What seems the most appealing to me is that it is open source meaning that if Apple, Linux, and even Windows adopted it, it could really open doors for having multiple operating systems on a drive and they would all have read/write abilities out of the box right? Or maybe I’m dreaming.

Another really cool thing is that it’s ‘endian’ agnostic. So in theory if you had an external storage device which was a ZFS pool/filesystem, you could quiet happily plug it into a SPARC Solaris machine, an x86/x64 Solaris machine, a Mac OS X machine, a linux machine, etc etc and everything would just work.

Yeah, comrade, we’ve been able to have ZFS in Linux for a long time now. A fair number of distributions even support it right out of the box.

In depth? Oh dear, oh dear, oh dear….

[...] from zamwi.com In the recent builds of Apple’s new operating system, Mac OS X Leopard, developers have started noticing that there is an option to format a partition using ZFS (Zettabyte file system). There is still no mention from Apple whether this will remain in the final builds, but the file system matches exactly Apple’s plans for Leopard. This article will explain you everything that you need to know about ZFS aswell as why I’m so exciting about its inclusion. Read on to learn more. [...]

It only gets better. The (new) best file system is well on its way to being ported to the (old) best operating system:

http://www.freebsd.org/news/status/report-oct-2006-dec-2006.html#Porting-ZFS-to-FreeBSD

Thank you, Sun!

No theoretical limit for files? One zettabyte overall? I think you should read over the FAQ again: http://www.opensolaris.org/os/community/zfs/faq/#whatlimits

A 16 exabyte system would bring us down to only 13 thousand years of high definition pr0n.

Paperstreetlab, putting science in terms a teenager could understand since 1996.

[...] read more | digg story [...]

anonymous critic – Time Machine was not then implemented through ZFS. That article references the first Leopard alpha. This references like Alpha 2 or 3 or so.

But where’s the file tagging (and extended attributes) support?

Someone asked about how much video could be encoded with this thing. Read http://blogs.sun.com/dcb/entry/zfs_boils_the_ocean_consumes to get a good idea of what this filesystem is capable of. Effectively, maxing out this filesystem would require you to boil the earth’s oceans, and make every grain of sand in the entire world produce the amount of power that the entire world currently generates. You’d also need a mass of harddrives the mass of about a thousand of our moon. So… good luck filling this up, I’m pretty sure every second of earth’s history could be recorded on this filesystem, and in hi-def (or near hi-def).

One correction: Time Machine is not built on ZFS. (Even though ZFS would have made the job far simpler.)

-jcr

Square Bottle, you are misinforming people about linux and ZFS, please stop.
Linux can indeed support ZFS but by using FUSE having support in userspace isn´t an ideal solution.
Linux can not currently support ZFS, maybe if a complete rewrite was done but then again what patents surround this file system is unknown to me but I do know the current license ZFS isn´t GPL friendly.

The current filesystem used on Mac OS X, HFS , was originally developed for Mac OS 8.1 back in 1998. I think that we all agree that such a modern operating system such as Mac OS X could use a modern file system.

I’m interested to see ZFS support grow (there are, however, current issues with not being able to boot from a ZFS volume).

But the argument that HFS was created 7 years ago and therefore is undeserving of a modern OS like OS X is just silly.

That’s not to say there aren’t issues with the file system, just that the argument that it’s old, and thus less deserving, is kind of a lame one.

Until you can boot ZFS, it can only serve as a secondary drive.

Great article! I thoroughly enjoyed it, as I didn’t know much about ZFS before. Hopefully we’ll begin seeing ZFS on Linux soon. I’d love to try it out!

Will it support hybrid drives which have a combination of flash and ceramic disk?

Vista is supporting this already; data which is constantly in use is written to the flash part of the drive (for quicker data access).

This will speed up HDD access a sh*t load more than just using a legacy filesystem.

p.s. I am not a windows fanboy, I am just saying…..

But the argument that HFS was created 7 years ago and therefore is undeserving of a modern OS like OS X is just silly.

Especially when you remember that OS X is just the latest revision of NeXTSTEP, which dates from the late 80s/early 90s.

It’s also worth noting that 16TiB “limit” for current filesystems being discussed is the maximum size of a _single file_, not the whole filesystem. ZFS certainly has higher limits, but it’s not like most people be bumping into the limits of current systems any time soon.

You’re right, as far as it goes, but VM systems do more than just swap memory out to disk. They also re-arrange memory in a dynamic way. Once upon a time you really did have to manually tell the computer how much RAM it had and where that RAM was. Many machines had special RAM that could only be used for special purposes, or RAM that was hidden unless you played programming tricks.

Lama, windows will cache the most commonly used files to the flash drive, now will lead to a problem due to the fact that flash can only be accessed a certain number of times.
This is the case with flash cards so it might apply or it might not.
Question is are you willing to use something that isn´t tried and tested properly.

Although unfortunately, the CDDL is incompatible with Linux, so it looks as though any of the responsible major distro’s will not support this technology.

Even if it has a clean room implementation, does Sun have plans of protecting its IP when its ported to a competing platform ?

OSX people don’t seem to get ZFS.

Size isn’t the important thing about ZFS — the concept of storage pools basically gives you the features that you get from an expensive volume manager like Veritas or EMC for free within the OS.

The ultimate application for ZFS is the Sun x4500 server, which gives you 24 disks that can be all striped into one big volume group.

Ben Rockwood’s blog has some great posts about what he and his company is doing with ZFS.

Other than the Alleged hashing algorithms I don’t see anything that really effects end users, its likely zfs could be slower, and I’ll assume it is till I see some tests.

[...] If you are not sure what ZFS is and why you should get excited about it, here is a really well written article about some of the nifty features ZFS has. To sum it up :- [...]

A “zetabyte” actually zettabyte is not 1 million terabytes.

Byte - 1
Kilobyte - 1,000
Megabyte - 1,000,000
Gigabyte - 1,000,000,000
Terabyte - 1,000,000,000,000
Petabyte - 1,000,000,000,000,000
Exabyte - 1,000,000,000,000,000,000
Zettabyte - 1,000,000,000,000,000,000,000

As you can see, a Zettabyte is 1 million Petabytes, or 1 billion Terabytes.

[...] Read all about it here. [...]

Ok, quick question. A curious child gets a decent video camera, say HiDef, at age 7 and uses it, and its successors, to video tape 12 hours a day of his life, 365 (and 1/4) days a year for his entire life. He dies at age 107. Never misses a day, though his great grandchildren have to assist the old guy in his last few years.

Each day he downloads his video to his 1 zettabyte hard drive on his PC(s).

Does he run out of room? If so at what age?

@Chris
Good god man, that has been addressed already. He’d have to be 26 thousand years old to use up the drive. Or he’d have used up 0.3% of the drive by the time he died.

It sure sounds nice to have a volume span 24 disks – until one breaks(or even worse, two break). What kind of recovery is built into ZFS, because with 24 disks, the chances of one breaking are not that slim anymore…

[...] We’ve all heard the recent rumors about OS X Leopard. We know that it will support ZFS (the Zettabyte file system), but what does that truly mean? This is an article explaining all about ZFS and why it’s so great.read more | digg story [...]

Motorcycle Guy

Other than the Alleged hashing algorithms I don’t see anything that really effects end users, its likely zfs could be slower, and I’ll assume it is till I see some tests.

Probably the biggest one is that it’s trivially simple to add more disk space and have it “just work”.
Eg: You have a ZFS filesystem at /bigspace which is nearly full and you want to add another 500G:
1. Install disk (or disks, if you want it to be redundant).
2. ‘zpool add bigspace [disk]‘ (or ‘zpool add bigspace mirror [disk1] [disk2] to add two disks as a mirror).
3. There is no step 3. Your /bigspace mount now has 500G more space available.
It should be pretty obvious to see how step 2 could be GUIfied and automated such that the process would basically be “install disks, click button” (and obviously OS X would map /bigspace to something more familiar, like a volume on the desktop).
m64

It sure sounds nice to have a volume span 24 disks – until one breaks(or even worse, two break). What kind of recovery is built into ZFS, because with 24 disks, the chances of one breaking are not that slim anymore…

ZFS can use mirroring, RAIDZ (like RAID5) or RAIDZ2 (like RAID6) for redundancy.

Damn that was a very “in-depth” article. Not only did it give a lot of false or incorrect information, but it sounded like a big, ugly fanboy propaganda.

ZFS is nice, but for none of the reasons you provided. NTFS is also nice in it’s latest form, and for many reasons you don’t even seem to understand.

One such error is : ZFS and NTFS maximum filesizes are the SAME (16 EiB). The Windows NTFS _driver_ is what limits any file to 16 TiB. Whenever these kind of files get _manageable_ by any kind of software, it will be updated to reflect this new need (as it already have been in the past). This is for performance optimization reasons and I, for one, am happy it is that way.

NTFS and ZFS both offer journaling (the ability to capture the state of one file at different points in time), through varying methods. Server 2003 and Vista uses that capability to offer Previous versions of wanted files.

Anyways, you might want to dig a bit more before posting such useless articles.

http://en.wikipedia.org/wiki/Comparison_of_filesystems

Great! Thanks for the update on using ZFS on Linux. I appreciate it.

Thanks for the love diggers.

[...] Why do geeks have lust for ZFS? at zamwi.com (tags: zfs filesystems) [...]

I run ZFS on my Solaris 10 Update 2 home server. It’s ridiculously easy to administer.

The X4500 supports 48 drives, by the way, not 24 (the capacity is 24TB with 500GB drives). The idea is you just leave one with 5 or 6 hot spare drives, and only replace failed drives once a year instead of as soon as one fails. The I/O measured throughput on it is well over 1GBps with ZFS.

ZFS supports hot spares and double-parity, so if a drive fails, you can still survive if another drive fails while the filesystem is rebuilding from parity data. Rebuilding is very stressful on drives and it is not unheard of to see it trigger additional drive failures on marginal drives. Of course, you can also use mirroring if you want to.

The primary draw for me is snapshots, which are vital when backup windows are short. You can even clone snapshots and have the ability to fork your filesystem the way you would CVS.

Looks like a very nice file system.

On the contrary, ZFS’s mirroring actually speeds up the reading performance of the hard drive because it intelligently balances the load between two drives whenever a read command is issued.

Linux software RAID1 does this already, and I imagine decent hardware raid controllers would too. They read from the disk who’s head is closest to the required data at the time.

[...] contact « Why do geeks have lust for ZFS? [...]

[...] Why do geeks have lust for ZFS? at zamwi.com “In the recent builds of Apple’s new operating system, Mac OS X Leopard, developers have started noticing that there is an option to format a partition using ZFS” – this is potentially very interesting; i’m hopeful we’ll see ZFS on Linux someday (tags: ZFS Apple Leopard OSX Solaris filesystems via:Mark) [...]

Guys, it’s not about storing video — that’s missing the point.

Back when we all stored things on audio tape or 130K disks, and processors took a while to get through “all” that data, applications were pretty limited. As computers grow in processing power, develop more demanding kinds of applications, and more of the world’s information is made digital, we’ll consume much MORE storage, even for what we might think of as the same tasks. That’s because, computers will understand the tasks better, and hold more data about them. A letter used to be JUST text: a single, small number, per character. Now, a letter on computer includes tons of typesetting information, language information, etc. Instead of those small characters, the smallest unit is a unicode character, which is up to 8 times larger, if I recall correctly (although not always). Unicode may grow some more before it’s mature, or even be replaced with something better. Audio used to be 8bits per sample at 8khz, now it’s 32 or more at 96khz or more. And that’s BEFORE the computer starts processing it and trying to figure out what it means, beyond a lot of noise. Add intelligence-helpers, like speech recognition data, and that audio file will grow a lot…

There’s also a coming explosion of metadata — we’re starting to index all of our files, and to tag everything on the net. Semantic desktop projects like KDE 4’s NEPOMUK and (the now defunct) GNOME Dashboard, we’ll be storing not just image data, but where it was downloaded from, who liked it, what kind of photography it is, etc. Then, we’ll have new applications that use a lot of space to manipulate that data, and come up with new forms of data.

Now imagine everything on the net, understood and connected by your computer, via virtually endless collections of metadata, like that provided by RDF.

Zetabyte filesystems will be commonplace soon enough, and we’ll wonder how we lived without them.

[...] ZFS (Zettabyte File System) is already used by Sun and looks pretty nice. Shucks, I’d use it! (from tech.zamy.com) Save this?These icons link to social bookmarking sites where readers can share and discover new web pages. [...]

[...] read more | digg story [...]

[...] Here’s an interesting article about ZFS (the Zettabyte file system), discussing its advantages. It looks like an interesting file system, and has a lot of great features like storage pooling and revisions of files and so on. [...]

Mac OS X has had built-in RAID 0 and RAID 1 for some time. It’s in the Disk Utility.

A “128-bit” file system is silly. (Hint: Avogadro’s number.) It’s true that a 64-bit file system is limited to 18 million terabytes of storage, so it may not be able to address all the storage in the world. However, assuming that storage would be addressed as individual blocks, rather than objects or files, is just wrong.

ZFS has some nice technology; the checksums are useful (as long as you are using RAID). It’s CPU-hungry, because of the software checksums (and software RAID, if enabled), and it makes poor use of disk bandwidth because it uses cache flushing rather than disk write ordering or journaling to ensure consistency, but it’s nice to see choice.

[...] Subsection on Cultural MemoryWhy do geeks have lust for ZFS? | Paulius http://tech.zamwi.com/2007/01/16/why-do-geeks-have-lust-for-zfs/ [...]

[...] Why do geeks have lust for ZFS? | Paulius http://tech.zamwi.com/2007/01/16/why-do-geeks-have-lust-for-zfs/ [...]

very exciting for OSX!

Hi our little brothers.n

64 bit windows vista advantages…

…

ZFS is a very cool system. I have it running on my OpenSolaris server at home.

Linux could not support ZFS (properly, not in user space) without almost a complete re-write, and even then the GPL license restricts you from implementing ZFS properly because ZFS is CDDL.

I guess, as long as they redesigned the kernel, users could compile it themselves and compile ZFS into it…Not sure about that though…

[...] the ideal file system for the bigger and bigger hard drives that we get to deal with." ref: Why do geeks have lust for ZFS? � zamwi.com __________________ My feedback: Head-Fi | [...]

Hello webmaster
I would like to share with you a link to your site
write me here preonrelt@mail.ru