NetApp versus Sun, Sun versus NetApp, and Both versus Common Sense

October 27, 2007 - by jason

As you might have heard and likely read in the back-and-forth blogging of Dave Hitz (a NetApp founder) and Jonathan Schwartz (CEO of Sun Microsystems), the two are at each other’s throats. Well not really at each other’s throats: NetApp went nuclear and Sun hit back even harder.

Basically NetApp says that Sun’s ZFS steals it’s existence from NetApp’s WAFL, an earlier Copy-On-Write (COW) file system. Sun in return wants to ensure that NetApp cannot sell its filers and pull NetApp’s ability to legally use NFS.

For those of you that need reminding, Sun invented NFS (the wikipedia page has a sufficient history) but NetApp did the best commercial appliance implementation. I forget if NetApp handed Sun their asses in the marketplace with a superior product, or if Sun just never delivered a filer-like appliance (I say “I forget” because while I can recall using NetApp, I can’t remember using a single offering from Sun beyond just a workstation).

NetApp filers do happen to be one of the best NFS appliances out there, and they happen to combine that with iSCSI and Fiber Channel in one rig. Pretty flexible, and if you look around you’ll see that it’s pretty unique. For example, an EMC Clariion does iSCSI or Fiber Channel, and if you want it to do NFS, then you basically buy a “gateway” (just a server attached by fiber).

We at Joyent are in a bit of a odd and unique position (I think). We happen to use what I call The Holy Z2D Trinity (ZFS, Zones and DTrace), and not just that we use ZFS on top of block storage from NetApp filers. And the team here from myself, to Mark, to Ben have written tools for filers, have managed many petabytes worth of them (often petabytes all at one time), and have been around them since they came out.

Hmm.

In fact, besides my common boast that we’re one of the larger or the largest OpenSolaris/Solaris Nevada installations in the world, I’d venture to guess that we have more ZFS on top of NetApp filers via iSCSI than just about anyone else.

Now let’s think how we even got here in the first place.

  • NetApp developed a nice little filesystem they call WAFL.
  • NetApp contributes to FreeBSD and FreeBSD is the core OS in some of their products (yes please don’t try and “educate” me on a rig I’ve used for a decade, I said “some”, I know they have their own OS).
  • WAFL or something like it has presumably been ported to FreeBSD.
  • When pulling up block storage LUNs from our filers, we still need to a filesystem on it. (Got that? Not everything is NFS, There still needs to be a filesystem on the server.)
  • Nothing like WAFL or a nice consistent COW filesystem was ever contributed back to FreeBSD.
  • FreeBSD instead just has UFS2, and while it has soft updates, you’ll still to need to run a fsck in single user mode for hours once you’re in the 100s of GBs, and ironically the busier the system is, the worse that is (again, yes I know there’s a background fsck, try running that on a busy mail server, it’ll crash, guaranteed).

So larger drives (or LUNs) plus a shitty, non-resilient file system meant that all of FreeBSD had to go. Despite everything great about it and a lot of time invested by the team over a decade in using it. We had to leave FreeBSD and go to wherever we could to get a good, easy to use, resilient, modern file system.

That file system happened to be ZFS and the operating system was OpenSolaris. The additions of DTrace and Zones (we used Jails before) formed our three pillars.

But stop and imagine for a minute that NetApp had been kind enough to give WAFL or a similar filesystem back to the FreeBSD project. Imagine that we had something like WAFL to put on top of our block storage LUNs that were coming up from the NetApp filers.

Got that?

We didn’t want or need WAFL on a real operating system to develop a competitive storage product, we needed WAFL-like on a real operating system in order to simply use our NetApps.

Then we wouldn’t have ever made our first step to OpenSolaris. We’d currently have a business based on FreeBSD 7 with WAFL, Jails and DTrace (hooray for that port!), and believe me leaving FreeBSD was painful in many ways, with the salve being ZFS and DTrace.

While I think there is a good degree of posturing going on betwee the two companies, and it’s fascinating to see it going on in blogs, both parties are full of it and don’t quite get it.

NetApp Dave:

You should have given WAFL or a WAFL-lite file system to FreeBSD and then all of us could happily use it on top of iSCSI or fibre-channel block storage. You would have made it the best operating system to put on top of any networked block storage, being smart guys, you could have figured out how to do it while making NetApp even more money, and without spawning a bunch of FreeBSD storage appliance clones. The fact that you didn’t do this is why Joyent uses Solaris, and you’re responsible for the new market need that’s out there for ZFS and via ZFS, Solaris itself.

Stop being so shortsighted and give a little, think FreeBSD 7.5 + WAFL.

We’re the poster child for OpenSolaris adoption, and that fact that we’re using it … you could have prevented it.

Sun Jonathan:

It’s great that ZFS was developed, magnificent that it was open-sourced, and I know that it was valid and true creation of Jeff Bonwick (who we count as a good friend). You make hardware, you created NFS, you’ve always touted the importance of the network in computing, yet I’ve had to always use NetApps to get fast and reliable NFS and iSCSI block storage (even to put my ZFS filesystems on).

Ship a decent piece of hardware + software capable of fast and reliable NFS + iSCSI. NetApp only exists because of Sun’s failures in actual product development.

“What Ifs”

If NetApp wins, there goes our beloved ZFS (yes I understand indemnity, but I care more about continued development) and NetApp, you don’t have an alternative for us. Thanks for nothin’.

If Sun wins, there’s goes our NetApps, and Sun, we can’t get an alternative from you. Thanks for nothin’.

And finally

I think we’ve been a good customer of both of you, when you fight, it hurts us more than anyone else.

:

Sign up now for Instant Cloud Access Get Started