ZFS: The Last Word in File Systems Part 1
Deirdré Straughan4 HN recommenders · 2016–2026
19K views · talk · evergreen · Uploaded 2012-06-12
Jeff Bonwick and Bill Moore, the creators of ZFS, present 'ZFS: The Last Word in File Systems' at the 2008 SNIA Software Developers' Conference. They explain the design, including copy-on-write, checksumming and avoiding the write hole.
HN recommends it as the definitive explanation of ZFS design straight from its inventors, with war stories about data integrity.
40:20"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
31:44The Value of Values with Rich Hickey
InfoQ
59:56AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
47:43"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
51:36Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
1:06:4401 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019)
CMU Database Group
54:52Introduction to NoSQL • Martin Fowler • GOTO 2012
GOTO Conferences
4:57Rich Hickey rants about HttpServletRequest in 1080p: Death By Specificity from Clojure Made Simple
Fred Overflow
6 comments from Hacker News
I would recommend anybody interested in filesystems to watch Jeff Bonwick (ZFS inventor) explain the design of ZFS: [VIDEO] . They have a few very nice war stories explaining why they found it useful to have the user data checksummed as well.
> Is ZFS any good? The presentation "ZFS: The Last Word in File Systems" from its debut in 2008 is a good introduction. Part 1 (of 3) probably best explains how different it is from just about anything else out there (even now, in 2019): * [VIDEO] Part 2 show some extra niceties / features, and Part 3 is a demo and Q&A. Watching Part 1 at least is worth it for anyone curious; 2 and 3 are optional.
Well, ZFS isn't exactly monolithic if you look under the hood: it has the ZPL (files, directories), DMU (objects, transactions on those objects), SPA (actual disk I/O). A potato-quality video from 2008 with Moore and Bonwick, the creators (timestamped to relevant section): * [VIDEO]
> initial issues You mean like the current advice not to use anything except mirroring and striping (RAID-0/1/10)? > Parity may be inconsistent after a crash (the "write hole"). The problem born when after "an unclean shutdown" a disk failure happens. But these are two distinct failures. These together break the BTRFS raid5 redundancy. If you run a scrub process after "an unclean shutdown" (with no disk failure in between) those data which match their checksum can still be read out while the mismatched data are lost forever. * [LINK] I've been using ZFS since it came out on Solaris 10 over a decade ago and it was specifically designed not to have a write hole due to its COW/ACID nature. See this 2008 SNIA presentation from Bonwick and Moore, the creators of ZFS talking about not having a write hole: * ACID/COW: [VIDEO] * Integrity: [VIDEO]
ZFS was an inspiration for TigerBeetle. I love this talk by Jeff Bonwick and Bill Moore: “ZFS: The Last Word in File Systems” [VIDEO]
ZFS: The last word in File Systems: Part 1: [VIDEO] Part 2: [LINK] ZFS In the trenches: [LINK] Some good stuff here: [LINK]