ZFS on root

This is part 2 of our blog post series on our current and future work around ZFS on root support in ubuntu. If you didn’t yet read the introductory post, I strongly recommend you to do this first!

Here we are going to discuss what landed by default ubuntu 19.10.

Upstream ZFS On Linux

We are shipping ZFS On Linux version 0.8.1, with features like native encryption, trimming support, checkpoints, raw encrypted zfs transmissions, project accounting and quota and a lot of performance enhancements. You can see more about 0.8 and 0.8.1 released on the ZOL project release page directly. 0.8.2 didn’t make it on time for a good integration and tests in Eoan. So, we backported some post-release upstream fixes as they fit, like newer kernel compatibility, to provide the best user experience and reliability. Some small upstream fixes and feedback were contributed by our team to upstream ZFS On Linux project.

Any existing ZFS on root user will automatically get those benefits as soon as they update to Ubuntu 19.10.

Installer integration

The ubiquity installer is now providing an experimental option for setting up ZFS on root on your system. While ZFS has been a a mature product for a long time, the installer ZFS support option is in alpha and people opting in should be conscious about it. It’s not advised to run this on production system or a system wherre you have critical data (apart if you have regular and verified backups, which we all do, correct?). To be fully clear, there may be breaking changes in the design as long as the feature is experimental, and we may, or may not, provide transition path to the next layout.

With that being said, what does ZFS on root means? It means that most of your system will run on ZFS. Basically even your “/” directory, is installed on ZFS.

Ready to jump in, despite all those disclaimers? If so, you download an ubuntu 19.10 ISO and you will see that the disk partitioning screen in Ubiquity has an additional option (please read the Warning!):

Yes, the current experimental support is limited right now to a whole disk installation. If you have multiple disks, the next screen will ask you to pick which one:

You will then get the “please confirm we’ll reformat your whole disk” screen.

… and finally the installation will proceed as usual:

In case you didn’t notice yet, this is experimental (what? ;)) and we have some known quirks, like the confirmation screen showing that it’s going to format and create an ext4 partition. This is difficult to fix for ubuntu 19.10 (for the technical users interested in details, what we are actually doing is creating multiple partitions in order to let partman handle the ESP, and then, overwrite the ext4 partition with ZFS, so it’s technically not lying ;)). It’s something we will fix before getting out of the experimental phase, hopefully next cycle.

Partitions layout

We’ll create the following partitions:

rpool

One ZFS partition for the “rpool” (as root pool), which will contain your main installation and user data. This is basically your main course and the one we’ll detail the dataset layout in the next article as we have a lot to say about it.

bpool

Another ZFS partition for your boot pool named “bpool”, which contains kernels and initramfs (basically your /boot without the EFI and bootloader). We have to separate this from your main pool as grub can’t support all ZFS features that we want to enable on the root pool, and so, your pool would be otherwise unreadable by your boot loader, which will sadly result in unbootable system! Consequently, this pool runs a different version of ZFS pool version (right now, version 28, but we are looking for next cycle to upgrade to version 5000, with some features disabled). Note that due to this, even if zpool status proposes that you to upgrade your bpool, you should never do that or you won’t be able to reboot. We will work on a patch to prevent this to happen.

ESP partition

There is the ESP partition (mounted as /boot/efi). Right now, it’s only created if you have a UEFI system, but we might get it created in Ubiquity systematically in the future, so that people who disabled secure boot and enable it later on can have a smooth transition.

grub partition

A grub partition (mounted as /boot/grub), which is formatted as ext4. This partition isn’t a ZFS one because it’s global to your machine, so the content and state should be shared between multiple installations on the same system. In addition, we don’t want to reference a grub menu which can be snapshotted and roll backed, as it means the grub menu won’t give access to “future system state” after a particular revert. If we succeed in having an ESP partition systematically created in the future, we can move grub itself to it unconditionally next cycle.

Continuing work on pure ZFS system

We are planning to continue reporting feedback upstream (probably post 19.10 release, once we have more room for giving detailed information and various use-case scenarios) as our default dataset layout is quite advanced (more on that later) and current upstream mount ordering generator doesn’t really cope with it. This is the reason why we took the decision to disable our revert GRUB feature for pure ZFS installation (but not Zsys!) in 19.10, as some use case could lead to unbootable systems. This is a very alpha experiment, but we didn’t want to risk user’s data on purpose.

But this is far from being the end of our road to our enhanced ZFS support in Ubuntu! Actually, the most interesting and exciting part (from a user’s perspective) will come with Zsys.

Zsys, ZFS System handler

Zsys is our work in progress, enhanced support of ZFS systems. It allows running multiple ZFS installations in parallel on the same machine and managing complex ZFS dataset layouts, separating user data from system and persistent data. It will soon provide automated snapshots, backups, system managements.

However, as we first wanted to have feedback in Ubuntu 19.10 about pure ZFS systems, we didn’t seed it by default. It’s available though an apt install zsys for the adventurous audience, and some Ubuntu flavors already jumped on the band wagon where it will be installed by default! Even if you won’t immediately see differences, this will unleash some of our grub, adduser and initramfs integration that are baked right in 19.10.

The excellent Ars Technica review by Jim Salter was wondering about the quite complex dataset layout we are setting up. We’ll shed some light on this on the next blog post which will explain what Zsys is really, what it does bring to the table and what our future plans are.

The future of ZFS on root on Ubuntu is bright, I’m personally really excited about what this is going to bring to both server and desktop users! (And yes, we can cook up some very nice features for our desktop users with ZFS)!

If you want to join the discussion, feel free to hop in our ubuntu discourse dedicated topic.