Benchmark of Ext4, XFS, Btrfs, ZFS With PostgreSQL

Most VPS hosting providers use Ext4 or XFS out of the box. There are other options of course and interestingly, the official Arch Linux cloud image uses Btrfs, with zstd compression. I like Btrfs and I think it’s a good match for a rolling release distribution targeted at developers such as Arch (and OpenSuse and recently Fedora).

• Pros
  • Snapshots to easily rollback when/if an update goes wrong
  • Easy backups
  • Container friendly
• Cons
  • Weak random read/write performance when used for a database or a virtual machine host

For development, the pros outweigh the cons and I wouldn’t mind the extra features on a web server. However in that case database performance is often the limiting factor. So I was curious, not whether Btrfs would outperform Ext4 and XFS but rather if it’s a viable choice.

Then there is ZFS, which is used by FreeBSD for good reasons (its previous filesystem UFS is about as good as Ext2) and is also promoted by Ubuntu these days.

Here are some benchmarks. First, the tests were run locally in a VM configured close to the specs of the VPS I was planning to use. Then some benchmarks on the VPS itself.

Tests on a VM

I used these instructions to benchmark Postgresql, it’s a basic benchmark with 10 and 100 clients. However, Btrfs and ZFS were tuned for a database as one would normally do.

• Specs
  • KVM, 2 CPU, 4 Gb RAM, 1 Gb disk
• Linux tuning
  • noatime
  • chattr +C for btrfs
  • ZFS tuned for Postgresql according to the arch wiki
  • No other Postgresql tuning
• FreeBSD tuning
  • Filesystem tuned similarly to Linux but without changing primarycache setting according to
    • https://redbyte.eu/en/blog/postgresql-benchmark-freebsd-centos-ubuntu-debian-opensuse/
    • https://news.ycombinator.com/item?id=16022258
  • ZFS uses lz4 by default

Tests were run using a raw preallocated image on a SSD Btrfs filesystem.

• TPS include connection establishing, rounded, higher is better.
• Latency in ms, lower is better.

10 clients

100 clients

Tests on a VPS

A variety of tests were run:

• Arch Linux using its official cloud image (Btrfs, no compression, has metadata and system DUP, noatime, autodefrag, virtio-block)
• Ubuntu as setup by the VPS provider (virtio-scsi)
• Arch Linux using a custom LVM + Btrfs + Ext4 for database
• Arch Linux with the -lts kernel (no preemption) and Ext4, XFS, Btrfs filesystems

Why Btrfs over LVM? It’s not as crazy as it sounds and it’s mentioned in the btrfs wiki as a possibility. The idea is to combine the best parts of a COW and a traditional filesystem at the cost of extra complexity.

10 clients

100 clients

Conclusion

These are a lot of numbers to absorb, but first a warning: This is a small synthetic test designed to measure a specific aspect of performance, database performance. The real performance of an application will depend on whether it is CPU bound, or network bound, or serving many files or finally the database performance. The only realistic benchmark is the one done on a real application in real conditions.

Having said that:

• Ext4 and XFS are the fastest, as expected. But they come with the smallest set of features compared to newer filesystems.
• Btrfs trails the other options for a database in terms of latency and throughput. But it is reasonably easy to setup and comes with useful features, such as snapshots and easy backups.
• ZFS performs ok, at least on a fresh installation. However I would expect it to get very fragmentated with time, as all COW filesystems do. It also uses a lot of memory which is limited on a VPS. And it’s complicated to setup if you do it yourself.
• The LVM+Btrfs+Ext4 combination performs well, but is not supported by cloud-utils and I had to do the resizing manually.

In other words, every choice is a compromise and one has to pick according to their priorities.