Benchmarking POSIX Performance

Fio (Flexible I/O Tester) is a powerful open-source tool for benchmarking the performance of storage systems. Because Alluxio can be mounted as a POSIX-compliant filesystem using FUSE, fio is an excellent tool for measuring its read/write IOPS and throughput.

This guide walks through how to set up and run fio to benchmark the performance of data cached in Alluxio.

Performance Highlights

The following results were achieved with a single Alluxio worker and a single client, based on the test environment detailed below. Your results may vary based on your specific hardware and setup.

Throughput (256k block size)

Bandwidth/Threads	Single Thread	32 Threads	128 Threads
Sequential Read	2101MiB/s	9519MiB/s	8089MiB/s
Random Read	202MiB/s	6684MiB/s	8276MiB/s

IOPS (4k block size)

IOPS/Threads	Single Thread	32 Threads	128 Threads
Sequential Read	55.9k	253k	192k
Random Read	2.3k	70.1k	162k

Test Environment and Setup

The following environment was used to generate the example results in this guide. All instances were located in the same AWS availability zone to minimize network latency.

Hardware and Software

• Alluxio Worker Node:

  • Instance: AWS i3en.metal

  • Storage: 8 NVMe SSDs in a RAID 0 array

  • OS: Ubuntu 24.04

• Alluxio Client Node:

  • Instance: AWS c5n.metal

  • OS: Ubuntu 24.04

  • FUSE: Version 3.16.2 or later

Fio Installation

On the client node, install fio using the appropriate package manager. For RPM-based distributions (like CentOS), use yum:

sudo yum install fio

For Debian-based distributions (like Ubuntu), use apt:

sudo apt-get install fio

Data Preparation

Before running the benchmark, you need a large test file. For this guide, a single 100GB file was placed in the UFS (an S3 bucket in the same region) and fully loaded into the Alluxio cache to ensure the benchmark measures the performance of cached reads.

Running Benchmark Tests

The following commands should be run on the Alluxio client node from a directory inside the FUSE mount point.

Testing Sequential Read Throughput

This test measures the maximum speed at which large, contiguous blocks of data can be read. It is representative of workloads like video streaming or large file processing. Use a larger block size (bs), such as 256k, for this test.

fio -iodepth=1 -rw=read -ioengine=libaio -bs=256k -numjobs=<numjobs> -group_reporting -size=100G -filename=/mnt/alluxio/100gb -name=read_test --readonly -direct=1 --runtime=60

Testing Random Read IOPS

This test measures how many random read operations can be performed per second. It is representative of workloads like databases or key-value stores. Use a small block size (bs), such as 4k, for this test.

fio -iodepth=1 -rw=randread -ioengine=libaio -bs=4k -numjobs=<numjobs> -group_reporting -size=100G -filename=/mnt/alluxio/100gb -name=read_test --readonly -direct=1 --runtime=60

Key fio parameters:

• rw: Specifies the I/O pattern (read for sequential, randread for random).

• bs: The block size for each I/O operation.

• numjobs: The number of concurrent threads to run.

• filename: The path to your test file within the FUSE mount.

• direct=1: Bypasses the OS page cache to ensure you are measuring the performance of Alluxio FUSE.