Resource Prerequisites and Compatibility

This guide covers the planning and installation of Alluxio, describing the necessary resource configurations for running Alluxio. It includes resource prerequisites such as CPU, memory, disk, and network requirements. Additionally, it describes compatibility considerations for both the operating system (server, client, and kernel).

Resource prerequisites

Alluxio on Kubernetes (k8s) is the recommended deployment type. The Alluxio coordinator, worker, and fuse components each require specific CPU, memory, and disk resources. Additionally, the entire Alluxio cluster needs specific network ports to be open for communication. Ensuring the proper configuration of these resources between components is key to achieving efficient operation of Alluxio.

ETCD is also required for service discovery and mounting tables. Please refer to the instructions for installing Alluxio on Kubernetes to set up ETCD.

Kubernetes deployment

In a Kubernetes environment, two sets of configurations are provided for deploying an Alluxio cluster: the minimum configuration and the preferred configuration.

1. The minimum configuration ensures that the Alluxio cluster can run and support basic IO functional testing.

2. The preferred configuration utilizes the default settings in the Alluxio operator deployment mode. It optimizes compute, memory, and storage resources for supporting basic Proof of Concept (PoC) scenarios.

These configurations are tailored to accommodate various use cases, offering flexibility and scalability while ensuring stable and efficient operation of the Alluxio cluster within Kubernetes environments.

Minimum and preferred

The table below compares the resources of the Minimum and Preferred configurations for the Alluxio system on K8S deployment.

Components	Minimum	Preferred (Operator default)
K8s Node	cpu: 8 CPU cores mem: 32G	cpu: 16 CPU cores mem: 64G
Alluxio coordinator	cpu request: 1 cpu limit: 1 mem request: 1G mem limit: 12G	cpu request: 1 cpu limit: 1 mem request: 8G mem limit: 10G java heap memory: 4G direct memory: 4G
Alluxio worker	cpu request: 1 cpu limit: 10 mem request: 512M mem limit: 20G	cpu request: 1 cpu limit: 10 mem request: 16G mem limit: 20G java heap memory: 8G direct memory: 8G
Alluxio fuse	cpu request: 1 cpu limit: 6 mem request: 512M mem limit: 16G	cpu request: 1 cpu limit: 6 mem request: 12G mem limit: 16G java heap memory: 4G direct memory: 8G
Page Store	100G	100G, but please adjust based on the number of workers and the actual data usage.

Network

From an architectural perspective, since Alluxio serves as a storage cache, it is recommended to deploy Alluxio components nearby the compute components that run the workload. We recommend deploying Alluxio in the same network. For high performance cases, additional network bandwidth and latency requirements need to be considered.

NOTE: In extreme cases Alluxio may require specialized high performance networking to support high throughput data transfer. Please contact us for detailed information based on your scenarios.

Compatibility

Operating system compatibility

Alluxio on K8s runs on pre-packaged container images and is able to be deployed on any container infrastructure environment including all cloud providers.

Kubernetes compatibility with the Host OS or container runtime is beyond the scope of this document. Please refer to and comply with the release notes of your selected Kubernetes distribution or version for detailed information.

Alluxio Cluster

To run Alluxio on Kubernetes, certain prerequisites must be met to ensure compatibility:

• A Kubernetes cluster with version at least 1.19, with feature gates enabled.

• Host OS libfuse 3.10+ for Linux

• Host architecture x86