Kubernetes Installation

This documentation shows how to install Alluxio on Kubernetes via Operator.

Overview

Artifacts

Your will receive download links for three artifacts:

Artifact

Filename

Purpose

Helm chart

alluxio-operator-3.5.2-helmchart.tgz

Deploys the Operator onto Kubernetes

Operator image

alluxio-operator-3.5.2-linux-amd64-docker.tar

Container image for the Operator pod

Alluxio image

alluxio-enterprise-AI-3.8-15.1.2-linux-amd64-docker.tar

Container image for Alluxio worker and coordinator pods

License

Required to activate the cluster

Platform: Use -linux-amd64-docker.tar for x86 nodes or -linux-arm64-docker.tar for ARM nodes.

The Docker images are never pulled from a public registry — they are loaded from the .tar files and pushed to your private registry before deployment.

Helm chart (.tgz)
  └─► deploys ─► Operator pod  (alluxio-operator image)
                    └─► watches AlluxioCluster CRD
                           └─► creates ─► Alluxio pods  (alluxio-enterprise image)

Kubernetes Components

A deployed Alluxio cluster consists of:

Operator — Manages the lifecycle of Alluxio clusters. Installed once per Kubernetes cluster.
Coordinator — Handles background operations (data loading, freeing). 1 replica.
Workers — Cache data and serve reads via S3 API or FUSE. Scale horizontally for more cache capacity.
ETCD — Service discovery and mount table storage. 3 replicas recommended for quorum.
Monitoring (optional) — Prometheus and Grafana. Enabled by default.

Before You Start

Run these checks before starting (~2 minutes). Skipping this step is the most common cause of deployment failures.

Kubernetes version meets requirements (see Prerequisites)
Helm is installed and version meets requirements (see Prerequisites):
helm version
kubectl is configured and pointing to the correct cluster:
kubectl cluster-info kubectl get nodes
Nodes available for each Alluxio component:
- Coordinator: at least 1 node
- Workers: at least 1 node
- ETCD: at least 1 node for testing, 3 nodes for quorum in production
StorageClass exists and supports dynamic provisioning (required for ETCD and coordinator metastore PVCs):
kubectl get storageclass
Platform differences may affect StorageClass and FUSE mount behavior. See Appendix G: Platform-Specific Notes for EKS, GKE, and kind requirements.
All three artifacts downloaded from the email link: Helm chart .tgz, operator image .tar, Alluxio image .tar
Private registry is available and your local Docker is authenticated to push to it
Alluxio license string is available
S3 bucket + credentials (if using S3) are available

Installation Steps

0. Push Alluxio Images to Your Private Registry

Skip this step if the Alluxio images are already present in your private registry.

Alluxio images are delivered as .tar files and must be loaded and pushed to your private registry before the Helm chart can deploy them.

Load the images into your local Docker:

docker load -i alluxio-operator-3.5.2-linux-amd64-docker.tar
docker load -i alluxio-enterprise-AI-3.8-15.1.2-linux-amd64-docker.tar

✅ Success: docker images shows both images:

REPOSITORY                    TAG
alluxio/operator              3.5.2
alluxio/alluxio-enterprise    AI-3.8-15.1.2

Retag and push to your private registry:

docker tag alluxio/operator:3.5.2 <PRIVATE_REGISTRY>/alluxio-operator:3.5.2
docker tag alluxio/alluxio-enterprise:AI-3.8-15.1.2 <PRIVATE_REGISTRY>/alluxio-enterprise:AI-3.8-15.1.2

docker push <PRIVATE_REGISTRY>/alluxio-operator:3.5.2
docker push <PRIVATE_REGISTRY>/alluxio-enterprise:AI-3.8-15.1.2

✅ Success: Both images are now in your private registry. Note down the full image paths — you will use them in Steps 1 and 4.

If your Kubernetes cluster also cannot reach public registries (air-gapped), third-party images (etcd, CSI) also need to be relocated. See Appendix A: Air-Gapped Deployment.

1. Prepare Helm Chart

Extract the Helm chart:

tar zxf alluxio-operator-3.5.2-helmchart.tgz

This creates the alluxio-operator directory containing the Helm chart.

Create alluxio-operator.yaml (outside the chart directory) to specify the operator image from your private registry:

global:
  image: <PRIVATE_REGISTRY>/alluxio-operator
  imageTag: 3.5.2

2. Create Namespace

kubectl create namespace alx-ns

3. Deploy Operator

cd alluxio-operator
helm -n alluxio-operator install operator -f alluxio-operator.yaml --create-namespace .

✅ Success: Helm prints STATUS: deployed immediately after the command completes:

NAME: operator
STATUS: deployed
REVISION: 1

Then verify all pods are running:

kubectl -n alluxio-operator get pod

✅ Success: All operator pods show READY 1/1 or 2/2, STATUS = Running, and RESTARTS = 0.

An example output is like:

NAME                                              READY   STATUS    RESTARTS   AGE
alluxio-cluster-controller-5db4f967f5-hg4v4       1/1     Running   0          30s
alluxio-console-5c7ff88b88-s4x4c                  1/1     Running   0          30s
alluxio-csi-controller-77d8d7bd56-27tqh           2/2     Running   0          30s
alluxio-csi-nodeplugin-2pzrq                      2/2     Running   0          31s
alluxio-csi-nodeplugin-94mjw                      2/2     Running   0          31s
alluxio-csi-nodeplugin-tgs86                      2/2     Running   0          31s
alluxio-doctor-controller-64847d459c-pczft        1/1     Running   0          30s
alluxio-license-controller-6c76b59f9b-lfkmz       1/1     Running   0          30s
alluxio-ufs-controller-5c8bdf48c9-tnw48           1/1     Running   0          30s

If pods fail with image pull errors on etcd or CSI images, see Appendix A: Air-Gapped Deployment.

4. Deploy Cluster

Create a minimal alluxio-cluster.yaml:

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
metadata:
  name: alluxio-cluster
  namespace: alx-ns
spec:
  image: <PRIVATE_REGISTRY>/alluxio-enterprise
  imageTag: AI-3.8-15.1.2
  properties:
    alluxio.license: <YOUR_CLUSTER_LICENSE>
  worker:
    count: 1
    pagestore:
      size: 90Gi       # Must be less than the node's available disk space. Check with: df -h /mnt/alluxio
      reservedSize: 9Gi
  etcd:
    replicaCount: 3

Deploy:

kubectl create -f alluxio-cluster.yaml

✅ Success: Startup typically takes 2–3 minutes. The first deployment may take longer if the Alluxio image (~1.8 GB) needs to be pulled from the registry. To watch progress in real time:

kubectl -n alx-ns get pod --watch

Once all pods are running, kubectl -n alx-ns get alluxiocluster shows CLUSTERPHASE = Ready

NAME              CLUSTERPHASE   AGE
alluxio-cluster   Ready          2m18s

and kubectl -n alx-ns get pod shows all pods to become Ready and STATUS = Running.

NAME                                          READY   STATUS    RESTARTS   AGE
alluxio-cluster-coordinator-0                 1/1     Running   0          2m3s
alluxio-cluster-etcd-0                        1/1     Running   0          2m3s
alluxio-cluster-etcd-1                        1/1     Running   0          2m3s
alluxio-cluster-etcd-2                        1/1     Running   0          2m3s
alluxio-cluster-worker-0                      1/1     Running   0          2m3s
alluxio-cluster-grafana-585d767c84-p8wcp      1/1     Running   0          2m3s
alluxio-cluster-prometheus-6f697b6db8-sbvvg   1/1     Running   0          2m3s

If any component fails to start, see Appendix F: Troubleshooting.

5. Mount Storage

Create ufs.yaml (S3 example; for other storage systems, see Underlying Storage):

apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: alluxio-s3
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: s3://<S3_BUCKET>/<S3_DIRECTORY>
  mountPath: /s3
  mountOptions:
    s3a.accessKeyId: <S3_ACCESS_KEY_ID>
    s3a.secretKey: <S3_SECRET_KEY>
    alluxio.underfs.s3.region: <S3_REGION>

Apply:

kubectl create -f ufs.yaml

✅ Success: kubectl -n alx-ns get ufs shows PHASE = Ready.

Example out:

NAME         PHASE   AGE
alluxio-s3   Ready   32s

6. Verify Cluster

kubectl -n alx-ns exec -i alluxio-cluster-coordinator-0 -- alluxio mount list

✅ Success: Output displays your mount point (e.g., s3://my-bucket/... on /s3/).

Example out:

Defaulted container "alluxio-coordinator" out of: alluxio-coordinator, path-permission (init), wait-etcd (init)
Listing all mount points
s3://<S3_BUCKET>/  on  /s3/ properties={alluxio.underfs.s3.region=<S3_REGION>}

7. Verify Data Access

kubectl -n alx-ns exec -i alluxio-cluster-coordinator-0 -- alluxio fs ls /

✅ Success: Returns a directory listing without errors.

Alluxio provides several APIs for applications to access data:

POSIX API via FUSE — Mount Alluxio as a local filesystem. See FUSE Guide.
S3 API — S3-compatible endpoint. See S3 API Guide.
Python API via FSSpec — Native Python interface. See FSSpec Guide.

Uninstall

To remove the Alluxio deployment from your cluster, run the following commands in order:

1. Delete the UFS mount and cluster:

kubectl delete -f ufs.yaml
kubectl delete -f alluxio-cluster.yaml

2. Uninstall the operator:

cd alluxio-operator
helm -n alluxio-operator uninstall operator

3. Delete the namespaces:

kubectl delete namespace alx-ns
kubectl delete namespace alluxio-operator

✅ Success: kubectl get namespace no longer shows alx-ns or alluxio-operator, and kubectl get alluxiocluster -A returns No resources found.

Recommended Production Configuration

The basic configuration in Step 4 is suitable for evaluation. For production deployments, apply the following additional settings for HA, resource tuning, and persistent metadata.

Label Nodes

A common practise is to assign dedicated nodes to each Alluxio component. This prevents resource contention between components (for example, etcd I/O interfering with worker cache I/O) and gives you predictable placement for capacity planning.

kubectl label nodes <coordinator-node> alluxio-role=coordinator
kubectl label nodes <worker-node-1> alluxio-role=worker
kubectl label nodes <worker-node-2> alluxio-role=worker
kubectl label nodes <worker-node-3> alluxio-role=worker
kubectl label nodes <etcd-node-1> alluxio-role=etcd
kubectl label nodes <etcd-node-2> alluxio-role=etcd
kubectl label nodes <etcd-node-3> alluxio-role=etcd

Worker pods have an anti-affinity rule by default — multiple worker pods will not be scheduled on the same node.

Production `alluxio-cluster.yaml`

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
metadata:
  name: alluxio-cluster
  namespace: alx-ns
spec:
  image: <PRIVATE_REGISTRY>/alluxio-enterprise
  imageTag: AI-3.8-15.1.2
  properties:
    alluxio.license: <YOUR_CLUSTER_LICENSE>
  coordinator:
    nodeSelector:
      alluxio-role: coordinator
    metastore:
      type: persistentVolumeClaim
      storageClass: "gp2"
      size: 4Gi
    resources:
      limits:
        cpu: "8"
        memory: "16Gi"
      requests:
        cpu: "4"
        memory: "12Gi"
    jvmOptions:
      - "-Xmx8g"
      - "-Xms8g"
  worker:
    nodeSelector:
      alluxio-role: worker
    count: 3
    pagestore:
      size: 1000Gi
      reservedSize: 100Gi
    resources:
      limits:
        cpu: "8"
        memory: "24Gi"
      requests:
        cpu: "4"
        memory: "12Gi"
    jvmOptions:
      - "-Xmx12g"
      - "-Xms12g"
      - "-XX:MaxDirectMemorySize=12g"
  etcd:
    replicaCount: 3
    nodeSelector:
      alluxio-role: etcd

Key differences from the basic configuration:

Node selectors: Pin each component to dedicated nodes to prevent resource contention and ensure predictable placement. See the label commands above.
Worker count: number of workers depending on the target cache volume and target throughput.
ETCD replicas: 3 for quorum-based HA. Deploy on dedicated, stable nodes.
Resource limits and JVM options: Explicitly set to prevent OOM. The container memory limit must exceed the sum of -Xmx and -XX:MaxDirectMemorySize.
Persistent metastore: Coordinator metadata survives pod restarts.

Other important settings for production deployment:

License Management: A cluster license is the simplest way to get started. For production environments, a deployment license is recommended. See Appendix E: License Management for details on both options.
Hash Ring Configuration: It is critical to configure the hash ring before deployment, as changes can be destructive. For detailed guidance, see Appendix B: Handling Hash Ring.
Heterogeneous Clusters: If your cluster includes workers with different capacities, you must define a specific data distribution strategy. See Appendix C: Handling Heterogeneous Workers for configuration steps.
Worker Page Store Sizing: Properly configure the pagestore on your workers. The size parameter sets the cache capacity, while reservedSize allocates space for internal operations, including temporary page writes and file metadata caching. We recommend setting reservedSize to ~10% of size (10–100 GiB) and ensuring the total (size + reservedSize) fits within the worker's storage.
Advanced Configuration: For other settings, such as resource and JVM tuning or using an external etcd, refer to Appendix D: Advanced Configuration.

Appendix

Use the table below to find the relevant appendix section for your scenario:

Scenario

Sections

Air-gapped (cluster cannot reach public registries)

A. Air-Gapped Deployment, D.9. Image Pull Secrets

Heterogeneous cluster (mixed worker disks)

B. Hash Ring, C. Heterogeneous Workers

External or custom ETCD

D.5. External ETCD, D.6. Customize ETCD

Performance tuning

D.2. Resource & JVM, D.3. PVC Page Store

Multi-cluster on shared nodes

D.7. nodeSelector

Production licensing

E. License Management

Something went wrong

F. Troubleshooting

A. Air-Gapped Deployment

Symptom: After deploying the operator or cluster, some pods are stuck in ImagePullBackOff. Your Kubernetes cluster cannot reach public registries to pull third-party component images (CSI, etcd, monitoring).

Alluxio images are already in your private registry from Prepare: Push Alluxio Images. The remaining images to relocate depend on your specific operator version. Identify them by inspecting the stuck pods:

kubectl get pods -A | grep -E "ImagePullBackOff|ErrImagePull"
kubectl describe pod <pod-name> -n <namespace> | grep "image:"

For each image that cannot be pulled: pull it from a machine with public internet access, retag it for your private registry, and push it. Then update alluxio-operator.yaml or alluxio-cluster.yaml to point to your private registry for that component.

The CSI images (part of the operator) can be overridden in alluxio-operator.yaml:

alluxio-csi:
  controllerPlugin:
    provisioner:
      image: <PRIVATE_REGISTRY>/csi-provisioner:<TAG>
  nodePlugin:
    driverRegistrar:
      image: <PRIVATE_REGISTRY>/csi-node-driver-registrar:<TAG>

The etcd image (part of the cluster) can be overridden in alluxio-cluster.yaml:

spec:
  etcd:
    image:
      registry: <PRIVATE_REGISTRY>
      repository: <PRIVATE_REPOSITORY>/etcd
      tag: <TAG>

B. Handling Hash Ring

The consistent hash ring determines how data is mapped to workers. It is critical to define your hash ring strategy before deploying the cluster, as changing these settings later is a destructive operation that will cause all cached data to be lost.

Key properties to consider, which should be set in alluxio-cluster.yaml under .spec.properties:

Hash Ring Mode (alluxio.user.dynamic.consistent.hash.ring.enabled):
- true (Default): Dynamic mode. Includes only online workers. Best for most environments.
- false: Static mode. Includes all registered workers, online or offline. Use if you need a stable ring view despite temporary worker unavailability.
Virtual Nodes (alluxio.user.worker.selection.policy.consistent.hash.virtual.node.count.per.worker):
- Default: 2000. Controls load balancing granularity.
Worker Capacity (alluxio.user.worker.selection.policy.consistent.hash.provider.impl):
- DEFAULT (Default): Assumes all workers have equal capacity.
- CAPACITY: Allocates virtual nodes based on worker storage capacity. Use this for heterogeneous clusters.

For more details, see Hash Ring Management.

C. Handling Heterogeneous Workers

The Alluxio operator allows you to manage heterogeneous worker configurations, which is particularly useful for clusters where nodes have different disk specifications. This feature enables you to define distinct worker groups, each with its own storage settings.

Note: While this provides flexibility, it is crucial to ensure consistency within each worker group. Misconfigurations can lead to unexpected errors. This guide covers the supported use case of configuring workers with different disk setups.

To set up heterogeneous workers, follow these steps:

Group Nodes by Specification: First, identify and group your Kubernetes nodes based on their disk configurations. For example, you might have one group of 10 nodes with a single 1TB disk and another group of 12 nodes with two 800GB disks.

Label the Nodes: Assign unique labels to each group of nodes. This allows you to target specific configurations to the correct machines.

# Label nodes with one disk
kubectl label nodes <node-name> apps.alluxio.com/disks=1
# Label nodes with two disks
kubectl label nodes <node-name> apps.alluxio.com/disks=2

Define Worker Groups and Enable Capacity-Based Hashing: In your alluxio-cluster.yaml, use the .spec.workerGroups field to define each group. Use a nodeSelector to apply the specific configuration to the nodes with the corresponding label.

For heterogeneous clusters, it is also recommended to configure the hash ring to be capacity-aware. This ensures that workers with more storage capacity are allocated a proportionally larger share of data. You can do this by setting alluxio.user.worker.selection.policy.consistent.hash.provider.impl to CAPACITY.

The example below shows a complete configuration for a heterogeneous cluster:

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  properties:
    alluxio.user.worker.selection.policy.consistent.hash.provider.impl: CAPACITY

  # Common configurations for all workers
  worker:
    resources:
      limits:
        memory: 40Gi
      requests:
        memory: 36Gi
    jvmOptions: ["-Xmx20g", "-Xms20g", "-XX:MaxDirectMemorySize=16g"]

  # Define specific configurations for each worker group
  workerGroups:
  - worker:
      count: 10
      nodeSelector:
        apps.alluxio.com/disks: 1
      pagestore:
        hostPath: /mnt/disk1/alluxio/pagestore
        size: 1Ti
        reservedSize: 100Gi
  - worker:
      count: 12
      nodeSelector:
        apps.alluxio.com/disks: 2
      pagestore:
        hostPath: /mnt/disk1/alluxio/pagestore,/mnt/disk2/alluxio/pagestore
        size: 800Gi,800Gi
        reservedSize: 100Gi

D. Advanced Configuration

This section describes common configurations to adapt to different scenarios.

D.1. Configuring Alluxio Properties

To modify Alluxio's configuration, edit the .spec.properties field in the alluxio-cluster.yaml file. These properties are appended to the alluxio-site.properties file inside the Alluxio pods.

D.2. Resource and JVM Tuning

You can configure resource limits and JVM options for each component.

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  worker:
    count: 2
    resources:
      limits:
        cpu: "12"
        memory: "36Gi"
      requests:
        cpu: "1"
        memory: "32Gi"
    jvmOptions:
      - "-Xmx22g"
      - "-Xms22g"
      - "-XX:MaxDirectMemorySize=10g"
  coordinator:
    resources:
      limits:
        cpu: "12"
        memory: "36Gi"
      requests:
        cpu: "1"
        memory: "32Gi"
    jvmOptions:
      - "-Xmx4g"
      - "-Xms1g"

Memory limit formula:

memory limit ≥ -Xmx + -XX:MaxDirectMemorySize + 2–4 GiB (JVM overhead)

For the worker config above (-Xmx22g, -XX:MaxDirectMemorySize=10g): minimum limit is 22 + 10 + 2 = 34 GiB, set to 36 GiB in the example.

If -XX:MaxDirectMemorySize is omitted, the JVM defaults it to the same value as -Xmx, so the container limit typically needs to be 2.5× -Xmx or more.

Diagnosing OOM

If a worker pod is killed due to OOM (exit code 137), use these commands to confirm the cause:

# Check pod events
kubectl describe pod <worker-pod-name> -n alx-ns
# Look for "OOMKilled" or "Exit Code: 137"

# Check the previous container's logs (before crash)
kubectl logs <worker-pod-name> -n alx-ns --previous | tail -50
# Look for "OutOfMemoryError" or "java.lang.OutOfMemoryError"

Symptom

Root Cause

Fix

Exit Code 137, no Java error

Container limit exceeded — killed by Linux OOM killer

Increase resources.limits.memory

java.lang.OutOfMemoryError: Java heap space

-Xmx too small

Increase -Xmx and raise container limit accordingly

java.lang.OutOfMemoryError: Direct buffer memory

-XX:MaxDirectMemorySize too small

Increase -XX:MaxDirectMemorySize and raise container limit accordingly

D.3. Use PVC for Page Store

To persist worker cache data, specify a PersistentVolumeClaim (PVC) for the page store.

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  worker:
    pagestore:
      type: persistentVolumeClaim
      storageClass: "" # Defaults to "standard", can be empty for static binding
      size: 100Gi
      reservedSize: 10Gi

D.4. Mount Custom ConfigMaps or Secrets

You can mount custom ConfigMap or Secret files into your Alluxio pods. This is useful for providing configuration files like core-site.xml or credentials.

Example: Mount a Secret

Create the secret from a local file:

kubectl -n alx-ns create secret generic my-secret --from-file=/path/to/my-file

Specify the secret to load and the mount path in alluxio-cluster.yaml:

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  secrets:
    worker:
      my-secret: /opt/alluxio/secret
    coordinator:
      my-secret: /opt/alluxio/secret

The file my-file will be available at /opt/alluxio/secret/my-file on the pods.

D.5. Use External ETCD

If you have an external ETCD cluster, you can configure Alluxio to use it instead of the one deployed by the operator.

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  etcd:
    enabled: false
  properties:
    alluxio.etcd.endpoints: http://external-etcd:2379
    # If using TLS for ETCD, add the following:
    # alluxio.etcd.tls.enabled: "true"

D.6. Customize ETCD configuration

The fields under spec.etcd follow the Bitnami ETCD helm chart. For example, to set node affinity for etcd pods, the affinity field can be used as described in the Kubernetes documentation.

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  etcd:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: topology.kubernetes.io/zone
              operator: In
              values:
              - antarctica-east1
              - antarctica-west1

D.7. nodeSelector

The nodeSelector field allows you to control which nodes Kubernetes schedules pods to. For instructions on labeling nodes and applying node selectors in production, see Recommended Production Configuration.

Additional scenario: Multiple clusters

If multiple Alluxio clusters are deployed and different clusters belong to different namespaces, services from different clusters may be scheduled by Kubernetes to the same node, causing deployment failures. You can label different nodes to indicate which cluster the node belongs to:

kubectl label nodes <node-name> cluster=alluxio-a
kubectl label nodes <node-name> cluster=alluxio-b

And specify the nodeSelector at the cluster level in your cluster.yaml:

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  image: <PRIVATE_REGISTRY>/alluxio-enterprise
  imageTag: AI-3.8-15.1.2
  nodeSelector:
    cluster: alluxio-a

D.8. Prepare Namespace

If you want to install Alluxio in a custom namespace (e.g., alluxio-test), creating the namespace is required before installation.

kubectl create namespace alluxio-test

D.9. Configure Image Pull Secrets

If your container images are stored in a private registry that requires authentication, you need to create a Kubernetes Secret to store your registry credentials.

This secret must be created in the namespace where you plan to install Alluxio.

kubectl create secret docker-registry <SECRET_NAME> \
  --docker-server=<REGISTRY_SERVER> \
  --docker-username=<USERNAME> \
  --docker-password=<PASSWORD> \
  -n <NAMESPACE>

<SECRET_NAME>: Name of the secret (e.g., alluxio-image-pull-secret).
<REGISTRY_SERVER>: Your private registry server address (e.g., https://index.docker.io/v1/ for Docker Hub).
<USERNAME>: Your registry username.
<PASSWORD>: Your registry password.
<NAMESPACE>: The namespace where Alluxio will be installed.

Once created, you can verify the secret exists:

kubectl get secret <SECRET_NAME> -n <NAMESPACE>

E. License Management

Alluxio requires a license provided by your sales representative. There are two types: a cluster license (for single test clusters) and a deployment license (recommended for production).

E.1. Cluster License

A cluster license is set directly in the alluxio-cluster.yaml file. This method is not recommended for production.

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  properties:
    alluxio.license: <YOUR_CLUSTER_LICENSE>

E.2. Deployment License

A deployment license is the recommended method for production and can cover multiple clusters. It is applied by creating a separate License resource after the cluster has been created.

Step 1: Create the Cluster without a License Deploy the Alluxio cluster as described in Step 5 of the main guide, but do not include the alluxio.license property in alluxio-cluster.yaml. The pods will start but remain in an Init state, waiting for the license.

Step 2: Apply the License Create an alluxio-license.yaml file. The name and namespace in this file must match the metadata of your AlluxioCluster.

apiVersion: k8s-operator.alluxio.com/v1
kind: License
metadata:
  name: alluxio-license
  namespace: alx-ns
spec:
  clusters:
  - name: alluxio-cluster
    namespace: alx-ns
  licenseString: <YOUR_DEPLOYMENT_LICENSE>

Apply this file with kubectl create -f alluxio-license.yaml. The Alluxio pods will detect the license and transition to Running.

Warning: Only specify running clusters in the clusters list. If the operator cannot find a listed cluster, the license operation will fail for all clusters.

E.3. Updating a Deployment License

To update an existing deployment license, update the licenseString in your alluxio-license.yaml and re-apply it:

kubectl delete -f alluxio-license.yaml
kubectl create -f alluxio-license.yaml

E.4. Checking License Status

You can check the license details and utilization from within the Alluxio coordinator pod.

# Get a shell into the coordinator pod
kubectl -n alx-ns exec -it alluxio-cluster-coordinator-0 -- /bin/bash

# View license details (expiration, capacity)
alluxio license show

# View current utilization
alluxio license status

F. Troubleshooting

F.1. etcd pod stuck in pending status

If etcd pods are Pending, it is often due to storage issues. Use kubectl describe pod <etcd-pod-name> to check events.

Symptom: Event message shows pod has unbound immediate PersistentVolumeClaims.

Cause: No storageClass is set for the PVC, or no PV is available.

Solution: Specify a storageClass in alluxio-cluster.yaml:

spec:
  etcd:
    persistence:
      storageClass: <YOUR_STORAGE_CLASS>
      size: 10Gi # Example size

Then, delete the old cluster and PVCs before recreating the cluster.

Symptom: Event message shows waiting for first consumer.

Cause: The storageClass does not support dynamic provisioning, and a volume must be manually created by an administrator.

Solution: Either use a dynamic provisioner or manually create a PersistentVolume that satisfies the claim.

F.2. etcd pods stuck in Pending due to anti-affinity (fewer than 3 nodes)

Symptom: etcd pods are Pending with event message 0/N nodes are available: N node(s) didn't match pod anti-affinity rules.

Cause: The operator deploys etcd with requiredDuringSchedulingIgnoredDuringExecution anti-affinity by hostname. With etcd.replicaCount: 3 (the default), Kubernetes requires 3 distinct nodes. If your cluster has fewer than 3 nodes, etcd pods cannot be scheduled.

Solution: For dev/test clusters with fewer than 3 nodes, reduce the replica count:

spec:
  etcd:
    replicaCount: 1   # For single-node dev/test only; use 3 for production

Do not use replicaCount: 1 in production — a single etcd instance has no quorum and is not fault-tolerant.

F.3. alluxio-cluster-fuse PVC in pending status

The alluxio-cluster-fuse PVC remaining in a Pending state is normal. It will automatically bind to a volume and become Bound once a client application pod starts using it.

F.3. Worker pod stuck in CrashLoopBackOff

Symptom: Worker pod repeatedly crashes and restarts.

Start by checking the worker logs:

kubectl -n alx-ns logs <worker-pod-name> --previous

Common causes include:

Pagestore quota exceeds disk space — log shows quota (NNN) exceeds the total disk space. This commonly occurs because cloud providers advertise disk size in GB (base-10), while Kubernetes interprets Gi as GiB (base-2). Fix: reduce pagestore.size to ~90% of actual available space (df -h /mnt/alluxio) and reservedSize to ~10% of size.
License expired or invalid — log shows a license error. Fix: apply a new license. See Appendix E: License Management.
OOM killed — log shows Exit Code 137 or OutOfMemoryError. Fix: increase container memory limits and adjust -Xmx / -XX:MaxDirectMemorySize. See D.2. Resource and JVM Tuning.

G. Platform-Specific Notes

The main installation steps are platform-agnostic. This section documents known differences for specific Kubernetes environments.

Amazon EKS

EBS CSI driver required for EKS 1.23+: The in-tree EBS volume driver was removed in Kubernetes 1.23. On EKS 1.23+, install the AWS EBS CSI driver add-on, or PVC provisioning will silently fail even if a StorageClass is listed.

Verify:

kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver --no-headers | head -1

If no output, the driver is not installed.

Google GKE

Read-only /mnt path: GKE nodes have a read-only root filesystem. Multiple Alluxio components default to hostPaths under /mnt/alluxio/, causing worker pods to fail with:

MountVolume.SetUp failed for volume "pagestore-0" : mkdir /mnt/alluxio: read-only file system
MountVolume.SetUp failed for volume "logs" : mkdir /mnt/alluxio: read-only file system
MountVolume.SetUp failed for volume "metastore" : mkdir /mnt/alluxio: read-only file system

Workaround: Redirect all hostPaths to a writable base directory (e.g., /home/alluxio/) in alluxio-cluster.yaml:

spec:
  coordinator:
    metastore:
      hostPath: /home/alluxio/metastore
    log:
      hostPath: /home/alluxio/logs
  worker:
    pagestore:
      hostPath: /home/alluxio/pagestore
    metastore:
      hostPath: /home/alluxio/metastore
    log:
      hostPath: /home/alluxio/logs
  fuse:
    log:
      hostPath: /home/alluxio/logs/fuse
    hostPathForMigration: /home/alluxio/migration
  gateway:
    log:
      hostPath: /home/alluxio/logs/gateway
  prometheus:
    persistence:
      hostPath: /home/alluxio/prometheus

Additionally, configure worker identity persistence to prevent workers from registering as new instances after each restart (leaving stale OFFLINE workers behind):

spec:
  worker:
    useExternalId: false
    systemInfo:
      hostPath: /home/alluxio/system-info

kind (Local Development)

Image loading: kind load docker-image can fail for multi-platform images with digest errors. Use the following workaround:

docker save <IMAGE> | docker exec -i <KIND_CONTAINER> ctr --namespace=k8s.io images import --snapshotter=overlayfs -

To find the kind container name: docker ps | grep kindest.

Last updated 1 day ago

hashtagOverview

hashtagArtifacts

hashtagKubernetes Components

hashtagBefore You Start

hashtagInstallation Steps

hashtag0. Push Alluxio Images to Your Private Registry

hashtag1. Prepare Helm Chart

hashtag2. Create Namespace

hashtag3. Deploy Operator

hashtag4. Deploy Cluster

hashtag5. Mount Storage

hashtag6. Verify Cluster

hashtag7. Verify Data Access

hashtagUninstall

hashtagRecommended Production Configuration

hashtagLabel Nodes

hashtagProduction alluxio-cluster.yaml

hashtagAppendix

hashtagA. Air-Gapped Deployment

hashtagB. Handling Hash Ring

hashtagC. Handling Heterogeneous Workers

hashtagD. Advanced Configuration

hashtagD.1. Configuring Alluxio Properties

hashtagD.2. Resource and JVM Tuning

hashtagDiagnosing OOM

hashtagD.3. Use PVC for Page Store

hashtagD.4. Mount Custom ConfigMaps or Secrets

hashtagD.5. Use External ETCD

hashtagD.6. Customize ETCD configuration

hashtagD.7. nodeSelector

hashtagD.8. Prepare Namespace

hashtagD.9. Configure Image Pull Secrets

hashtagE. License Management

hashtagE.1. Cluster License

hashtagE.2. Deployment License

hashtagE.3. Updating a Deployment License

hashtagE.4. Checking License Status

hashtagF. Troubleshooting

hashtagF.1. etcd pod stuck in pending status

hashtagF.2. etcd pods stuck in Pending due to anti-affinity (fewer than 3 nodes)

hashtagF.3. alluxio-cluster-fuse PVC in pending status

hashtagF.3. Worker pod stuck in CrashLoopBackOff

hashtagG. Platform-Specific Notes

hashtagAmazon EKS

hashtagGoogle GKE

hashtagkind (Local Development)

Overview

Artifacts

Kubernetes Components

Before You Start

Installation Steps

0. Push Alluxio Images to Your Private Registry

1. Prepare Helm Chart

2. Create Namespace

3. Deploy Operator

4. Deploy Cluster

5. Mount Storage

6. Verify Cluster

7. Verify Data Access

Uninstall

Recommended Production Configuration

Label Nodes

Production `alluxio-cluster.yaml`

Appendix

A. Air-Gapped Deployment

B. Handling Hash Ring

C. Handling Heterogeneous Workers

D. Advanced Configuration

D.1. Configuring Alluxio Properties

D.2. Resource and JVM Tuning

Diagnosing OOM

D.3. Use PVC for Page Store

D.4. Mount Custom ConfigMaps or Secrets

D.5. Use External ETCD

D.6. Customize ETCD configuration

D.7. nodeSelector

D.8. Prepare Namespace

D.9. Configure Image Pull Secrets

E. License Management

E.1. Cluster License

E.2. Deployment License

E.3. Updating a Deployment License

E.4. Checking License Status

F. Troubleshooting

F.1. etcd pod stuck in pending status

F.2. etcd pods stuck in Pending due to anti-affinity (fewer than 3 nodes)

F.3. alluxio-cluster-fuse PVC in pending status

F.3. Worker pod stuck in CrashLoopBackOff

G. Platform-Specific Notes

Amazon EKS

Google GKE

kind (Local Development)