Docker Installation

Deploy Alluxio on bare-metal Linux hosts, EC2 instances, or Slurm-managed clusters using Docker — no Kubernetes required.

Overview

Architecture

Each Alluxio component runs in its own Docker container using --net=host, sharing the host network stack. Components communicate directly by IP — no port mapping needed.

Host	Containers
Coordinator node	ETCD, Alluxio Coordinator, Prometheus, Grafana
Worker node(s)	Alluxio Worker (one per host)
FUSE client node	Alluxio FUSE (one per host)

Artifacts

You will receive a download link for two artifacts:

Artifact	Filename	Purpose
Alluxio image	alluxio-enterprise-AI-3.8-15.1.2-linux-amd64-docker.tar	Single image for coordinator, worker, and FUSE roles
License		Required to activate the cluster

Platform: Use -linux-amd64-docker.tar for x86 hosts, -linux-arm64-docker.tar for ARM.

The same image is loaded on every node. The role — coordinator, worker, or fuse — is set by the argument passed to docker run.

Before You Start

Run these checks before starting. Skipping them is the most common cause of deployment failures.

• Docker installed and running on every host:

  docker --version
  docker info

• Network connectivity between all hosts — test from each:

  ping <COORDINATOR_PRIVATE_IP>

• Firewall / security groups open the following ports between all hosts:

  Port	Component	Purpose
  2379	ETCD	Client requests
  2380	ETCD	Peer communication
  19998	Coordinator	RPC
  19999	Coordinator	Web UI / metrics
  29998	Worker	S3 API
  29999	Worker	RPC
  30000	Worker	Web UI / metrics
  49998	FUSE	RPC
  49999	FUSE	Metrics

• Alluxio image .tar file downloaded and available

• Alluxio license string available

• UFS credentials ready (S3 access key/secret, or IAM role attached to the instance)

EC2 + IAM roles: Attach the IAM instance profile before launch. No access keys are needed in the docker run commands.

Installation Steps

0. Load the Alluxio Image

Run on every host (coordinator, each worker, FUSE client):

docker load -i alluxio-enterprise-AI-3.8-15.1.2-linux-amd64-docker.tar

✅ Verify:

docker images | grep alluxio

alluxio/alluxio-enterprise   AI-3.8-15.1.2   4091c3d8dbc4   ...   2.58GB

Note the image name and tag — you will use them in every subsequent docker run command.

1. Start ETCD

SSH into the ETCD node:

docker run -p 2379:2379 -p 2380:2380 -d --name etcd-standalone \
  quay.io/coreos/etcd:v3.5.9 etcd \
  --listen-client-urls http://0.0.0.0:2379 \
  --advertise-client-urls http://<COORDINATOR_PRIVATE_IP>:2379

Use the private IP for --advertise-client-urls — not the public DNS or hostname — to ensure workers on other hosts can reach ETCD reliably.

For production, run a 3-node ETCD cluster for high availability. Single-node is suitable for evaluation only.

✅ Verify:

docker exec etcd-standalone etcdctl endpoint status --cluster -w table

+-----------------------------------+------------------+---------+---------+-----------+
|             ENDPOINT              |        ID        | VERSION | DB SIZE | IS LEADER |
+-----------------------------------+------------------+---------+---------+-----------+
| http://172.31.31.133:2379         | 8e9e05c52164694d |   3.5.9 |   20 kB |      true |
+-----------------------------------+------------------+---------+---------+-----------+

2. Start Coordinator (Coordinator host)

docker run -d --net=host --name=alluxio-coordinator \
  -e ALLUXIO_JAVA_OPTS="\
    -Dalluxio.license=<YOUR_LICENSE> \
    -Dalluxio.etcd.endpoints=http://<COORDINATOR_PRIVATE_IP>:2379 \
    -Dalluxio.coordinator.hostname=<COORDINATOR_PRIVATE_IP> \
    -Dalluxio.mount.table.source=ETCD" \
  alluxio/alluxio-enterprise:AI-3.8-15.1.2 coordinator

Key properties:

Property	Purpose
alluxio.license	License string
alluxio.etcd.endpoints	ETCD address (private IP + port 2379)
alluxio.coordinator.hostname	Private IP workers use to register
alluxio.mount.table.source=ETCD	Persist mount table in ETCD across restarts

✅ Verify:

docker logs alluxio-coordinator 2>&1 | grep -i "started\|listening\|etcd"

No ERROR lines in the first 30 seconds indicates a healthy start.

3. Start Workers (each Worker host)

SSH into each worker host:

docker run -d --net=host --name=alluxio-worker \
  -e ALLUXIO_JAVA_OPTS="\
    -Xmx8g -Xms2g -XX:MaxDirectMemorySize=8g \
    -Dalluxio.license=<YOUR_LICENSE> \
    -Dalluxio.etcd.endpoints=http://<COORDINATOR_PRIVATE_IP>:2379 \
    -Dalluxio.coordinator.hostname=<COORDINATOR_PRIVATE_IP> \
    -Dalluxio.mount.table.source=ETCD \
    -Dalluxio.worker.page.store.dirs=/data/alluxio-cache \
    -Dalluxio.worker.page.store.sizes=<CACHE_SIZE>" \
  alluxio/alluxio-enterprise:AI-3.8-15.1.2 worker

Set <CACHE_SIZE> to ~80% of available space on /data/alluxio-cache (e.g., 50GB, 200GB, 1TB). Avoid /tmp — it is cleared on reboot.

S3 API (optional): Add -Dalluxio.worker.s3.api.enabled=true to enable the S3-compatible endpoint on each worker (port 29998). Only needed if clients will access Alluxio via the S3 API.

JVM sizing — Alluxio stores cached data on disk, not in heap. The JVM heap does not need to be large relative to cache size:

Host RAM	-Xmx	-XX:MaxDirectMemorySize
16 GB	4g	4g
32 GB	8g	8g
64 GB	16g	16g
128 GB+	32g	32g

✅ Verify (from coordinator host):

docker exec alluxio-coordinator alluxio info nodes

WorkerId                                         Address                          Status
worker-15ed4a17-2154-4454-ba0b-32b46ff06bfb     ip-172-31-26-247...:29999        ONLINE
worker-704c4d42-d189-41ff-b6f5-775a7d1551b3     ip-172-31-18-67...:29999         ONLINE

Workers may take 10–15 seconds to register after starting.

4. Mount Storage

Run alluxio mount add from any host that has access to the coordinator. For full UFS configuration options, see Underlying Storage.

S3 with IAM role (recommended on EC2):

docker exec alluxio-coordinator alluxio mount add \
  --path /s3 \
  --ufs-uri s3://<S3_BUCKET>/ \
  --option alluxio.underfs.s3.region=<S3_REGION>

S3 with access key/secret:

docker exec alluxio-coordinator alluxio mount add \
  --path /s3 \
  --ufs-uri s3://<S3_BUCKET>/ \
  --option alluxio.underfs.s3.region=<S3_REGION> \
  --option s3a.accessKeyId=<ACCESS_KEY> \
  --option s3a.secretKey=<SECRET_KEY>

✅ Verify:

docker exec alluxio-coordinator alluxio mount list

Listing all mount points
s3://<S3_BUCKET>/  on  /s3/  properties={alluxio.underfs.s3.region=<S3_REGION>}

5. Verify Data Access

docker exec alluxio-coordinator alluxio fs ls /s3/

Returns the contents of your S3 bucket — no errors means the mount is working.

6. Start FUSE (FUSE client host)

SSH into the FUSE client host. Create the mount directory:

sudo mkdir -p /mnt/alluxio
sudo chown $(whoami) /mnt/alluxio
chmod 755 /mnt/alluxio

Start the FUSE container:

docker run -d --privileged --net=host --name=alluxio-fuse \
  -v /mnt/alluxio:/mnt/alluxio:shared \
  -e ALLUXIO_JAVA_OPTS="\
    -Xmx4g -Xms1g -XX:MaxDirectMemorySize=4g \
    -Dalluxio.etcd.endpoints=http://<COORDINATOR_PRIVATE_IP>:2379 \
    -Dalluxio.coordinator.hostname=<COORDINATOR_PRIVATE_IP> \
    -Dalluxio.mount.table.source=ETCD" \
  alluxio/alluxio-enterprise:AI-3.8-15.1.2 fuse -o allow_other /mnt/alluxio/fuse

--privileged is required for FUSE to mount inside the container and propagate to the host via -v /mnt/alluxio:/mnt/alluxio:shared.

✅ Verify (wait ~10 seconds):

ls /mnt/alluxio/fuse/

s3

Each Alluxio mount point appears as a subdirectory. /mnt/alluxio/fuse/s3/ maps directly to s3://<S3_BUCKET>/.

Test read and write:

# Read an existing file
cat /mnt/alluxio/fuse/s3/test.txt

# Write a new file
echo "hello alluxio" > /mnt/alluxio/fuse/s3/hello.txt

# Verify it appeared in S3
aws s3 ls s3://<S3_BUCKET>/hello.txt

Uninstall

Stop and remove containers on each host in reverse order:

FUSE client host:

docker stop alluxio-fuse && docker rm alluxio-fuse
sudo umount -l /mnt/alluxio/fuse   # if mount is stuck after container removal

Each worker host:

docker stop alluxio-worker && docker rm alluxio-worker

Coordinator host:

docker stop alluxio-coordinator && docker rm alluxio-coordinator
docker stop etcd-standalone && docker rm etcd-standalone

Mount table persistence: ETCD stores the mount table in its container filesystem. Running docker rm on the ETCD container will lose all mount points. To persist them across restarts, add -v /data/etcd:/etcd-data --data-dir /etcd-data to the ETCD docker run command.

Monitoring (Optional)

Set up Prometheus and Grafana on the coordinator host using Docker Compose.

mkdir -p ~/monitoring/prometheus ~/monitoring/grafana

Create ~/monitoring/prometheus/prometheus.yml:

global:
  scrape_interval: 60s

scrape_configs:
  - job_name: "coordinator"
    static_configs:
      - targets: ["<COORDINATOR_PRIVATE_IP>:19999"]
  - job_name: "workers"
    static_configs:
      - targets: ["<WORKER1_PRIVATE_IP>:30000", "<WORKER2_PRIVATE_IP>:30000"]
  - job_name: "fuse"
    static_configs:
      - targets: ["<FUSE_PRIVATE_IP>:49999"]

Create ~/monitoring/grafana/datasource.yml:

apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090
    isDefault: true
    access: proxy
    editable: true

Create ~/monitoring/compose.yaml:

services:
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
    ports:
      - 9090:9090
    restart: unless-stopped
    volumes:
      - ./prometheus:/etc/prometheus
      - prom_data:/prometheus
  grafana:
    image: grafana/grafana
    container_name: grafana
    ports:
      - 3000:3000
    restart: unless-stopped
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=grafana
    volumes:
      - ./grafana:/etc/grafana/provisioning/datasources
volumes:
  prom_data:

cd ~/monitoring && docker compose up -d

Access Grafana at http://<COORDINATOR_PUBLIC_IP>:3000 (login: admin / grafana).

Import the Alluxio dashboard JSON from the monitoring documentation.

Appendix

A. Troubleshooting

Workers not appearing in alluxio info nodes

1. Verify ETCD is reachable from the worker host:

   curl http://<COORDINATOR_PRIVATE_IP>:2379/health

   Expected: {"health":"true"}

2. Check worker logs:

   docker logs alluxio-worker 2>&1 | grep -i "error\|etcd\|register" | tail -20

3. Confirm alluxio.coordinator.hostname is set to an IP reachable from the worker. If unreachable, registration silently fails.

FUSE mount not visible after container starts

1. Check that the mount directory exists:

   ls -la /mnt/alluxio/fuse

   If missing, the container starts successfully but the mount is silently skipped.

2. Check container logs:

   docker logs alluxio-fuse 2>&1 | tail -20

   Mount point '/mnt/alluxio/fuse' does not exist confirms the directory was missing.

3. Fix the directory, then recreate the container:

   docker rm -f alluxio-fuse
   # re-run the docker run command from Step 6

Transport endpoint is not connected after FUSE container is removed

The FUSE filesystem stays registered with the kernel after the container exits. Unmount manually:

sudo umount -l /mnt/alluxio/fuse

alluxio mount add fails with unknown command

Use named flags — the old positional syntax is no longer supported:

alluxio mount add --path /s3 --ufs-uri s3://<BUCKET>/ --option alluxio.underfs.s3.region=<REGION>

B. Worker Identity After Reboot

If a worker host reboots, the container loses its identity file and registers as a new worker. The old entry stays in ETCD as OFFLINE. Remove it:

docker exec alluxio-coordinator alluxio process remove-worker -n <WORKER_ID>

Worker IDs are shown in alluxio info nodes.

C. Collecting Logs for Support

# Coordinator logs
docker cp alluxio-coordinator:/opt/alluxio/logs /tmp/coordinator-logs

# Worker logs (run on each worker host)
docker cp alluxio-worker:/opt/alluxio/logs /tmp/worker-logs

# Run collectInfo (from coordinator)
docker exec alluxio-coordinator bash -c "cd /tmp && /opt/alluxio/bin/collectinfo.sh"
docker cp alluxio-coordinator:/tmp/output.tar.gz /tmp/alluxio-collectinfo.tar.gz

Related Documentation

• Amazon S3 UFS — S3 credentials and configuration options

• POSIX API (FUSE) — FUSE mount options and tuning

• S3 API — Using the Alluxio S3-compatible endpoint

• Monitoring — Grafana dashboard import and metrics reference