Basic Configuration Settings

Specific configuration areas are described below; please refer to the deployment instructions for complete examples.

Alluxio Site Properties

Alluxio Edge can be configured by setting the values of configuration properties within alluxio-site.properties.

License configurations

Alluxio Edge requires a license and an ETCD cluster to connect to. These are configured by the following properties.

alluxio.license=<YOUR_LICENSE_STRING>
# list the URLs for each etcd host in the etcd cluster, ex. http://trino-edge-etcd1:2379,http://trino-edge-etcd2:2379,http://trino-edge-etcd3:2379
alluxio.etcd.endpoints=<YOUR ETCD ENDPOINTS HERE>

As needed, set the username and password to authenticate to ETCD.

alluxio.etcd.username=username
alluxio.etcd.password=password

Specify the Root Under File System

AWS S3

This sample code specifies AWS S3 as the root under file system that Alluxio Edge will be accessing.

The minimal configuration needed is to set credentials for S3 so that Alluxio Edge can access S3. Please update the values for s3a.accessKeyId and s3a.secretKey corresponding to your own S3 credentials mentioned in prerequisites.

# ex. alluxio.dora.client.ufs.root=s3://myBucket
alluxio.dora.client.ufs.root=<YOUR_S3_URL>
s3a.accessKeyId=<MY_KEY_ID>
s3a.secretKey=<MY_SECRET_KEY>

If the environment is configured with Instance Profiles the secret key and access key properties are not needed.

(Optional) To connect to S3 from a different region.

# For S3 endpoints
alluxio.underfs.s3.endpoint=s3.<PUT_YOUR_AWS_REGION_HERE>.amazonaws.com
alluxio.underfs.s3.endpoint.region=<PUT_YOUR_AWS_REGION_HERE> # Example: us-east-1
# For S3 buckets
alluxio.underfs.s3.region=<PUT_YOUR_AWS_REGION_HERE> # Example: us-east-1

(Optional) To enable TLS in Alluxio Edge if server side encryption is on.

alluxio.underfs.s3.secure.http.enabled=true
alluxio.underfs.s3.server.side.encryption.enabled=true

(Optional) If assume role is enabled in S3.

alluxio.underfs.s3.assumerole.enabled=true
alluxio.underfs.s3.assumerole.rolearn=<PUT_YOUR_ASSUME_ROLE_ARN> # Example: arn:aws:iam::...

(Optional) If S3 VPC endpoint is used.

alluxio.underfs.s3.force.global.bucket.access.enabled=true
alluxio.underfs.s3.endpoint=<YOUR_VPC_ENDPoINT> # Example: https://bucket.vpce-xxx.s3.us-east-1.vpce.amazonaws.com
alluxio.underfs.s3.endpoint.region=<YOUR_REGION> # Example: us-east-1
alluxio.underfs.s3.region=<YOUR_REGION> # us-east-1

Specify the Cache Storage Directories

Trino worker and coordinator needs to know the Alluxio Edge cache size alluxio.user.client.cache.size and path alluxio.user.client.cache.dirs. This sample code specifies the cache storage directories Alluxio Edge will use.

Enable edge cache on RAM disk, assuming it is mounted on /dev/shm:

#alluxio.user.client.cache.enabled=true
#alluxio.user.client.cache.size=1GB
#alluxio.user.client.cache.dirs=/dev/shm/alluxio_cache

Enable edge cache on 2 NVMe volumes

alluxio.user.client.cache.enabled=true
alluxio.user.client.cache.size=600GB,600GB
alluxio.user.client.cache.dirs=/mnt/nvme0/alluxio_cache,/mnt/nvme1/alluxio_cache

Enable Alluxio Edge metrics collection

alluxio.user.metrics.collection.enabled=true