Deploy Alluxio Edge for Trino
Last updated
Last updated
This document describes how to deploy Alluxio Edge for Trino to work with AWS S3 as UFS. The key step is to place Alluxio client jar files to the Trino path, update a few configurations, then proceed with the normal deployment of Trino.
Your AWS S3 credentials
Preparation of storage for Alluxio Edge: After identifying the storage mounted for Alluxio Edge cache, please note:
The size of the local storage to be provisioned for Alluxio Edge alluxio.user.client.cache.size
The path where it is mounted alluxio.user.client.cache.dirs
Assuming you already have Trino in your environment, and set up similarly as . Note down the installation directory of Trino. We will refer to it as ${TRINO_HOME}
throughout this document which you will need to update.
Request a trial version of Alluxio Edge. Contact your Alluxio account representative at sales@alluxio.com
to request a trial version of Alluxio Edge. Follow their instructions to download the installation tar file into the directory you prepared.
The tar file follows the naming convention alluxio-enterprise-*.tar.gz
. For example, if the tarball is named alluxio-enterprise-3.x-4.0.0-bin-97a746cf8c.tar.gz
, the alluxio version is 3.x-4.0.0
.
Running a command like this will usually work:
Two Alluxio Edge Java JAR files must be installed on each Trino node.
First, extract the Alluxio Edge client JAR file from the tar file using this command:
Then, extract the Alluxio Edge S3 under store filesystem integration JAR file using this command:
You can configure Alluxio configuration properties via the Trino jvm.config
file, which is usually inside ${TRINO_HOME}/etc/
, with the following format
Defines the Alluxio Edge conf directory.
For example, to set ${TRINO_HOME}/etc/alluxio/
as the configuration directory for Alluxio Edge under, you would do the following
We will use
Configure Alluxio Edge metrics for Prometheus integration.
Create jmx_export_config.yaml
in configuration directory ${TRINO_HOME}/etc/alluxio/
with the following sample content.
In the jvm.config
file, add the following:
Create the alluxio-site.properties
file and place it in the config directory set in previous step. In our example, it is ${TRINO_HOME}/etc/alluxio/
.
Add the following to ${TRINO_HOME}/etc/alluxio/alluxio-core-site.xml
to include the fs.s3a.impl property to ensure that Trino uses the S3AFileSystem when a Hive table LOCATION is set to the s3a://
scheme.
Configure the Trino catalog (such as HIVE and Delta Lake catalog) to include alluxio-core-site.xml
file to the resources. You can likely find the catalog files in ${TRINO_HOME}/etc/catalog/
. Note that the hive.config.resources
property may already be set with existing xml files, ie. /etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml
In this case, alluxio-core-site.xml
should be appended to the end of the list.
Some optional configurations
Now you can relaunch Trino with Alluxio Edge.
is the recommended database for observing the metrics that Alluxio Edge emits. If the JMX exporter for Prometheus is not already set up in your environment, you can download the Java agent JAR from https://github.com/prometheus/jmx_exporter/releases . The file is named similarly to jmx_prometheus_javaagent-0.20.0.jar
. Place it in the ${TRINO_HOME}/lib/
directory.
Please refer to for details.