Kubernetes is supported in the major ecosystems such as Amazon AWS EKS. To deploy Alluxio Edge for Trino with Kubernetes, the key step is to add Alluxio Edge jar to the Trino image, update a few configurations and the helm chart, then proceed with the normal deployment of Trino. This document walks through these steps with AWS S3 as UFS for example.
Package Alluxio Edge with Trino
To use Alluxio Edge with Trino, we will add several Alluxio Edge JARs to the Trino image via the Dockerfile and then rebuild the image. Afterward, we'll follow the same steps as those for deploying Trino itself.
Prerequisites
The docker command line tool
The tar command line tool
Your AWS S3 credentials
Note down the installation directory of Trino we will refer to it as ${TRINO_HOME}
Preparation of storage for Alluxio Edge: After identifying the storage attached to the Kubernetes Trino container for the Alluxio Edge cache, please note:
The size of the local storage to be provisioned for Alluxio Edge. It will be needed for variable alluxio.user.client.cache.size in the configuration step.
The path where it is mounted inside the container. It will be needed for variable alluxio.user.client.cache.dirs in the configuration step.
Request a trial version of Alluxio Edge. Contact your Alluxio account representative at sales@alluxio.com to request a trial version of Alluxio Edge. Follow their instructions to download the installation tar file into the directory you prepared.
The tar file follows the naming convention alluxio-enterprise-*.tar.gz. For example, if the tarball is named alluxio-enterprise-3.x-4.0.0-bin-97a746cf8c.tar.gz, the alluxio version is 3.x-4.0.0.
Prepare the necessary Jar
Extract Alluxio Edge jars
Two Alluxio Edge Java JAR files must be installed on each Trino node.
First, extract the Alluxio Edge client JAR file from the tar file using this command:
$ tar xf alluxio-enterprise-*.tar.gz alluxio-enterprise-*/client/alluxio-emon-*-client.jar
Then, extract the Alluxio Edge S3 under store filesystem integration JAR file using this command:
$ tar xf alluxio-enterprise-*.tar.gz alluxio-enterprise-*/lib/alluxio-underfs-emon-s3a-*.jar
Create a subdirectory called jars in the project's root directory and move all the JARs into ./jars.
In the project directory, create a file named Dockerfile with the following contents, setting the corresponding values for:
The following Dockerfile is an example:
# Identify the Trino image. It can be from the Trino official website or custom build
ARG IMAGE=trinodb/trino:430
FROM $IMAGE
# Identify the Trino installation path
ARG TRINO_HOME=/user/lib/trino
# Remove old versions of Alluxio jar files from the container
RUN find ${TRINO_HOME} -name alluxio*shaded* -exec rm {} \;
# pre-req: extract jars and put under jars/
# Copy the Alluxio Edge client jar file to the Trino catalog dirs
COPY jars/alluxio-emon-*-client.jar ${TRINO_HOME}/plugin/hive
COPY jars/alluxio-emon-*-client.jar ${TRINO_HOME}/plugin/hudi
COPY jars/alluxio-emon-*-client.jar ${TRINO_HOME}/plugin/delta-lake
COPY jars/alluxio-emon-*-client.jar ${TRINO_HOME}/plugin/iceberg
# Make a home directory for Alluxio Edge
RUN mkdir -p /home/trino/alluxio/lib && mkdir -p /home/trino/alluxio/conf
# Copy the Alluxio Edge under store jar file to the Trino lib dir
COPY jars/alluxio-underfs-emon-s3a-*.jar /home/trino/alluxio/lib
# Copy the metrics property file that enables the Alluxio JmxSink
COPY metrics.properties /home/trino/alluxio/conf/metrics.properties
# Copy the JVX Prometheus agent jar file to the Alluxio lib dir
COPY jars/jmx_prometheus_javaagent-*.jar /home/trino/alluxio/
# (Optional) Enable cache filter
COPY cache_filter.properties /home/trino/alluxio/conf/cache_filter.properties
USER trino:trino
Build the updated image
At this point, the project root directory prepared in step 1 should have the following files
Dockerfile
metrics.properties
(Optional) cache_filter.properties
jars/
Alluxio Edge client jar
Alluxio Edge S3 understorage jar
Prometheus java agent jar
Run docker build -t <repository>:<tag> . to build Alluxio Edge with Trino.
The repository naming convention is <path>/<image_name>. For example:
Please keep a note of the image info <repository> and <tag>, we will use it later.
Push the new image to a private image registry
For the cluster to fetch the locally built docker image, it will need to be pushed to a private image registry (ex. AWS ECR) where the servers have access to fetch images from.
Update the Trino Helm Chart configuration yaml file
To override the default values of Trino Kubernetes configuration, one can use a YAML configuration file to define the parameters of your deployment.
If you already have a Trino configuration YAML file, update the corresponding sections below. If you don’t already have a Trino configuration YAML file, create a new file. Let’s refer to it as values_custom.yaml.
Update image
We need to update the image path in order to use the new image we built before. Here Repository and tag corresponding to the one we built. In our example it looks like this.
is the recommended database for observing the metrics that Alluxio Edge emits. If the JMX exporter for Prometheus is not already set up in your environment, you can download the Java agent JAR from https://github.com/prometheus/jmx_exporter/releases . The file is named similarly to jmx_prometheus_javaagent-0.20.0.jar. Place it in the jars directory.
This is an advanced feature. Please see . For this example, we will put the content in cache_filter.properties.
Please refer to for full list of properties to configure for Alluxio Edge in alluxio-site.properties and explanation. See below for template.
Now you can follow instruction by Trino with the new image. All the steps are the same, except that when we do helm install we will use the Trino configuration YAML file in section 2. In the example of Trino document example helm install -f example.yaml example-trino-cluster trino/trino, we will replace example.yaml with values_custom.yaml.