S3 Compatible Storages

Alluxio can connect to various storage systems that provide an S3-compatible interface, such as MinIO, Ceph, and others. This guide explains how to configure Alluxio to mount these storage systems as an Under File System (UFS).

The primary difference when connecting to S3-compatible storage versus AWS S3 is the need to specify a custom endpoint.

Prerequisites

• Alluxio is installed and running. See Prerequisites for details.

• The S3-compatible storage service is running and accessible from the Alluxio cluster.

• You have the following information for your S3-compatible storage:

  • Endpoint URL: The address where the S3 service is listening (e.g., http://minio.example.com:9000).

  • Access Key ID and Secret Access Key: Credentials for accessing the storage.

  • Bucket Name: The name of the bucket you want to mount.

Basic Configurations

Specify storage endpoint

When using a non-AWS S3-compatible storage, you must configure Alluxio to use a custom endpoint. Otherwise, Alluxio will default to the AWS global S3 service endpoint.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>

If there is no endpoint region to set, it can be left unspecified by omitting the property. Note that when an endpoint is set, alluxio.underfs.s3.region=<S3_REGION> will no longer take effect.

Disable DNS bucket URLs for path style access

By default, the constructed request URLs will conform to virtual hosted style access. If a storage expects path style access requests, the following configuration needs to be set.

alluxio.underfs.s3.disable.dns.buckets=true

Supported storage providers

The following storage providers are known to be supported with Alluxio's S3 integration. Note that this list is not exhaustive — other S3-compatible systems may also work with appropriate configuration.

MinIO

MinIO is an object storage solution for on-premise deployments.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>
s3a.accessKeyId=<ACCESS_KEY_ID>
s3a.secretKey=<SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false

The endpoint address is typically of the form http://<minioServerHostname>:<port>, ex. http://127.0.0.1:9000. If the bucket was created without a region specified, the endpoint region can be left unspecified by omitting the property.

Oracle Cloud Infrastructure (OCI) object storage

OCI object storage is an object storage service offered by Oracle.

OCI Object Storage provides an S3-compatible API. The endpoint follows this pattern:

https://<NAMESPACE>.compat.objectstorage.<REGION>.oraclecloud.com

Where <NAMESPACE> is your OCI tenancy's object storage namespace (found in the OCI Console under Object Storage settings) and <REGION> is the OCI region identifier (e.g., us-phoenix-1, eu-frankfurt-1).

The required configuration properties are:

alluxio.underfs.s3.endpoint=https://<NAMESPACE>.compat.objectstorage.<REGION>.oraclecloud.com
alluxio.underfs.s3.endpoint.region=<REGION>
s3a.accessKeyId=<OCI_CUSTOMER_SECRET_KEY_ID>
s3a.secretKey=<OCI_CUSTOMER_SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false
alluxio.underfs.s3.sdk.version=1

Key notes for OCI:

• alluxio.underfs.s3.disable.dns.buckets=true: Required for OCI. OCI's S3-compatible API uses path-style access (endpoint/bucket), not virtual-hosted-style (bucket.endpoint). Without this, requests will fail.

• alluxio.underfs.s3.endpoint.region: Must match the OCI region identifier. When endpoint is set, alluxio.underfs.s3.region is ignored — only endpoint.region takes effect.

• alluxio.underfs.s3.sdk.version=1: Required for OCI writes. OCI's S3-compatible endpoint does not accept AWS SDK v2 chunked transfer encoding; without this property, write operations return HTTP 501 "AWS chunked encoding not supported".

• Credentials: Use OCI Customer Secret Keys (generated in OCI Console under User Settings → Customer Secret Keys). These are OCI's equivalent of AWS Access Key / Secret Key.

Kubernetes CR example:

apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: oci-bucket
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: s3://<OCI_BUCKET>/
  mountPath: /oci
  mountOptions:
    s3a.accessKeyId: "<OCI_CUSTOMER_SECRET_KEY_ID>"
    s3a.secretKey: "<OCI_CUSTOMER_SECRET_KEY>"
    alluxio.underfs.s3.endpoint: "https://<NAMESPACE>.compat.objectstorage.<REGION>.oraclecloud.com"
    alluxio.underfs.s3.endpoint.region: "<REGION>"
    alluxio.underfs.s3.disable.dns.buckets: "true"
    alluxio.underfs.s3.inherit.acl: "false"
    alluxio.underfs.s3.sdk.version: "1"

Mounting buckets from multiple OCI regions:

Each mount specifies its own endpoint and region. If all buckets share the same credentials, set them globally in the AlluxioCluster CR spec.properties and only specify per-mount endpoint/region in each UnderFileSystem CR:

# oci-mounts.yaml
apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: oci-phoenix
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: s3://oci-bucket-phoenix/
  mountPath: /oci-phoenix
  mountOptions:
    alluxio.underfs.s3.endpoint: "https://<NAMESPACE>.compat.objectstorage.us-phoenix-1.oraclecloud.com"
    alluxio.underfs.s3.endpoint.region: "us-phoenix-1"
    alluxio.underfs.s3.disable.dns.buckets: "true"
    alluxio.underfs.s3.inherit.acl: "false"
    alluxio.underfs.s3.sdk.version: "1"
---
apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: oci-frankfurt
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: s3://oci-bucket-frankfurt/
  mountPath: /oci-frankfurt
  mountOptions:
    alluxio.underfs.s3.endpoint: "https://<NAMESPACE>.compat.objectstorage.eu-frankfurt-1.oraclecloud.com"
    alluxio.underfs.s3.endpoint.region: "eu-frankfurt-1"
    alluxio.underfs.s3.disable.dns.buckets: "true"
    alluxio.underfs.s3.inherit.acl: "false"
    alluxio.underfs.s3.sdk.version: "1"

kubectl apply -f oci-mounts.yaml

Tigris data

Tigris is an object storage service offered by Tigris data.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>
s3a.accessKeyId=<ACCESS_KEY_ID>
s3a.secretKey=<SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false

As an example, a development bucket uses the endpoint https://fly.storage.tigris.dev and the endpoint region can be left unspecified by omitting the property.