S3 Compatible Storages

Various storage providers implement the S3 interface and therefore can also be mounted to Alluxio as an under storage system. Additional mount options need to be set to accommodate specific storages.

Common mount options

Specify storage endpoint

When using a non-AWS S3-compatible storage, you must configure Alluxio to use a custom endpoint. Otherwise, Alluxio will default to the AWS global S3 service endpoint.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>

If there is no endpoint region to set, it can be left unspecified by omitting the property. Note that when an endpoint is set, alluxio.underfs.s3.region=<S3_REGION> will no longer take effect.

Disable DNS bucket URLs for path style access

By default, the constructed request URLs will conform to virtual hosted style access. If a storage expects path style access requests, the following configuration needs to be set.

alluxio.underfs.s3.disable.dns.buckets=true

Supported storage providers

The following storage providers are known to be supported with Alluxio's S3 integration. Note that this list is not exhaustive — other S3-compatible systems may also work with appropriate configuration.

MinIO

MinIO is an object storage solution for on-premise deployments.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>
s3a.accessKeyId=<ACCESS_KEY_ID>
s3a.secretKey=<SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false

The endpoint address is typically of the form http://<minioServerHostname>:<port>, ex. http://127.0.0.1:9000. If the bucket was created without a region specified, the endpoint region can be left unspecified by omitting the property.

Oracle Cloud Infrastructure (OCI) object storage

OCI object storage is an object storage service offered by Oracle.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>
s3a.accessKeyId=<ACCESS_KEY_ID>
s3a.secretKey=<SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false

Tigris data

Tigris is an object storage service offered by Tigris data.

alluxio.underfs.s3.endpoint=<S3_ENDPOINT>
alluxio.underfs.s3.endpoint.region=<S3_ENDPOINT_REGION>
s3a.accessKeyId=<ACCESS_KEY_ID>
s3a.secretKey=<SECRET_KEY>
alluxio.underfs.s3.disable.dns.buckets=true
alluxio.underfs.s3.inherit.acl=false

As an example, a development bucket uses the endpoint https://fly.storage.tigris.dev and the endpoint region can be left unspecified by omitting the property.