Alluxio
ProductsLanguageHome
AI-3.3
AI-3.3
  • Overview
  • Getting Started with K8s
    • Resource Prerequisites and Compatibility
    • Install on Kubernetes
    • Monitoring and Metrics
    • Cluster Administration
    • System Health Check & Quick Recovery
    • Collecting Cluster Information
  • Storage Integrations
    • Storage Integrations Overview
    • Amazon AWS S3
    • HDFS
    • Aliyun OSS
    • COS
    • TOS
    • GCS
  • Client APIs
    • Alluxio Python Filesystem API based on FSSpec
    • FUSE based POSIX API
    • S3 API
  • Features
    • Alluxio Namespace and Under File System Namespaces
    • Cache Preloading
    • Client Writeback
    • Cache Evicting
    • Cache Filtering
    • Cache Free
    • Directory-Based Cluster Quota
    • File Replication
    • File Segmentation
    • Index Service
    • I/O Resiliency
  • Performance Benchmarks
    • Fio Tests
    • MLPerf Storage Benchmark
    • Performance Optimization
    • COSBench performance benchmark
  • Reference
    • User CLI
    • S3 API Usage
    • Third Party Licenses
Powered by GitBook
On this page
  1. Storage Integrations

Storage Integrations Overview

Last updated 6 months ago

This guide will cover general prerequisites and running Alluxio locally with your desired under storage system. To learn how to configure Alluxio with each individual storage system, please look at their respective pages. Refer to to learn about the commands to mount the storages into Alluxio's namespace.

Prerequisites

In preparation for using your chosen storage system with Alluxio, please be sure you have all the required location, credentials, and additional properties before you begin configuring Alluxio to your under storage system.

For the purposes of this guide, the following are placeholders.

Storage System
Location
Credentials
Additional Properties

S3_BUCKET, S3_DIRECTORY

S3_ACCESS_KEY_ID, S3_SECRET_KEY

HDFS_NAMENODE, HDFS_PORT

Specify Hadoop version: HADOOP_VERSION

OSS_BUCKET, OSS_DIRECTORY

OSS_ACCESS_KEY_ID, OSS_ACCESS_KEY_SECRET, OSS_ENDPOINT

COS_BUCKET, COS_DIRECTORY

COS_ACCESS_KEY_ID, COS_ACCESS_KEY_SECRET, COS_REGION, COS_APP_ID

TOS_BUCKET, TOS_DIRECTORY

TOS_ACCESS_KEY_ID, TOS_ACCESS_KEY_SECRET, TOS_REGION, TOS_ENDPOINT

GCS_BUCKET, GCS_DIRECTORY

google_application_credentials or GCS_ACCESS_KEY_ID, GCS_SECRET_ACCESS_KEY

alluxio.underfs.gcs.version

Amazon AWS S3
HDFS
Aliyun Object Storage Service (OSS)
COS
TOS
GCS
configuring mount points