Alluxio
ProductsLanguageHome
  • Introduction
  • Overview
    • Architecture
    • Job Service
    • Quick Start Guide
    • FAQ
    • Use Cases
  • Core Services
    • Caching
    • Unified Namespace
  • Install Alluxio
    • Local Machine
    • Cluster
    • Cluster with HA
    • Docker
    • Software Requirements
  • Kubernetes
    • Deploy
    • Spark on Kubernetes
    • Metrics
  • Cloud Native
    • Alibaba Cloud ACK
    • AWS EMR
    • Tencent EMR
    • Google Dataproc
  • Compute Integration
    • Apache Spark
    • Apache Hadoop MapReduce
    • Apache Flink
    • Apache Hive
    • Presto on Iceberg (Experimental)
    • Presto
    • Trino
    • Tensorflow
  • Storage Integrations
    • Amazon AWS S3
    • HDFS
    • Azure Blob Store
    • Azure Data Lake Storage Gen2
    • Azure Data Lake Storage
    • Google Cloud Storage
    • Qiniu Kodo
    • COSN
    • CephObjectStorage
    • MinIO
    • NFS
    • Aliyun Object Storage Service
    • Ozone
    • Swift
    • WEB
    • CephFS
  • Security
  • Operations
    • Configuration Settings
    • User CLI
    • Admin CLI
    • Web UI
    • Journal Management
    • Metastore Management
    • Metrics
  • Administration
    • Troubleshooting
    • Basic Logging
    • Remote Logging
    • Performance Tuning
    • Scalability Tuning
    • StressBench (Experimental)
    • Upgrading
  • Solutions
  • Client APIs
    • Java API
    • S3 API
    • REST API
    • POSIX API
  • Contributor Resources
    • Building Alluxio From Source
    • Contribution Guide
    • Code Conventions
    • Documentation Conventions
    • Contributor Tools
  • Reference
    • List Of Configuration Properties
    • List of Metrics
  • REST API
    • Master REST API
    • Worker REST API
    • Proxy REST API
    • Job REST API
  • Javadoc
Powered by GitBook
On this page
  • Prerequisites
  • Basic Setup
  • o3fs
  • ofs
  • Ozone HA Mode
  • o3fs
  • ofs
  • Mount Ozone with Specific Versions
  • Example: Running Alluxio Locally with Ozone
  • Advanced Setup
  • Mount Ozone
  • Supported Ozone Versions
  • Contributed by the Alluxio Community
  1. Storage Integrations

Ozone

Last updated 7 months ago

This guide describes how to configure as Alluxio's under storage system. Ozone is a scalable, redundant, and distributed object store for Hadoop. Apart from scaling to billions of objects of varying sizes, Ozone can function effectively in containerized environments such as Kubernetes and YARN.

Prerequisites

To run an Alluxio cluster on a set of machines, you must deploy Alluxio binaries to each of these machines. You can either with the correct Hadoop version (recommended), or (for advanced users).

In preparation for using Ozone with Alluxio, follow the to install a Ozone cluster, and follow the to create volume and bucket for Ozone cluster.

Basic Setup

To configure Alluxio to use Ozone as under storage, you will need to modify the configuration file conf/alluxio-site.properties. If the file does not exist, create the configuration file from the template.

$ cp conf/alluxio-site.properties.template conf/alluxio-site.properties

Edit conf/alluxio-site.properties file to set the under storage address to the Ozone bucket and the Ozone directory you want to mount to Alluxio. Ozone supports two different schemas o3fs and ofs

o3fs

For example, the under storage address can be o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/ if you want to mount the whole bucket to Alluxio, or o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/alluxio/data if only the directory /alluxio/data inside the ozone bucket <OZONE_BUCKET> of <OZONE_VOLUME> is mapped to Alluxio.

set the property alluxio.master.mount.table.root.option.alluxio.underfs.hdfs.configuration in conf/alluxio-site.properties to point to your ozone-site.xml. Make sure this configuration is set on all servers running Alluxio.

alluxio.master.mount.table.root.ufs=o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/
alluxio.master.mount.table.root.option.alluxio.underfs.hdfs.configuration=/path/to/hdfs/conf/ozone-site.xml

ofs

For example, the under storage address can be ofs://<OZONE_MANAGER>/<OZONE_VOLUME>/<OZONE_BUCKET>/ if you want to mount the whole bucket to Alluxio, or ofs://<OZONE_MANAGER>/<OZONE_VOLUME>/<OZONE_BUCKET>/alluxio/data if only the directory /alluxio/data inside the ozone bucket <OZONE_BUCKET> of <OZONE_VOLUME> is mapped to Alluxio.

alluxio.master.mount.table.root.ufs=ofs://<OZONE_MANAGER>/<OZONE_VOLUME>/<OZONE_BUCKET>/

Ozone HA Mode

o3fs

To make Alluxio mount Ozone in HA mode, you should configure Alluxio's server so that it can find the OzoneManager. Please note that once set up, your application using the Alluxio client does not require any special configuration. In HA mode alluxio.master.mount.table.root.ufs needs to specify <OM_SERVICE_IDS> such as:

alluxio.master.mount.table.root.ufs=o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>.<OM_SERVICE_IDS>/
alluxio.master.mount.table.root.option.alluxio.underfs.hdfs.configuration=/path/to/hdfs/conf/ozone-site.xml

ofs

alluxio.master.mount.table.root.ufs=ofs://<OZONE_MANAGER>/<OZONE_VOLUME>/<OZONE_BUCKET>/
alluxio.master.mount.table.root.option.alluxio.underfs.hdfs.configuration=/path/to/hdfs/conf/ozone-site.xml

<OM_SERVICE_IDS> can be found in ozone-site.xml. In the following example ozone-site.xml file, <OM_SERVICE_IDS> is omservice1:

<property>
    <name>ozone.om.service.ids</name>
    <value>omservice1</value>
</property>

Mount Ozone with Specific Versions

Users can mount an Ozone cluster of a specific version as an under storage into Alluxio namespace. Before mounting Ozone with a specific version, make sure you have built a client with that specific version of Ozone. You can check the existence of this client by going to the lib directory under the Alluxio directory.

When mounting the under storage at the Alluxio root with a specific Ozone version, one can add the following line to the site properties file (conf/alluxio-site.properties).

alluxio.master.mount.table.root.option.alluxio.underfs.version=<OZONE_VERSION>

Example: Running Alluxio Locally with Ozone

Start the Alluxio servers:

$ ./bin/alluxio format
$ ./bin/alluxio-start.sh local

Run a simple example program:

$ ./bin/alluxio runTests

Use the HDFS shell or Ozone shell to Visit your Ozone directory o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/<OZONE_DIRECTORY> to verify the files and directories created by Alluxio exist. For this test, you should see files named like <OZONE_BUCKET>.<OZONE_VOLUME>/<OZONE_DIRECTORY>/default_tests_files/BasicFile_CACHE_PROMOTE_MUST_CACHE.

Stop Alluxio by running:

$ ./bin/alluxio-stop.sh local

Advanced Setup

Mount Ozone

$ ./bin/alluxio fs mount \
  --option alluxio.underfs.hdfs.configuration=<DIR>/ozone-site.xml \
  /ozone o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/

If you need to mount an Ozone cluster of a specific version, you can specify it through the command line option alluxio.underfs.version=<OZONE_VERSION>.

$ ./bin/alluxio fs mount \
  --option alluxio.underfs.hdfs.configuration=<DIR>/ozone-site.xml \
  --option alluxio.underfs.version=<OZONE_VERSION> \
  /ozone o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/

Possible ozone-site.xml

  • ozone-site.xml

<configuration>
  <property>
    <name>ozone.om.address</name>
    <value>localhost</value>
  </property>
</configuration>

Make sure the related config file is on all servers nodes running Alluxio.

Supported Ozone Versions

Currently, the only tested Ozone version with Alluxio is 1.0.0, 1.1.0, 1.2.1.

Contributed by the Alluxio Community

This will start an Alluxio master and an Alluxio worker. You can see the master UI at .

An Ozone location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio's can be used for this purpose. For example, the following command mounts a directory inside an Ozone bucket into Alluxio directory /ozone:

Ozone UFS integration is contributed and maintained by the Alluxio community. The source code is located . Feel free submit pull requests to improve the integration and update the documentation if any information is missing or out of date.

Ozone
download the precompiled binaries directly
compile the binaries from Alluxio source code
Ozone On Premise Installation
Cli Commands
http://localhost:19999
here
here
Mount Command