Azure Blob Storage

This guide describes how to configure Alluxio with Azure Blob Store as the under storage system.

Prerequisites

In preparation for using Azure Blob Store with Alluxio, create a new container in your Azure storage account or use an existing container. You should also note that the directory you want to use in that container, either by creating a new directory in the container, or using an existing one. For the purposes of this guide, the Azure storage account name is called <AZURE_ACCOUNT>, the container in that storage account is called <AZURE_CONTAINER> and the directory in that container is called <AZURE_DIRECTORY>. For more information about Azure storage account, Please seehere.

Basic Setup

For the general mount mechanism and UnderFileSystem CR field reference, see Underlying Storage.

Kubernetes (Operator)

An example ufs.yaml to create an Azure Blob Storage mount point with the operator:

apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: alluxio-wasbs
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: wasbs://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/
  mountPath: /wasbs
  mountOptions:
    fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net: <YOUR ACCESS KEY>

Docker / Bare-Metal

FAQ

What should I do if I get http not support error?

If you mount the Blob and configure the Blob path start with wasb://, you may see the error as follows:

alluxio.exception.AlluxioException: com.microsoft.azure.storage.StorageException: The account being accessed does not support http.

You can change the Blob path start with wasbs://.