Azure Blob Store
This guide describes how to configure Alluxio with Azure Blob Store as the under storage system.
Prerequisites
In preparation for using Azure Blob Store with Alluxio, create a new container in your Azure
storage account or use an existing container. You should also note that the directory you want to
use in that container, either by creating a new directory in the container, or using an existing
one. For the purposes of this guide, the Azure storage account name is called <AZURE_ACCOUNT>
, the
container in that storage account is called <AZURE_CONTAINER>
and the directory in that container is
called <AZURE_DIRECTORY>
. For more information about Azure storage account, Please seehere.
Basic Setup
Use the mount table operations to add a new mount point, specifying the Alluxio path to create the mount on and the Azure blob store path as the UFS URI. Credentials and configuration options can also be specified as part of the mount operation as described byconfiguring mount points.
An example ufs.yaml
to create a mount point with the operator:
apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
name: alluxio-wasbs
namespace: alx-ns
spec:
alluxioCluster: alluxio-cluster
path: wasbs://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/
mountPath: /wasbs
mountOptions:
fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net: <YOUR ACCESS KEY>
An example command to mount wasbs://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/
to /wasbs
if not using the operator:
bin/alluxio mount add --path /wasbs/ --ufs-uri wasbs://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/ \
--option fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net=<YOUR ACCESS KEY>
This property key fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net
will let the Alluxio use the AccountKey to access Azure Blob Store.
FAQ
What should I do if I get http not support error?
If you mount the Blob and configure the Blob path start with wasb://
, you may see the error as follows:
alluxio.exception.AlluxioException: com.microsoft.azure.storage.StorageException: The account being accessed does not support http.
You can change the Blob path start with wasbs://
.
Last updated