Ctrlk
Home
For the complete documentation index, see llms.txt. This page is also available as Markdown.

WEB

This guide describes the instructions to configure WEB as Alluxio's under storage system.

Initial Setup

The Alluxio binaries must be available on the machine.

Configuring Alluxio

Alluxio unifies access to different storage systems through the unified namespace feature. An WEB location can be either mounted at the root of the Alluxio namespace or at a nested directory.

ROOT MOUNT

Configure Alluxio to use under storage systems by modifying conf/alluxio-site.properties. If it does not exist, create the configuration file from the template.

$ cp conf/alluxio-site.properties.template conf/alluxio-site.properties

the following environment variable assignment needs to be added to conf/alluxio-site.properties, and we actually support both http:// and https:// protocols.

alluxio.master.hostname=localhost
# alluxio.master.mount.table.root.ufs=[https|http]://<HOSTNAME>:<PORT>/DIRECTORY/
# A sample for this article
alluxio.master.mount.table.root.ufs=https://downloads.alluxio.io/downloads/files/

Specify the settings for parsing a WEB page(Optional):

alluxio.underfs.web.connnection.timeout=<WEB_CONNECTION_TIMEOUT>
alluxio.underfs.web.header.last.modified=<WEB_HEADER_LAST_MODIFIED>
alluxio.underfs.web.parent.names=<WEB_PARENT_NAMES>
alluxio.underfs.web.titles=<WEB_TITLES>

Here, alluxio.underfs.web.connnection.timeout is the timeout setting for an http connection (unit: second, default: 60s). alluxio.underfs.web.header.last.modified represents the format to parse the last modified field for a directory or a file from an http response header (default: "EEE, dd MMM yyyy HH:mm:ss zzz"). alluxio.underfs.web.parent.names indicates the start row index of the files list, which can be set as multiple flags separated by commas (default: "Parent Directory,..,../"). alluxio.underfs.web.titles is a flag that can be used to check if a web page is a directory. It can also be set with multiple values separated by commas (default: "Index of ,Directory listing for ").

NESTED MOUNT

An WEB location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio's Mount Command can be used for this purpose. For example, the following command mounts a directory inside an WEB directory into Alluxio directory /web:

Running Alluxio with WEB

Run the following command to start Alluxio filesystem.

To verify that Alluxio is running, you can visit http://localhost:19999, or see the log in the logs folder.

Visit your WEB volume by running the following command:

After waiting for a while, you can see the following result:

Stop Alluxio by running:

Last updated