# User CLI

Alluxio's command line interface provides user access to various operations, such as:

* Start or stop processes
* Filesystem operations
* Administrative commands

Invoke the executable to view the possible subcommands:

```shell
$ ./bin/alluxio
Usage:
  bin/alluxio [command]

Available Commands:
  conf        Get, set, and validate configuration settings, primarily those defined in conf/alluxio-site.properties
  exec        Run the main method of an Alluxio class, or end-to-end tests on an Alluxio cluster.
  fs          Operations to interface with the Alluxio filesystem
  help        Help about any command
  index       Commands to manage the Index Service
  info        Retrieve and/or display info about the running Alluxio cluster
  job         Command line tool for interacting with the job service.
  license     Check and manage license status
  mount       Operations to manage mount points
  priority    Commands to manage the priority eviction rules
  process     Start/stop cluster processes or remove workers
  quota       Operations to manage quota definitions
  ttl         Operations to manage TTL policy definitions

Flags:
      --debug-log               True to enable debug logging
      --skip-license-warnings   Skip the automatic warnings before expiration

Use "bin/alluxio [command] --help" for more information about a command.

```

To set JVM system properties as part of the command, set the `-D` flag in the form of `-Dproperty=value`.

To attach debugging java options specified by `$ALLUXIO_USER_ATTACH_OPTS`, set the `--attach-debug` flag

Note that, as a part of Alluxio deployment, the Alluxio shell will also take the configuration in `${ALLUXIO_HOME}/conf/alluxio-site.properties` when it is run from Alluxio installation at `${ALLUXIO_HOME}`.

## conf

Get, set, and validate configuration settings, primarily those defined in conf/alluxio-site.properties

### conf get

Usage: `bin/alluxio conf get [key] [flags]`

The get command prints the configured value for the given key. If the key is invalid, it returns a nonzero exit code. If the key is valid but isn't set, an empty string is printed. If no key is specified, the full configuration is printed.

> Note: This command does not require the Alluxio cluster to be running.

Flags:

* `--master`: Show configuration properties used by the master (Default: false)
* `--source`: Show source of the configuration property instead of the value (Default: false)
* `--unit`: Unit of the value to return, converted to correspond to the given unit. E.g., with "--unit KB", a configuration value of "4096B" will return 4 Possible options include B, KB, MB, GB, TP, PB, MS, S, M, H, D (Default: "")

Examples:

```shell
# Display all the current node configuration
$ ./bin/alluxio conf get
```

```shell
# Display the value of a property key
$ ./bin/alluxio conf get alluxio.master.hostname
```

```shell
# Display the configuration of the current running Alluxio leading master
$ ./bin/alluxio conf get --master
```

```shell
# Display the source of the configuration
$ ./bin/alluxio conf get --source
```

```shell
# Display the values in a given unit
$ ./bin/alluxio conf get alluxio.user.block.size.bytes.default --unit KB
$ ./bin/alluxio conf get alluxio.master.journal.flush.timeout --unit S

```

## exec

Run the main method of an Alluxio class, or end-to-end tests on an Alluxio cluster.

### exec edgeTest

Usage: `bin/alluxio exec edgeTest [flags]`

Test whether the edge runs successfully.

Flags:

* `--no-cluster`: only interact with edge and ufs, not to access Dora cluster. (Default: false)
* `--path`: (Required) The path can be:

1. An Alluxio path like 'alluxio:///data' only when not set '--no-cluster' flag
2. A path without scheme, like '/' or '/s3'. This is a syntactic sugar for Alluxio paths.
3. A UFS path with scheme, like 's3://bucket/data' The path must exists in one of the mount points in Alluxio namespace.

### exec ufsIOTest

Usage: `bin/alluxio exec ufsIOTest [flags]`

A benchmarking tool for the I/O between Alluxio and UFS. This test will measure the I/O throughput between Alluxio workers and the specified UFS path. Each worker will create concurrent clients to first generate test files of the specified size then read those files. The write/read I/O throughput will be measured in the process.

Flags:

* `--io-size`: specifies the amount of data each thread writes/reads. (Default: "")
* `--java-opt`: The java options to add to the command line to for the task. This can be repeated. The options must be quoted and prefixed with a space. For example: --java-opt " -Xmx4g" --java-opt " -Xms2g". (Default: \[])
* `--path`: (Required) specifies the path to write/read temporary data in.
* `--threads`: specifies the number of threads to concurrently use on each worker. (Default: 4)

Examples:

```shell
# This runs the I/O benchmark to HDFS in your process locally
$ ./bin/alluxio exec ufsIOTest --path hdfs://<hdfs-address>
```

```shell
# This invokes the I/O benchmark to HDFS in the Alluxio worker
# 2 threads will be created on each worker
# Each thread is writing then reading 512m of data
$ ./bin/alluxio exec ufsIOTest --path hdfs://<hdfs-address> --io-size 512m --threads 2
```

### exec ufsTest

Usage: `bin/alluxio exec ufsTest [flags]`

Test the integration between Alluxio and the given UFS to validate UFS semantics

Flags:

* `--path`: (Required) the full UFS path to run tests against.
* `--test`: Test name, this option can be passed multiple times to indicate multipleZ tests (Default: \[])

## fs

Operations to interface with the Alluxio filesystem For commands that take Alluxio URIs as an argument such as ls or mkdir, the argument should be either

* A complete Alluxio URI, such as alluxio://\<masterHostname>:\<masterPort>/\<path>
* A path without its scheme header, such as /path, in order to use the default hostname and port set in alluxio-site.properties

> Note: All fs commands require the Alluxio cluster to be running.

Most of the commands which require path components allow wildcard arguments for ease of use. For example, the command`bin/alluxio fs rm '/data/2014*'`deletes anything in the data directory with a prefix of 2014.

Some shells will attempt to glob the input paths, causing strange errors. As a workaround, you can disable globbing (depending on the shell type; for example, set -f) or by escaping wildcards For example, the command `bin/alluxio fs cat /\\*` uses the escape backslash character twice. This is because the shell script will eventually call a java program which should have the final escaped parameters `cat /\\*`.

### fs cat

Usage: `bin/alluxio fs cat [path]`

The cat command prints the contents of a file in Alluxio to the shell.

Examples:

```shell
# Print the contents of /output/part-00000
$ ./bin/alluxio fs cat /output/part-00000
```

### fs check-cached

Usage: `bin/alluxio fs check-cached --path|--index-file <path> [--limit <limit-size>] [flags]`

Checks if files under a path have been cached in alluxio.

Flags:

* `--index-file`: The index file on local that contains a list of file paths to check. Each line should contain a ufs file path. (Default: "")
* `--limit`: Limit number of files to check (Default: 1000)
* `--path`: The path to check caching status. The path can be either an Alluxio path or a UFS path. (Default: "")
* `--recursive`: Whether to check files recursively in the given path (Default: false)

### fs checksum

Usage: `bin/alluxio fs checksum [path]`

The checksum command outputs the md5 value of a file in Alluxio. This can be used to verify the contents of a file stored in Alluxio.

Examples:

```shell
# Compare the checksum values
# value from Alluxio filesystem
$ ./bin/alluxio fs checksum /LICENSE
md5sum: bf0513403ff54711966f39b058e059a3
# value from local filesystem
md5 LICENSE
MD5 (LICENSE) = bf0513403ff54711966f39b058e059a3
```

### fs compact

Usage: `bin/alluxio fs compact [path] [flags]`

The compact command compact the position write file.

Flags:

* `--commitTimeout`: Default is 1d. Uncommitted files (files with an mtime greater than the manifest's mtime) are retained during compaction. You can configure this value to delete uncommitted files. For example, we can set it to 8h, meaning files with an mtime greater than 8 hours before the current time will be deleted, while files with an mtime less than 8 hours will be retained. (Default: "")
* `--deleteOrphanFrameFiles`: Default is false. Whether delete orphan frame files. (Default: false)
* `--deleteOrphanFrameListFiles`: Default is false. Whether delete orphan frame list files. (Default: false)
* `--deleteSnapshotsReferencedFiles`: Default is false. Whether delete the files referenced by the snapshots. If deleted, the files cannot be rolled back to a previous version. (Default: false)
* `--exportPath`: Default is null. If the path is specified, during compaction, the frames will be merged into a complete file and output to the given path. (Default: "")
* `--fileLifecycle`: Default is 0. Unreferenced files will be deleted during compaction. You can configure this value to retain some recently created files. For example, we can set it to 8h, meaning files with an mtime (modification time) greater than 8 hours before the current time will be deleted, while files with an mtime less than 8 hours will be retained. (Default: "")
* `--maxRetainedSnapshots`: Default is 4. The maximum number of snapshots to be saved. When the number of snapshots exceeds this limit, the oldest snapshot will be deleted. (Default: "")
* `--parallelism`: Default is 1. The concurrency of compaction tasks. (Default: "")
* `--recursive`: Default is false. Whether compact given path recursively. (Default: false)

Examples:

```shell
# Compact file /log.txt
$ ./bin/alluxio fs compact [--deleteOrphanFrameListFiles] [--deleteOrphanFrameFiles] [--fileLifecycle 6h] [--commitTimeout 1h] [--maxRetainedSnapshots 4] [--deleteSnapshotsReferencedFiles] [--exportPath /log.compaction.txt] [--recursive] [--parallelism 8] /log.txt
```

### fs cp

Usage: `bin/alluxio fs cp [srcPath] [dstPath] [flags]`

Copies a file or directory in the Alluxio filesystem or between local and Alluxio filesystems. The file:// scheme indicates a local filesystem path and the alluxio:// scheme or no scheme indicates an Alluxio filesystem path.

Flags:

* `--buffer-size`: Read buffer size when coping to or from local, with defaults of 64MB and 8MB respectively (Default: "")
* `--forced`,`-f`: Overwrite the destination path if it exists (Default: false)
* `--preserve`,`-p`: Preserve file permission attributes when copying files; all ownership, permissions, and ACLs will be preserved (Default: false)
* `--recursive`,`-R`: True to copy the directory subtree to the destination directory (Default: false)
* `--thread`: Number of threads used to copy files in parallel, defaults to 2 \* CPU cores (Default: 0)

Examples:

```shell
# Copy within the Alluxio filesystem
$ ./bin/alluxio fs cp /file1 /file2
```

```shell
# Copy a local file to the Alluxio filesystem
$ ./bin/alluxio fs cp file:///file1 /file2
```

```shell
# Copy a file in Alluxio to local
$ ./bin/alluxio fs cp alluxio:///file1 file:///file2
```

```shell
# Recursively copy a directory within the Alluxio filesystem
$ ./bin/alluxio fs cp -R /dir1 /dir2

```

### fs head

Usage: `bin/alluxio fs head [path] [flags]`

The head command prints the first 1KB of data of a file to the shell. Specifying the -c flag sets the number of bytes to print.

Flags:

* `--bytes`,`-c`: Byte size to print (Default: "")

Examples:

```shell
# Print first 2048 bytes of a file
$ ./bin/alluxio fs head -c 2048 /output/part-00000
```

### fs location

Usage: `bin/alluxio fs location [path]`

Displays the list of hosts storing the specified file.

### fs ls

Usage: `bin/alluxio fs ls [path] [flags]`

The ls command lists all the immediate children in a directory and displays their file info. Using ls on a file will only display the information for that specific file.

Flags:

* `--cache-filter`,`-c`: Show the cacheability rules for metadata and data of the file based on the cache filter configuration; if cache filter is not enabled, show Disabled. Note the resolved cacheability are based on cache filter observed by the command line. (Default: false)
* `--help`: help for this command (Default: false)
* `--human-readable`,`-h`: Print sizes in human readable format (Default: false)
* `--list-dir-as-file`,`-d`: List directories as files (Default: false)
* `--recursive`,`-R`: List subdirectories recursively (Default: false)

Examples:

```shell
# List immediate children of /s3/data
$ ./bin/alluxio fs ls /s3/data
```

### fs mkdir

Usage: `bin/alluxio fs mkdir [path1 path2 ...]`

The mkdir command creates a new directory in the Alluxio filesystem. It is recursive and will create any parent directories that do not exist. Note that the created directory will not be created in the under storage system until a file in the directory is persisted to the underlying storage. Using mkdir on an invalid or existing path will fail.

Examples:

```shell
# Creating a folder structure
$ ./bin/alluxio fs mkdir /users
$ ./bin/alluxio fs mkdir /users/Alice
$ ./bin/alluxio fs mkdir /users/Bob
```

### fs mv

Usage: `bin/alluxio fs mv [srcPath] [dstPath]`

The mv command moves a file or directory to another path in Alluxio. The destination path must not exist or be a directory. If it is a directory, the file or directory will be placed as a child of the directory. The command is purely a metadata operation and does not affect the data blocks of the file.

Examples:

```shell
# Moving a file
$ ./bin/alluxio fs mv /data/2014 /data/archives/2014
```

### fs rm

Usage: `bin/alluxio fs rm [path] [flags]`

The rm command removes a file from Alluxio space and the under storage system. The file will be unavailable immediately after this command returns, but the actual data may be deleted a while later.

Flags:

* `--recursive`,`-R`: True to recursively remove files within the specified directory subtree (Default: false)
* `--skip-ufs-check`,`-U`: True to skip checking if corresponding UFS contents are in sync (Default: false)

Examples:

```shell
# Remove a file from Alluxio and the under storage system
$ ./bin/alluxio fs rm /tmp/unused-file

```

### fs stat

Usage: `bin/alluxio fs stat [path] [flags]`

The stat command dumps the FileInfo representation of a file or a directory to the shell.

Flags:

* `--format`,`-f`: Display info in the given format: "%N": name of the file "%z": size of file in bytes "%u": owner "%g": group name of owner "%i": file id of the file "%y": modification time in UTC in 'yyyy-MM-dd HH:mm:ss' format "%Y": modification time as Unix timestamp in milliseconds "%b": Number of blocks allocated for file (Default: "")

Examples:

```shell
# Display file's stat
$ ./bin/alluxio fs stat /data/2015/logs-1.txt
```

```shell
# Display directory's stat
$ ./bin/alluxio fs stat /data/2015
```

```shell
# Display the size of file
$ ./bin/alluxio fs stat -f %z /data/2015/logs-1.txt

```

### fs tail

Usage: `bin/alluxio fs tail [path] [flags]`

The tail command prints the last 1KB of data of a file to the shell. Specifying the --bytes flag sets the number of bytes to print.

Flags:

* `--bytes`: Byte size to print (Default: "")

Examples:

```shell
# Print last 2048 bytes of a file
$ ./bin/alluxio fs tail --bytes 2048 /output/part-00000
```

### fs test

Usage: `bin/alluxio fs test [path] [flags]`

Test a property of a path, returning 0 if the property is true, or 1 otherwise

Flags:

* `--dir`,`-d`: Test if path is a directory (Default: false)
* `--exists`,`-e`: Test if path exists (Default: false)
* `--file`,`-f`: Test if path is a file (Default: false)
* `--not-empty`,`-s`: Test if path is not empty (Default: false)
* `--zero`,`-z`: Test if path is zero length (Default: false)

### fs touch

Usage: `bin/alluxio fs touch [path]`

Create a 0 byte file at the specified path, which will also be created in the under file system

## index

Operations to manage the Index Service of Alluxio.

### index invalidate

Usage: `bin/alluxio index invalidate [flags]`

Invalidate the cached directory listing from Alluxio Index Service. On the next time this directory is listed, Alluxio will load the latest directory listing from the UFS.

Flags:

* `--R`: Invalidate cache recursively (Default: false)
* `--path`: (Required) Path of the directory to invalidate the cache for. The path can be either an Alluxio path or a UFS path.

Examples:

```shell
# Invalidate the listing cache on /frequently_updated_directory
$ bin/alluxio index invalidate --path /frequently_updated_directory
```

```shell
# Invalidate the listing cache of all paths starting from /frequently_updated_directory recursively
# Beware this recursive operation can take a long time to complete if the directory tree is huge.
# If possible, try to invalidate your target directories directly instead of using -R.
$ bin/alluxio index invalidate -R --path /frequently_updated_directory
```

## info

Retrieve and/or display info about the running Alluxio cluster

### info cluster

Usage: `bin/alluxio info cluster`

Print the bound cluster information

### info nodes

Usage: `bin/alluxio info nodes`

Show all registered workers' status

### info production

Usage: `bin/alluxio info production`

Print the production ID

### info version

Usage: `bin/alluxio info version`

Print Alluxio version.

## job

Command line tool for interacting with the job service.

### job copy

Usage: `bin/alluxio job copy [flags]`

The copy operator copies a file or directory in the Alluxio file system distributed across workers using the scheduler. If copy is run on a directory, files in the directory will be recursively copied.

Flags:

* `--batch-size`: \[submit] # of batch size to copy at every worker (Default: 0)
* `--check-content`: \[submit] Whether to check content hash after copying files (Default: false)
* `--dst`: \[all] Destination path of copy operation (Default: "")
* `--format`: \[progress] Format of output, either TEXT or JSON (Default: "")
* `--local-index-file`: \[all] copy index file. containing on each line a pair of source and destination paths, separated by a -> (Default: "")
* `--progress`: View progress of submitted job (Default: false)
* `--src`: \[all] Source path of copy operation (Default: "")
* `--stop`: Stop running job (Default: false)
* `--submit`: Submit job (Default: false)
* `--ufs-index-file`: \[all] copy index file. containing on each line a pair of source and destination paths, separated by a -> (Default: "")
* `--verbose`: \[progress] Verbose output (Default: false)

Examples:

```shell
# Submit a copy job to copy /src to /dst
$ ./bin/alluxio job copy --src /src --dst /dst --submit
```

```shell
# View the progress of a submitted job
$ ./bin/alluxio job copy --src /src/ --dst /dst --progress
Progress for jobId 1c849041-ef26-4ed7-a932-2af5549754d7 copying path '/src' to '/dst':
        Settings: "check-content: false"
        Job Submitted: 2023-06-30 12:30:45.0
        Job Id: 111111
        Job State: RUNNING
        Files qualified so far: 1, 826.38MB
        Files Failed: 0
        Files Skipped: 0
        Files Succeeded: 1
        Bytes Copied: 826.38MB
        Throughput: 1621.09KB/s
        Files failure rate: 0.00%
```

```shell
# Stop a submitted job
$ ./bin/alluxio job copy --src /src/ --dst /dst --stop
```

### job free

Usage: `bin/alluxio job free [flags]`

The free command triggers a scheduler job to free a directory and release cached pages in all workers.

Flags:

* `--batch-size`: \[submit] The speed of free files per second per worker. If the value is 0 or empty, it will be set to the value of 'alluxio.job.batch.size' (Default: 0)
* `--force`: \[submit] Trigger free even if some workers are offline (Default: false)
* `--format`: \[progress] Format of output, either TEXT or JSON (Default: "")
* `--local-index-file`: \[All] Source local index file of free operation (Default: "")
* `--path`: \[All] Source path of free operation. The path can be either an Alluxio path or a UFS path. (Default: "")
* `--progress`: View progress of submitted job (Default: false)
* `--recursive`: \[submit] recursive free all files (Default: true)
* `--replicas`: \[submit] # of replicas to free (Default: 1)
* `--stop`: Stop running job (Default: false)
* `--submit`: Submit job (Default: false)
* `--ufs-index-file`: \[All] Source index file of free operation (Default: "")
* `--verbose`: \[progress] Verbose output (Default: false)

Examples:

```shell
# Submit a free job to free path cache
$ ./bin/alluxio job free --path s3://bucket/path --submit
```

```shell
# View the progress of a submitted job
$ ./bin/alluxio job free --path s3://alluxio/path --progress
Progress for Free path file 's3://alluxio/path':
	Job Id: b21ce9fb-f332-4d39-8bb4-554f9a4fa601
	Job Submitted: Fri Feb 02 21:28:56 CST 2024
	Job path: s3://alluxio/path
	Job State: SUCCEEDED, finished at Fri Feb 02 21:29:01 CST 2024
	Free Info :	totalFile:4 totalByte:3072.00KB
	Free Files Failed: 0
	Free Bytes Failed: 0B
	Free Files Succeeded: 4
	Free Bytes Succeeded: 3072.00KB
```

```shell
# Stop a submitted job with best effort. Workers will stop freeing cache under the specified path as soon as possible.
$ ./bin/alluxio job free --path s3://alluxio/path --stop
```

### job list

Usage: `bin/alluxio job list [flags]`

According to job type and job state list jobs.

Flags:

* `--job-state`: job state. RUNNING|VERIFYING|STOPPED|SUCCEEDED|FAILED|ALL (Default: "")
* `--job-type`: job type. LOAD|FREE|COPY|MOVE|ALL (Default: "")

Examples:

```shell
$./bin/alluxio job list --job-type all --job-state all
List job type:ALL state:ALL
{"mJobId":"1c723247-8381-4d69-bd29-b0f4fb34fc8d","mFreePath":"s3://alluxio/test-dir/","mJobType":"free","mJobState":"SUCCEEDED","mFreeBytes":81920000,"mFreeFileNums":20000,"mFreeFailFileNums":0,"mFreeFailBytes":0,"mFailInfos":{},"mTasks":{"WorkerInfo{id=0, identity=worker-1, address=WorkerNetAddress{host=hostname, containerHost=, rpcPort=29999, dataPort=29997, webPort=30000, domainSocketPath=, secureRpcPort=0, httpServerPort=0}, lastContactSec=0, state=LIVE, capacityBytes=0, usedBytes=0, startTimeMs=0, capacityBytesOnTiers=null, usedBytesOnTiers=null, version=, revision=}":"SUCCESS"},"mVerbose":true,"mStartTime":1722490595894,"mEndTime":1722490606649}
{"mPath":"s3://alluxio/test-dir/","mIndexFile":"","mJobType":"load","mVerbose":true,"mJobState":"SUCCEEDED","mVerificationEnabled":false,"mSkippedByteCount":0,"mLoadedByteCount":81920000,"mScannedInodesCount":20000,"mLoadedNonEmptyFileCopiesCount":20000,"mTotalByteCount":81920000,"mThroughput":1462857,"mFailureFilesPercentage":0.0,"mFailureSubTasksPercentage":0.0,"mRetrySubTasksPercentage":0.0,"mFailedFileCount":0,"mRecentFailedSubtasksWithReasons":[],"mRecentRetryingSubtasksWithReasons":[],"mSkipIfExists":false,"mMetadataOnly":false,"mRunningStage":"","mRetryDeadLetterQueueSize":0,"mTimeElapsed":52693,"mSegmentEnabled":false}
```

### job load

Usage: `bin/alluxio job load [flags]`

The load command moves data from the under storage system into Alluxio storage. For example, load can be used to prefetch data for analytics jobs. If load is run on a directory, files in the directory will be recursively loaded.

Flags:

* `--batch-size`: \[submit] # of batch size to load at every worker. If the value is 0 or empty, it will be set to the value of alluxio.job.batch.size (Default: 0)
* `--file-filter-regx`: \[submit] Skip files that match the regx pattern (Default: "")
* `--force`: \[submit] Trigger metadata sync even if some workers are offline (Default: false)
* `--format`: \[progress] Format of output, either TEXT or JSON (Default: "")
* `--index-file`: \[all] Source path of the index file for load operation (Default: "")
* `--local-index-file`: \[all] Source local path of the index file for load operation (Default: "")
* `--metadata-only`: \[submit] Only load file metadata (Default: false)
* `--partial-listing`: \[submit] Use partial directory listing, initializing load before reading the entire directory but cannot report on certain progress details (Default: false)
* `--path`: \[all] Source path of load operation. The path can be either an Alluxio path or a UFS path. (Default: "")
* `--progress`: View progress of submitted job (Default: false)
* `--replicas`: \[submit] # of replicas to load (Default: 1)
* `--skip-if-exists`: \[submit] Skip existing fullly cached files (Default: false)
* `--skip-quota-check`: \[submit] skip quota check and force submit load hob (Default: false)
* `--stop`: Stop running job (Default: false)
* `--submit`: Submit job (Default: false)
* `--verbose`: \[progress] Verbose output (Default: false)
* `--verify`: \[submit] Run verification when load finishes and load new files if any (Default: false)

Examples:

```shell
# Submit a load job
$ ./bin/alluxio job load --path <full-ufs-path> --submit
OR
$ ./bin/alluxio job load --index-file <full-ufs-path> --submit
OR
$ ./bin/alluxio job load --local-index-file <local-path> --submit
```

```shell
If --index-file is used, load command will read the index file and load the file paths in it.
Both file and directory are supported. In the index file, each line represents an UFS path.
```

```shell
An example index file looks like the following:
```

```shell
hdfs://cluster/root/file
s3://bucket/directory/
#file:///ignored
```

```shell
The hdfs path will be loaded as a single file,
the s3 path will be loaded recursively as a directory,the s3 path will be loaded recursively as a directory,
because it ends with the separator / ,
while the local file will be ignored as it starts with # and considered as a line of comment.
```

```shell
# View the progress of a submitted job
$ ./bin/alluxio job load --path <full-ufs-path> --progress
OR
$ ./bin/alluxio job load --index-file <full-ufs-path> --progress
OR
$ ./bin/alluxio job load --local-index-file <local-path> --progress
# Example output
Progress for loading path '/path':
        Settings:       bandwidth: unlimited    verify: false
        Job State: SUCCEEDED
        Files Processed: 1000
        Bytes Loaded: 125.00MB
        Throughput: 2509.80KB/s
        Block load failure rate: 0.00%
        Files Failed: 0
```

```shell
# Stop a submitted job
$ ./bin/alluxio job load --path <full-ufs-path> --stop
OR
$ ./bin/alluxio job load --index-file <full-ufs-path> --stop
OR
$ ./bin/alluxio job load --local-index-file <local-path> --stop

```

### job rebalance

Usage: `bin/alluxio job rebalance [flags]`

The rebalance command triggers a scheduler job to rebalance the data between workers

Flags:

* `--force`: \[submit] Trigger rebalance even if some workers are offline (Default: false)
* `--format`: \[progress] Format of output, either TEXT or JSON (Default: "")
* `--load-bandwidth`: \[submit] The load bandwidth of each worker per second in bytes, examples: 1048574/5M/10GB/... (Default: "")
* `--load-batch-size`: \[submit] The batch size for loading the file (Default: 0)
* `--progress`: View progress of submitted job (Default: false)
* `--prune-bandwidth`: \[submit] The prune bandwidth of each worker per second in bytes, examples: 1048574/5M/10GB/... (Default: "")
* `--skip-prune`: \[submit] Whether the prune process can be skipped (Default: false)
* `--stop`: Stop running job (Default: false)
* `--submit`: Submit job (Default: false)
* `--target`: \[all] The rebalance target, worker- for a specific worker and ALL for all workers (Default: "")
* `--verbose`: \[progress] Verbose output (Default: false)

Examples:

```shell
# Submit a rebalance job
 $ ./bin/alluxio job rebalance --submit --target [ALL|worker-.*]
 
 # View the progress of a submitted job
 $ ./bin/alluxio job rebalance --progress --target [ALL|worker-.*]
	Progress for the rebalance job:
	Time Elapsed: 00:00:05
	Job started at: 2024-11-21 18:48:58
	Job State: SUCCEEDED
	Total workers: 1
	Succeeded workers: 1
	Failed workers: 0
	Running workers: 0
	Files pruned: 0
	Bytes pruned: 0B
	Files loaded: 0
	Bytes loaded: 0B
	Files preserved: 7
	Bytes preserved: 49B
	Files failed: 0
 
 # Stop a submitted job with best effort. Workers will stop running rebalancing as soon as possible.
 $ ./bin/alluxio job rebalance --stop --target [ALL|worker-.*]
```

## license

Check and manage license status

### license allocate

Usage: `bin/alluxio license allocate [flags]`

Allocates the given deployment license to the provided ETCD clusters

Flags:

* `--clusters`: Base64 encoded JSON of cluster configuration (Default: "")
* `--licenseString`: License string (Default: "")

### license check-expiration

Usage: `bin/alluxio license check-expiration`

Validates the license and print warnings if any constraint is about to be exceeded

### license show

Usage: `bin/alluxio license show [flags]`

Show the details of the license based on the prod jar and configuration

Flags:

* `--output`,`-o`: Output format, could be json/yaml (Default: "")

### license status

Usage: `bin/alluxio license status [flags]`

List the current status of the cluster that license may use

Flags:

* `--raw`: Output raw JSON data instead of human-readable format for bytes, datetime, and duration. (Default: false)

### license update

Usage: `bin/alluxio license update [flags]`

Use the license from the current site properties file to update

Flags:

* `--process`: The process you want to update with the new license (Default: "")

## mount

The mount command manges the mapping from under storage path to an Alluxio path, where files and folders created in Alluxio space under the path will be backed by a corresponding file or folder in the under storage path.

### mount add

Usage: `bin/alluxio mount add [flags]`

The add command can be used to make data in another storage system available in Alluxio.

Note that the --readonly flag mounts are useful to prevent accidental write operations. If multiple Alluxio satellite clusters mount a remote storage cluster which serves as the central source of truth, the --readonly option could help prevent any write operations on the satellite cluster from wiping out the remote storage.

To connect to the UFS for a mount point, Alluxio looks for the corresponding connector under ${ALLUXIO\_HOME}/lib/ and will use the first one that supports the path. The connector jars look like lib/alluxio-underfs-hdfs-2.7.1.jar. The logic to decide whether a connector supports a path depends on the UnderFileSystemFactory implementation. When there are multiple connectors for the same UFS, like lib/alluxio-underfs-hdfs-2.7.1.jar, lib/alluxio-underfs-hdfs-2.7.1-patch1.jar, lib/alluxio-underfs-hdfs-2.7.1-patch2.jar, the option "alluxio.underfs.strict.version.match.enabled" can be used to make sure the correct one is picked up. For example, if the HDFS is running with 2.7.1-patch1, you can use "alluxio.underfs.version" and "alluxio.underfs.strict.version.match.enabled=true" to ensure "lib/alluxio-underfs-hdfs-2.7.1-patch1.jar" is used to connect to the target HDFS at hdfs\://ns1/

Flags:

* `--master`: Set true to create mount point tracked by Alluxio master (Default: false)
* `--option`: Configuration options, in the form of =, associated with the mount point, such as credentials (Default: \[])
* `--path`: (Required) Alluxio path to mount onto
* `--shared`: Sets the permission bits of the mount point to be accessible for all Alluxio users (Default: false)
* `--ufs-uri`: (Required) UFS URI to mount

Examples:

```shell
# Add a mount point at /hdfs to access hdfs://host1:9000/data/
# Note that the mount point must be under the root directory
$ ./bin/alluxio mount add --path /hdfs --ufs-uri hdfs://host1:9000/data/
```

```shell
# Add a read only mount that is accessible to all Alluxio users
$ ./bin/alluxio mount add --path /hdfs2 --ufs-uri hdfs://host2:9000/data/ --shared --readonly
```

```shell
# Add a S3 bucket mount with S3 credentials required to access the bucket
$ ./bin/alluxio mount add --path /s3 --ufs-uri s3://data-bucket/ --option s3a.accessKeyId=<accessKey> --option s3a.secretKey=<secretKey> --option alluxio.underfs.s3.region=us-east-1
```

```shell
# Add a HDFS mount using a specific UFS jar in lib/
$ ./bin/alluxio mount add --path /ns1 --ufs-uri hdfs://ns1/ --option alluxio.underfs.version=2.7.1-patch1 --option alluxio.underfs.strict.version.match.enabled=true
```

### mount list

Usage: `bin/alluxio mount list [flags]`

List all known mount points set on the Alluxio filesystem

Flags:

* `--master`: Set true to list mount points under the master based registration (Default: false)

Examples:

```shell
# List mount points
$ ./bin/alluxio mount list
```

### mount remove

Usage: `bin/alluxio mount remove [flags]`

Removes the mount point at the specified path

Flags:

* `--master`: Set true to delete mount point under the master based registration (Default: false)
* `--path`: (Required) Alluxio path to unmount

Examples:

```shell
# Remove mount point at /mnt/hdfs
$ ./bin/alluxio mount remove --path /mnt/hdfs
```

## priority

Operations to manage the priority eviction rules. You can add, remove, or update rules, as well as list the existing rules. Rules consist of a UFS path and a priority level. Files that are prefixed by the path of a rule is assigned with the priority of that rule. If there is insufficient space in the cache of a worker, files with higher priority will not be evicted until all files with lower priorities have been evicted.

### priority add

Usage: `bin/alluxio priority add [flags]`

Add a priority eviction rule with the path and the priority level

Flags:

* `--path`: (Required) The path to add a rule for. The path can be either an Alluxio path or a UFS path.
* `--priority`: (Required) The priority level of the rule. Available options: high, medium, low.

Examples:

```shell
# Add a priority eviction rule for s3://bucket/vip_data with priority HIGH
$ bin/alluxio priority add --path s3://bucket/vip_data --priority high
```

### priority list

Usage: `bin/alluxio priority list`

List the current priority eviction rules

Examples:

```shell
$ bin/alluxio priority list
```

### priority remove

Usage: `bin/alluxio priority remove [flags]`

Remove a priority rule specified by a path

Flags:

* `--path`: (Required) The path to remove a rule for. The path can be either an Alluxio path or a UFS path.

Examples:

```shell
# Remove a rule for
$ bin/alluxio priority remove --path s3://bucket/vip_data
```

### priority update

Usage: `bin/alluxio priority update [flags]`

Update the priority level of an existing rule. For example, increase the priority level of a path so that cached files of that path are more likely to get preserved in case of a cache space shortage. Note that it takes a few moments for the updated rule to take effect, as workers need to adjust their internal states about the affected files.

Flags:

* `--path`: (Required) The path of a rule to update the priority level for. The path can be either an Alluxio path or a UFS path.
* `--priority`: (Required) new priority level. Available options are: high, medium, low

Examples:

```shell
# Update the priority level for s3://bucket/vip_data to medium
$ bin/alluxio priority update --path s3://bucket/vip_data --priority medium
```

## process

Start/stop cluster processes or remove workers

### process remove-worker

Usage: `bin/alluxio process remove-worker [flags]`

Remove given worker from the cluster, so that clients and other workers will not consider the removed worker for services. The worker must have been stopped before it can be safely removed from the cluster.

Flags:

* `--name`,`-n`: (Required) Worker id

### process start

Usage: `bin/alluxio process start [flags]`

Starts a single process locally or a group of similar processes across the cluster. For starting a group, it is assumed the local host has passwordless SSH access to other nodes in the cluster. The command will parse the hostnames to run on by reading the conf/coordinator and conf/workers files, depending on the process type.

Flags:

* `--async`,`-a`: Asynchronously start processes without monitoring for start completion (Default: false)
* `--console-log`,`-c`: Log output to stdout in addition to log file (Default: false)
* `--direct`,`-d`: (For use in docker) Directly run the start command, skipping all other steps and avoid using nohup to launch (Default: false)
* `--skip-kill-prev`,`-N`: Avoid killing previous running processes when starting (Default: false)

### process stop

Usage: `bin/alluxio process stop [flags]`

Stops a single process locally or a group of similar processes across the cluster. For stopping a group, it is assumed the local host has passwordless SSH access to other nodes in the cluster. The command will parse the hostnames to run on by reading the conf/coordinator and conf/workers files, depending on the process type.

Flags:

* `--soft`,`-s`: Soft kill only, don't forcibly kill the process (Default: false)

## quota

The quota command manges all quota definitions in Alluxio system. Alluxio supports setting quota on directories in Alluxio path.

### quota add

Usage: `bin/alluxio quota add [flags]`

Use the add command to set a new quota definition on a directory. The quota definition will be applied to the total cache in all Alluxio workers.

Flags:

* `--directory`: (Required) The path to set quota on. The path can be either an Alluxio path or a UFS path.
* `--quota-size`: (Required) Size of the quota definition, eg. 10GB, 1TB

Examples:

```shell
# Add a quota of 10GB to the directory /s3/ in Alluxio namespace
# The cache in all Alluxio workers combined will be limited to 10GB
$ bin/alluxio quota add --directory /s3/ --quota-size 10GB
```

### quota list

Usage: `bin/alluxio quota list [flags]`

Use the list command to list the current usage of all quotas defined in Alluxio. You can also use the --interval option to refresh the quota usage information every specified interval.

Flags:

* `--interval`: Interval to refresh the quota usage information. (Default: "")
* `--precise-bytes`: Print sizes in bytes instead of human readable numbers. (Default: false)
* `--verbose`: Print detailed quota usage in each worker. (Default: false)

Examples:

```shell
# Lists current quota usage
$ bin/alluxio quota list
Alluxio path                                    Capacity                    Used        State
/local                                           11.00MB                 10.00MB        Available
/s3/data                                         10.00GB             Calculating        Available
```

```shell
# Keeps running until interrupted by user by pressing ctrl+C
$ bin/alluxio quota list --interval 5s
Polling quota usage summary from the master every 5s
Alluxio path                                    Capacity                    Used        State
/local                                           11.00MB                 10.00MB        Available
/s3/data                                         10.00GB             Calculating        Available
Alluxio path                                    Capacity                    Used        State
/local                                           11.00MB                 10.00MB        Available
/s3/data                                         10.00GB             Calculating        Available
Alluxio path                                    Capacity                    Used        State
/local                                           11.00MB                 10.00MB        Available
/s3/data                                         10.00GB                 10.00MB        Available
```

### quota remove

Usage: `bin/alluxio quota remove [flags]`

Use the remove command to remove an existing quota definition on a path.

Flags:

* `--directory`: (Required) The directory to remove quota on. The path can be either an Alluxio path or a UFS path.

Examples:

```shell
$ bin/alluxio quota remove --directory /s3/
Successfully removed quota definition for path /s3/data.

```

### quota update

Usage: `bin/alluxio quota update [flags]`

Use the update command to update an existing quota definition.

Flags:

* `--directory`: (Required) The path to set quota on. The path can be either an Alluxio path or a UFS path.
* `--force`: Whether to force the update of the quota, even if it violates the existing space usage (Default: false)
* `--quota-size`: (Required) Size of the quota definition, eg. 10GB, 1TB

Examples:

```shell
$ bin/alluxio quota update --directory /local/data/ --quota-size 100GB
```

## ttl

The TTL command manges all TTL policy definitions in Alluxio system. Alluxio supports setting TTl policy on directories in Alluxio path.

### ttl add

Usage: `bin/alluxio ttl add [flags]`

Use the add command to set a new TTL policy definition on a directory. Cache under this directory will be removed from Alluxio after the specified TTL.

Flags:

* `--path`: (Required) The path to set TTL policy on. The path can be either an Alluxio path or a UFS path.
* `--time`: (Required) The time to live for this path, eg. 1h, 2d

Examples:

```shell
# Add a TTL policy of 1 hour to the path /s3/a in Alluxio namespace
$ bin/alluxio ttl add --path /s3/a --time 1h
```

### ttl list

Usage: `bin/alluxio ttl list`

Use the list command to list all the new TTL policies in Alluxio.

Examples:

```shell
# List the TTL policies in Alluxio namespace
$ bin/alluxio ttl list
```

### ttl remove

Usage: `bin/alluxio ttl remove [flags]`

Use the remove command to remove a TTL policy definition on a directory.

Flags:

* `--path`: (Required) The path to delete its TTL policy. The path can be either an Alluxio path or a UFS path.

Examples:

```shell
# Remove the TTL policy for path /s3/a in Alluxio namespace
$ bin/alluxio ttl remove --path /s3/a
```

### ttl trigger

Usage: `bin/alluxio ttl trigger`

The trigger command can be used to force all Alluxio workers to evict expired cache according to the TTL rules immediately. The scan will start immediately but it may take some time to find and evict those expired cache data. The command returns when all workers are triggered, and eviction will execute asynchronously in the background.

Examples:

```shell
# Force trigger all workers to scan their cache with TTL rules
$ bin/alluxio ttl trigger
```

### ttl update

Usage: `bin/alluxio ttl update [flags]`

Use the update command to update a new TTL policy definition on a directory.

Flags:

* `--path`: (Required) The path to update TTL policy on. The path can be either an Alluxio path or a UFS path.
* `--time`: (Required) The time to live for this path, eg. 1h, 2d

Examples:

```shell
# Update the TTL policy to 1 hour for path /s3/a in Alluxio namespace
$ bin/alluxio ttl update --path /s3/a --time 1h
```
