REST API
Coordinator
The APIs on the coordinator are designed for jobs, configurations, and cluster management.
Distributed Load
List all load jobs
Method: GET
Path: /api/v1/load
Parameters: none
Responses:
200 - OK
: successful operationfor more information on the job description, please refer to the “Get the progress of a load job”
500 - Internal Server Error
: unexpected error
Get the progress of a load job
Method: GET
Path: /api/v1/load
Parameters:
Name
Description
id
job id returned by POST
Responses:
200 - OK
: successful operation(empty value will be omitted)404 - Not Found
: no job found500 - Internal Server Error
: unexpected error
Submit or resume a load job
Method: POST
Path: /api/v1/load
Parameters: none
Request body: Content-Type: application/json
At most one of the parameters is required.
Name
Description
paths
a list of ufs paths to load
index
the path of an index file on ufs (must be mounted)
An options
JSON can be used to configure the options of the load job. All options are optional. Possible configs are:
Name
Description
batchSize
the batch size for workers to load the data from ufs
fileFilterRegex
a regex to filter files to load
replicas
# of replicas to load, the default value is 1. If multiple replicas is enabled, can specify this argument to match the # of replicas
skipIfExists
if the file already exists, skip the loading process
sync
Whether the job needs to run synchronously and obtain the execution results
An example JSON request payload looks like the following:
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing409 - Conflict
: resume the load job with the same path and return the previous job id500 - Internal Server Error
: unexpected error
Stop a load job
Method: DELETE
Path: /api/v1/load
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing410 - Gone
: the job doesn’t exist or has already finished500 - Internal Server Error
: unexpected error
Free Cache
List all free jobs
Method: GET
Path: /api/v1/free
Parameters: none
Responses:
200 - OK
: successful operationfor more information on the job description, please refer to the “Get the progress of a free job”
500 - Internal Server Error
: unexpected error
Get the progress of a free job
Method: GET
Path: /api/v1/free
Parameters:
Name
Description
id
job id returned by POST
Responses:
200 - OK
: successful operation(empty value will be omitted)404 - Not Found
: no job found500 - Internal Server Error
: unexpected error
Submit or resume a free job
Method: POST
Path: /api/v1/free
Parameters: none
Request body: Content-Type: application/json
At most one of the parameters is required.
Name
Description
paths
a list of ufs paths to load
index
the path of an index file on ufs (must be mounted)
An example JSON request payload looks like the following:
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing409 - Conflict
: resume the free job with the same path and return the previous job id500 - Internal Server Error
: unexpected error
Stop a free job
Method: DELETE
Path: /api/v1/free
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing410 - Gone
: the job doesn’t exist or has already finished500 - Internal Server Error
: unexpected error
Rebalance
List all rebalance jobs
Method: GET
Path: /api/v1/rebalance
Parameters: none
Responses:
200 - OK
: successful operationfor more information on the job description, please refer to the “Get the progress of a rebalance job”
500 - Internal Server Error
: unexpected error
Get the progress of a rebalance job
Method: GET
Path: /api/v1/rebalance
Parameters:
At most one of the parameters is required.
Name
Description
id
job id returned by POST
target
the target workers of the rebalance job, can bea worker id (e.g. worker-54b88939-de49-46ef-acab-fe489f46d1a0)ALL (for all workers)
Responses:
200 - OK
: successful operation(empty value will be omitted)404 - Not Found
: no job found500 - Internal Server Error
: unexpected error
Submit or resume a rebalance job
Method: POST
Path: /api/v1/rebalance
Parameters: none
Request body: Content-Type: application/json
Required field
Name
Description
target
The target workers of the rebalance job, can be either a worker id (e.g. worker-54b88939-de49-46ef-acab-fe489f46d1a0
) or ALL
(for all workers)
An options
JSON can be used to configure the options of the load job. All options are optional. Possible configs are:
Name
Description
loadBatchSize
the batch size in the loading phase, the same as the batchSize option in the load job
loadBandwidth
The load bandwidth of each worker per second in bytes. e.g.: 10485746000...
pruneBandwidth
The prune bandwidth of each worker per second in bytes. e.g.: 10485746000...
skipPrune
If rebalance should only load data and skip pruning the data that does not blong to the worker(s)
An example JSON request payload looks like the following:
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing409 - Conflict
: resume the load job with the same path and return the previous job id500 - Internal Server Error
: unexpected error
Stop a rebalance job
Method: DELETE
Path: /api/v1/rebalance
Parameters: none
Request body: Content-Type: application/json
At most one of the parameters is required.
Name
Description
id
job id returned by POST
target
the target workers of the rebalance job, can be either a worker id (e.g. worker-54b88939-de49-46ef-acab-fe489f46d1a0
) or ALL
(for all workers)
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing410 - Gone
: the job doesn’t exist or has already finished500 - Internal Server Error
: unexpected error
Clear Stale Cache
The free job requires a list of files or directories as input arguments to specify what to free from the workers. The Clear Stale Cache operation does not require such input because what counts as “stale cache” is determined automatically by scanning the worker cache storage and consulting the current consistent hash ring.
The free job requires setting the correct number of replicas, otherwise it may free fewer replicas than expected, if file replication is enabled. The clear stale cache operation simply broadcasts to all the workers in the cluster, so it always triggers the clearing of stale cache on all workers, regardless of how many replicas there are.
Start clearing stale cache
Method: POST
Path: /api/v1/cache
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: an empty errors object indicates that the operation has been successfully submitted to all workers for execution200 - OK
: non-empty errors object indicates that the operation fails to be submitted to at least one of the workers for execution400 - Bad Request
: the request doesn’t have a body or the required fields are missing500 - Internal Server Error
: unexpected error501 - Not Implemented
: the specified operation type is not implemented
Stop clearing stale cache
Method: DELETE
Path: /api/v1/cache
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: an empty errors object indicates that all workers have been successfully notified to stop clearing stale cache200 - OK
: non-empty errors object indicates that the coordinator fails to notify at least one of the workers to stop clearing stale cache400 - Bad Request
: the request doesn’t have a body or the required fields are missing500 - Internal Server Error
: unexpected error501 - Not Implemented
: the specified operation type is not implemented
Mount Table
List mount points
Method: GET
Path: /api/v1/mount
Parameters: none
Responses:
200 - OK
: successful operation(options will be omitted if empty)500 - Internal Server Error
: unexpected error
Get mount point information
Method: GET
Path: /api/v1/mount
Parameters:
Name
Description
path
alluxio path of the mount point
Responses:
200 - OK
: successful operation(options will be omitted if empty)404 - Not Found
: mount point not found500 - Internal Server Error
: unexpected error
Create a mount point
Method: POST
Path: /api/v1/mount
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing409 - Conflict
: the path is already mounted, or the UFS has been mounted to another path501 - Not Implemented
: the mount table doesn’t support managing via an API (e.g.: static file mount table)500 - Internal Server Error
: unexpected error
Delete a mount point
Method: DELETE
Path: /api/v1/mount
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing410 - Gone
: the path isn’t mounted to any UFS501 - Not Implemented
: the mount table doesn’t support managing via an API (e.g.: static file mount table)500 - Internal Server Error
: unexpected error
Quota
List all quota status
Method: GET
Path: /api/v1/quota
Parameters: none
Responses:
200 - OK
: successful operation501 - Not Implemented
: quota is not enabled500 - Internal Server Error
: unexpected error
Get quota status
Method: GET
Path: /api/v1/quota
Parameters:
Name
Description
path
alluxio path to the data
Responses:
200 - OK
: successful operation404 - Not Found
: quota not found501 - Not Implemented
: quota is not enabled500 - Internal Server Error
: unexpected error
Add or update quota
Method: POST
Path: /api/v1/quota
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing400 - Bad Request
: the path doesn’t refer to a mounted UFS but it must be bound with an existing UFS501 - Not Implemented
: quota is not enabled500 - Internal Server Error
: unexpected error
Delete quota
Method: DELETE
Path: /api/v1/quota
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing400 - Bad Request
: quota not found or other problems501 - Not Implemented
: quota is not enabled500 - Internal Server Error
: unexpected error
TTL
List all TTL policies
Method: GET
Path: /api/v1/ttl
Parameters: none
Responses:
200 - OK
: successful operation501 - Not Implemented
: TTL policy is not enabled500 - Internal Server Error
: unexpected error
Get TTL policy
Method: GET
Path: /api/v1/ttl
Parameters:
Name
Description
path
alluxio path to the data
Responses:
200 - OK
: successful operation404 - Not Found
: TTL not found501 - Not Implemented
: TTL policy is not enabled500 - Internal Server Error
: unexpected error
Add or update TTL
Method: POST
Path: /api/v1/ttl
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing400 - Bad Request
: the path doesn’t refer to a mounted UFS but it must be bound with an existing UFS501 - Not Implemented
: TTL policy is not enabled500 - Internal Server Error
: unexpected error
Delete TTL
Method: DELETE
Path: /api/v1/ttl
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing400 - Bad Request
: TTL not found501 - Not Implemented
: TTL is not enabled500 - Internal Server Error
: unexpected error
Priority Eviction
List all priority eviction policies
Method: GET
Path: /api/v1/priority
Parameters: none
Responses:
200 - OK
: successful operation501 - Not Implemented
: Priority eviction is not enabled500 - Internal Server Error
: unexpected error
Get priority eviction policy
Method: GET
Path: /api/v1/priority
Parameters:
Name
Description
path
alluxio path to the data
Responses:
200 - OK
: successful operation404 - Not Found
: priority eviction not found501 - Not Implemented
: priority eviction is not enabled500 - Internal Server Error
: unexpected error
Add or update priority
Method: POST
Path: /api/v1/priority
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing400 - Bad Request
: priority string misformatted or other problems501 - Not Implemented
: priority eviction is not enabled500 - Internal Server Error
: unexpected error
Delete priority
Method: DELETE
Path: /api/v1/priority
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation400 - Bad Request
: the request doesn’t have a body or the required fields are missing501 - Not Implemented
: priority eviction is not enabled500 - Internal Server Error
: unexpected error
Node management
List all nodes
Method: GET
Path: /api/v1/nodes
Parameters: none
Responses:
200 - OK
: successful operation500 - Internal Server Error
: unexpected error
Query a node
Method: GET
Path: /api/v1/nodes
Parameters:
Name
Description
id
worker id (e.g. worker-587899dd-5da5-45d4-af40-ce481acc4087)
Responses:
200 - OK
: successful operation (empty value will be omitted)404 - Not Found
: worker not found500 - Internal Server Error
: unexpected error
Decommission a node (remove a node from etcd)
Method: DELETE
Path: /api/v1/nodes
Parameters:
Name
Description
id
worker id (e.g. worker-587899dd-5da5-45d4-af40-ce481acc4087)
Responses:
200 - OK
: successful operation404 - Not Found
: worker not found500 - Internal Server Error
: unexpected error
Global File Index
List all nodes
Method: GET
Path: /api/v1/file_index
Parameters: none
Responses:
200 - OK
: successful operation500 - Internal Server Error
: unexpected error
Add files into the global file index
Method: POST
Path: /api/v1/file_index
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation (# of entries added)404 - Not Found
: feature not enabled500 - Internal Server Error
: unexpected error
Remove files from the global file index
Method: DELETE
Path: /api/v1/file_index
Parameters: none
Request body: Content-Type: application/json
Responses:
200 - OK
: successful operation (# of entries removed)404 - Not Found
: feature not enabled500 - Internal Server Error
: unexpected error
Last updated