Home

S3 API

S3 API Reference

This document provides a detailed reference for the Alluxio S3 API, which allows you to interact with Alluxio using the popular Amazon S3 protocol. The examples provided use both the AWS CLI and standard REST clients like curl.

Bucket Operations

CreateBucket

Creates a new bucket in Alluxio.

Method: PUT

Path: /{bucket}

Responses:

  • 200 - OK: The bucket was created successfully.

Examples:

AWS CLI
$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api create-bucket \
  --bucket=testbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}
REST Client
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/..." \
  -X PUT http://localhost:29998/testbucket

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/..." \
  -X GET http://localhost:29998/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

DeleteBucket

Deletes an existing bucket. The bucket must be empty before it can be deleted.

Method: DELETE

Path: /{bucket}

Responses:

  • 204 - No Content: The bucket was deleted successfully.

Examples:

AWS CLI
REST Client

ListBuckets

Lists all buckets owned by the sender of the request.

Method: GET

Path: /

Responses:

  • 200 - OK: Returns an XML document containing a list of buckets.

Examples:

AWS CLI
REST Client

HeadBucket

This operation is useful to determine if a bucket exists and you have permission to access it.

Method: HEAD

Path: /{bucket}

Responses:

  • 200 - OK: The bucket exists and you have permission to access it.

  • 404 - Not Found: The bucket does not exist.

Examples:

AWS CLI
REST Client

GetBucketTagging

Retrieves the tags associated with a bucket.

Method: GET

Path: /{bucket}?tagging

Responses:

  • 200 - OK: Returns an XML document with the tag set.

Examples:

AWS CLI
REST Client

PutBucketTagging

Adds a set of tags to an existing bucket.

Method: PUT

Path: /{bucket}?tagging

Request Body: An XML document containing the tag set.

Responses:

  • 200 - OK: The tags were added successfully.

Examples:

AWS CLI
REST Client

DeleteBucketTagging

Deletes the tags from a bucket.

Method: DELETE

Path: /{bucket}?tagging

Responses:

  • 204 - No Content: The tags were deleted successfully.

Examples:

AWS CLI
REST Client

Object Operations

GetObject

Retrieves an object from a bucket.

Method: GET

Path: /{bucket}/{key}

Responses:

  • 200 - OK: The object data is returned in the response body.

Examples:

AWS CLI
REST Client

PutObject

Adds an object to a bucket.

Method: PUT

Path: /{bucket}/{key}

Request Body: The object content.

Responses:

  • 200 - OK: The object was uploaded successfully.

Examples:

AWS CLI
REST Client

CopyObject

Creates a copy of an object that is already stored in Alluxio.

Method: PUT

Path: /{bucket}/{key}

Request Headers:

  • x-amz-copy-source: The name of the source bucket and key, separated by a slash (/).

Responses:

  • 200 - OK: Returns an XML document with the result of the copy operation.

Examples:

AWS CLI
REST Client

DeleteObject

Removes an object from a bucket.

Method: DELETE

Path: /{bucket}/{key}

Responses:

  • 204 - No Content: The object was deleted successfully.

Examples:

AWS CLI
REST Client

DeleteObjects

Removes multiple objects from a bucket in a single request.

Method: POST

Path: /{bucket}?delete

Request Body: An XML document specifying the keys of the objects to delete.

Responses:

  • 200 - OK: Returns an XML document with the results of the delete operation for each object.

Examples:

AWS CLI
REST Client

HeadObject

Retrieves metadata from an object without returning the object itself.

Method: HEAD

Path: /{bucket}/{key}

Responses:

  • 200 - OK: The metadata is returned in the response headers.

Examples:

AWS CLI
REST Client

ListObjects

Returns some or all (up to 1,000) of the objects in a bucket.

Method: GET

Path: /{bucket}

Responses:

  • 200 - OK: Returns an XML document with the list of objects.

Examples:

AWS CLI
REST Client

ListObjectsV2

Returns some or all (up to 1,000) of the objects in a bucket. This is the newer version of ListObjects.

Method: GET

Path: /{bucket}?list-type=2

Responses:

  • 200 - OK: Returns an XML document with the list of objects.

Examples:

AWS CLI
REST Client

GetObjectTagging

Retrieves the tags associated with an object.

Method: GET

Path: /{bucket}/{key}?tagging

Responses:

  • 200 - OK: Returns an XML document with the tag set.

Examples:

AWS CLI
REST Client

PutObjectTagging

Adds a set of tags to an existing object.

Method: PUT

Path: /{bucket}/{key}?tagging

Request Body: An XML document containing the tag set.

Responses:

  • 200 - OK: The tags were added successfully.

Examples:

AWS CLI
REST Client

DeleteObjectTagging

Deletes the tags from an object.

Method: DELETE

Path: /{bucket}/{key}?tagging

Responses:

  • 204 - No Content: The tags were deleted successfully.

Examples:

AWS CLI
REST Client

Multipart Upload Operations

CreateMultipartUpload

Initiates a multipart upload and returns an upload ID.

Method: POST

Path: /{bucket}/{key}?uploads

Responses:

  • 200 - OK: Returns an XML document with the UploadId.

Examples:

AWS CLI
REST Client

UploadPart

Uploads a part in a multipart upload.

Method: PUT

Path: /{bucket}/{key}

Query Parameters:

  • partNumber: Part number of the part being uploaded. This is a positive integer between 1 and 10,000.

  • uploadId: Upload ID of the multipart upload.

Request Body: The content of the part.

Responses:

  • 200 - OK: The part was uploaded successfully.

Examples:

AWS CLI
REST Client

UploadPartCopy

Uploads a part by copying data from an existing object as a data source.

Method: PUT

Path: /{bucket}/{key}

Query Parameters:

  • partNumber: Part number of the part being uploaded.

  • uploadId: Upload ID of the multipart upload.

Request Headers:

  • x-amz-copy-source: The name of the source bucket and key.

Responses:

  • 200 - OK: The part was copied successfully.

Examples:

AWS CLI
REST Client

ListParts

Lists the parts that have been uploaded for a specific multipart upload.

Method: GET

Path: /{bucket}/{key}

Query Parameters:

  • uploadId: Upload ID of the multipart upload.

Responses:

  • 200 - OK: Returns an XML document with the list of parts.

Examples:

AWS CLI
REST Client

ListMultipartUploads

This operation lists in-progress multipart uploads.

Method: GET

Path: /{bucket}?uploads

Responses:

  • 200 - OK: Returns an XML document with a list of in-progress multipart uploads.

Examples:

AWS CLI
REST Client

AbortMultipartUpload

Aborts a multipart upload. After a multipart upload is aborted, no more parts can be uploaded using that upload ID.

Method: DELETE

Path: /{bucket}/{key}

Query Parameters:

  • uploadId: The ID of the multipart upload to abort.

Responses:

  • 204 - No Content: The multipart upload was aborted successfully.

Examples:

AWS CLI
REST Client

CompleteMultipartUpload

Completes a multipart upload by assembling previously uploaded parts.

Method: POST

Path: /{bucket}/{key}

Query Parameters:

  • uploadId: The ID of the multipart upload to complete.

Request Body: An XML document listing the parts to be assembled.

Responses:

  • 200 - OK: Returns an XML document with information about the completed object.

Examples:

AWS CLI
REST Client

Last updated