Alluxio supports a RESTful API that is compatible with the basic operations of the Amazon S3 API .
The Alluxio S3 API should be used by applications designed to communicate with an S3-like storage and would benefit from the other features provided by Alluxio, such as data caching, data sharing with file system based applications, and storage system abstraction (e.g., using Ceph instead of S3 as the backing store). For example, a simple application that downloads reports generated by analytic tasks can use the S3 API instead of the more complex file system API.
Limitations and Disclaimers
Alluxio Filesystem Limitations
Only top-level Alluxio directories are treated as buckets by the S3 API.
Hence the root directory of the Alluxio filesystem is not treated as an S3 bucket. Any root-level objects (eg: alluxio://file
) will be inaccessible through the Alluxio S3 API.
To treat sub-directories as a bucket, the separator :
must be used in the bucket name (eg: s3://sub:directory:bucket/file
).
Note that this is purely a convenience feature and hence is not returned by API Actions such as ListBuckets.
Alluxio uses /
as a reserved separator. Therefore, any S3 paths with objects or folders named /
(eg: s3://example-bucket//
) will cause undefined behavior. For additional limitations on object key names please check this page: Alluxio limitations
No Bucket Virtual Hosting
Virtual hosting of buckets is not supported in the Alluxio S3 API. Therefore, S3 clients must utilize path-style requests (i.e: http://s3.amazonaws.com/{bucket}/{object}
and NOT http://{bucket}.s3.amazonaws.com/{object}
).
S3 Writes Implicitly Overwrite
As described in the AWS S3 docs for PutObject :
Amazon S3 is a distributed system. If it receives multiple write requests for the same object simultaneously, it overwrites all but the last object written. Amazon S3 does not provide object locking; if you need this, make sure to build it into your application layer or use versioning instead.
Note that at the moment the Alluxio S3 API does not support object versioning
Alluxio S3 will overwrite the existing key and the temporary directory for multipart upload.
Folders in ListObjects(V2)
All sub-directories in Alluxio will be returned in ListObjects(V2) as 0-byte folders. This behavior is in accordance with if you used the AWS S3 console to create all parent folders for each object.
User-defined tags on buckets & objects are limited to 10 and obey the S3 tag restrictions .
Set the property key alluxio.proxy.s3.tagging.restrictions.enabled=false
to disable this behavior.
The maximum size for user-defined metadata in PUT-requests is 2KB by default in accordance with S3 object metadata restrictions .
Set the property key alluxio.proxy.s3.header.metadata.max.size
to change this behavior.
The S3 API leverages the Alluxio REST proxy , introducing an additional network hop for Alluxio clients. For optimal performance, it is recommended to run the proxy server and an Alluxio worker on each compute node. It is also recommended to put all the proxy servers behind a load balancer.
Global request headers
AWS4-HMAC-SHA256 Credential={user}/..., SignedHeaders=..., Signature=...
There is currently no support for access & secret keys in the Alluxio S3 API. The only supported authentication scheme is the [SIMPLE](../security/Security.md#simple) authentication type. By default, the user that is used to perform any operations is the user that was used to launch the Alluxio proxy process.
Therefore this header is used exclusively to specify an Alluxio ACL username to perform an operation with. In order to remain compatible with other S3 clients, the header is still expected to follow the AWS Signature Version 4 format.
When supplying an access key to an S3 client, put the intended Alluxio ACL username. The secret key is unused so you may use any dummy value.
If you are using [S3 Third-Party Authentication](../security/S3-Third-Party-Authentication.md), you must include the corresponding security token in you request by adding the x-amz-security-token header.
When you obtain temporary securty credentials from [Security Token API](../security/S3-Third-Party-Authentication.md#requesting-temporary-security-credentials), temporary security credentials include a session token. You provide the session token value in the x-amz-security-token header when you send HTTPs Request to Alluxio S3 API.
Supported S3 API Actions
The following table describes the support status for current S3 API Actions :
S3 API Action
Supported Headers
Supported Query Parameters
x-amz-metadata-directive,
## Property Keys
The following table contains the configurable Alluxio property keys which pertain to the Alluxio S3 API.
Property Name
Default
Description
alluxio.proxy.s3.api.nocache.ufs.read.through.enabled
(Experimental) If enabled, reading files with a read type of NO_CACHE will be directly read from UFS.
alluxio.proxy.s3.api.noprefix.enabled
(Experimental) remove the /api/v1/s3 prefix and support /bucket/object way to access proxy.
alluxio.proxy.s3.bucket.naming.restrictions.enabled
Toggles whether or not the Alluxio S3 API will enforce AWS S3 bucket naming restrictions. See https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html.
alluxio.proxy.s3.bucketpathcache.timeout
Expire bucket path statistics in cache for this time period. Set 0min to disable the cache. If enabling the cache, be careful that Alluxio S3 API will behave differently from AWS S3 API if bucket path cache entries become stale.
alluxio.proxy.s3.complete.multipart.upload.keepalive.enabled
Whether or not to enabled sending whitespace characters as a keepalive message during CompleteMultipartUpload. Enabling this will cause any errors to be silently ignored. However, the errors will appear in the Proxy logs.
alluxio.proxy.s3.complete.multipart.upload.keepalive.time.interval
The complete multipart upload maximum keepalive time. The keepalive whitespace characters will be sent after 1 second, exponentially increasing in duration up to the configured value.
alluxio.proxy.s3.complete.multipart.upload.min.part.size
The minimum required file size of parts for multipart uploads. Parts which are smaller than this limit aside from the final part will result in an EntityTooSmall error code. Set to 0 to disable size requirements.
alluxio.proxy.s3.complete.multipart.upload.pool.size
The complete multipart upload thread pool size.
alluxio.proxy.s3.deletetype
Delete type when deleting buckets and objects through S3 API. Valid options are `ALLUXIO_AND_UFS` (delete both in Alluxio and UFS), `ALLUXIO_ONLY` (delete only the buckets or objects in Alluxio namespace).
alluxio.proxy.s3.global.read.rate.limit.mb
Limit the maximum read speed for all connections. Set value less than or equal to 0 to disable rate limits.
alluxio.proxy.s3.header.metadata.max.size
The maximum size to allow for user-defined metadata in S3 PUTrequest headers. Set to 0 to disable size limits.
alluxio.proxy.s3.multipart.upload.cleaner.enabled
Enable automatic cleanup of long-running multipart uploads.
alluxio.proxy.s3.multipart.upload.cleaner.pool.size
The abort multipart upload cleaner pool size.
alluxio.proxy.s3.multipart.upload.cleaner.retry.count
The retry count when aborting a multipart upload fails.
alluxio.proxy.s3.multipart.upload.cleaner.retry.delay
The retry delay time when aborting a multipart upload fails.
alluxio.proxy.s3.multipart.upload.cleaner.timeout
The timeout for aborting proxy s3 multipart upload automatically.
alluxio.proxy.s3.multipart.upload.stream.through
The complete multipart upload write type.
alluxio.proxy.s3.multipart.upload.write.through
The complete multipart upload write type.
alluxio.proxy.s3.single.connection.read.rate.limit.mb
Limit the maximum read speed for each connection. Set value less than or equal to 0 to disable rate limits.
alluxio.proxy.s3.tagging.restrictions.enabled
Toggles whether or not the Alluxio S3 API will enforce AWS S3 tagging restrictions (10 tags, 128 character keys, 256 character values) See https://docs.aws.amazon.com/AmazonS3/latest/userguide/tagging-managing.html.
alluxio.proxy.s3.throttle.max.wait.time.ms
The maximum waiting time when the request is throttled.
alluxio.proxy.s3.use.position.read.range.size
When the requested range length is less than this value, the S3 proxy will use 'positionRead' to read data from the worker. Setting a value less than or equal to 0 indicates disabling this feature. In the current implementation, each request for a position read uses a byte array of the same size as the range to temporarily store data, which consumes additional memory. Therefore, in practical use, we limit this value to 4MB. This means that if a value exceeding 4MB is configured, it will be modified to 4MB.
alluxio.proxy.s3.v2.async.context.timeout.ms
Timeout(in milliseconds) for async context. Set zero or less indicates no timeout.
alluxio.proxy.s3.v2.async.heavy.pool.core.thread.number
Core thread number for async heavy thread pool.
alluxio.proxy.s3.v2.async.heavy.pool.maximum.thread.number
Maximum thread number for async heavy thread pool.
alluxio.proxy.s3.v2.async.heavy.pool.queue.size
Queue size for async heavy thread pool.
alluxio.proxy.s3.v2.async.light.pool.core.thread.number
Core thread number for async light thread pool.
alluxio.proxy.s3.v2.async.light.pool.maximum.thread.number
Maximum thread number for async light thread pool.
alluxio.proxy.s3.v2.async.light.pool.queue.size
Queue size for async light thread pool.
alluxio.proxy.s3.v2.async.processing.enabled
(Experimental) If enabled, handle S3 request in async mode when v2 version of Alluxio s3 proxy service is enabled.
alluxio.proxy.s3.v2.version.enabled
(Experimental) V2, an optimized version of Alluxio s3 proxy service.
alluxio.proxy.s3.writetype
Write type when creating buckets and objects through S3 API. Valid options are `MUST_CACHE` (write will only go to Alluxio and must be stored in Alluxio), `CACHE_THROUGH` (try to cache, write to UnderFS synchronously), `THROUGH` (no cache, write to UnderFS synchronously).
## Example Usage
S3 API Actions
AWS CLIYou can use the [AWS command line interface](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html) to send S3 API requests to the Alluxio S3 API. Note that you will have to provide the `--endpoint` parameter to specify the location of the Alluxio S3 REST API with the server's base URI included (i.e: `--endpoint "http://{alluxio.proxy.web.hostname}:{alluxio.proxy.web.port}/api/v1/s3/"`).
As a pre-requisite for operations which involve the Authorization
header you may need to configure AWS credentials .
Copy $ aws configure --profile alluxio-s3
AWS Access Key ID [None]: {user}
AWS Secret Access Key [None]: {dummy value}
Default region name [None]:
Default output format [None]:
REST ClientsYou can directly use any HTTP client to send S3 API requests to the Alluxio S3 API. Note that the base URI for the Alluxio S3 API's REST server is `/api/v1/s3/` (i.e: your requests should be directed to `"http://{alluxio.proxy.web.hostname}:{alluxio.proxy.web.port}/api/v1/s3/"`).
At the moment, access key and secret key validation does not exist for the Alluxio S3 API. Therefore the Authorization header is used purely to specify the intended user to perform a request. The header follows the AWS Signature Version 4 format.
Copy $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." ...
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \ --bucket=testbucket { "Contents": [ { "Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/", "LastModified": "2022-05-03T13:00:13.429000+00:00", "Size": 0 }, { "Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1", "LastModified": "2022-05-03T13:00:13.584000+00:00", "Size": 27040 }, { "Key": "test.txt", "LastModified": "2022-05-03T11:55:01.925000+00:00", "Size": 27040 } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api abort-multipart-upload
--bucket=testbucket --key=multipart_copy.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
$ % aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2
--bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:55:01.925000+00:00", "Size": 27040 } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:17 GMT
Content-Type: application/xml
Content-Length: 583
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Marker/>
<Prefix/>
<IsTruncated>false</IsTruncated>
<Name>testbucket</Name>
<Contents>
<Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/</Key>
<Size>0</Size>
<LastModified>2022-05-03T16:44:17.490Z</LastModified>
</Contents>
<Contents>
<Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1</Key>
<Size>27040</Size>
<LastModified>2022-05-03T16:44:17.715Z</LastModified>
</Contents>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:45:30 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:36 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Marker/>
<Prefix/>
<IsTruncated>false</IsTruncated>
<Name>testbucket</Name>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api complete-multipart-upload \ --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 \ --multipart-upload="Parts=[{PartNumber=1},{PartNumber=2}]" { "Location": "/testbucket/multipart.txt", "Bucket": "testbucket", "Key": "multipart.txt", "ETag": "911df44b7ff57801ca8d74568e4ebfbe" }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-object
--bucket=testbucket --key=multipart.txt { "LastModified": "2022-05-03T20:01:43+00:00", "ContentLength": 27040, "ETag": "0cc175b9c0f1b6a831c399e269772661", "ContentType": "application/octet-stream", "Metadata": {} }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ cat complete_upload.xml
<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Part>
<PartNumber>1</PartNumber>
</Part>
<Part>
<PartNumber>2</PartNumber>
</Part>
</CompleteMultipartUpload>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "Content-Type: application/xml" -d "@complete_upload.xml" \
-X POST "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"
Date: Tue, 03 May 2022 23:59:17 GMT
Content-Type: text/xml;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.43.v20210629)
<CompleteMultipartUploadResult>
<Location>/testbucket/multipart.txt</Location>
<Bucket>testbucket</Bucket>
<Key>multipart.txt</Key>
<ETag>911df44b7ff57801ca8d74568e4ebfbe</ETag>
<Code/>
<Message/>
</CompleteMultipartUploadResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
--head "http://localhost:39999/api/v1/s3/testbucket/multipart.txt"
HTTP/1.1 200 OK
Date: Wed, 04 May 2022 00:00:40 GMT
Last-Modified: Tue, 03 May 2022 23:59:18 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api copy-object \ --copy-source=testbucket/test.txt --bucket=testbucket --key=test_copy.txt { "CopyObjectResult": { "ETag": "911df44b7ff57801ca8d74568e4ebfbe", "LastModified": "2022-05-03T11:37:16.015000+00:00" } }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects
--bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:35:59.243000+00:00", "Size": 27040 }, { "Key": "test_copy.txt", "LastModified": "2022-05-03T11:37:16.185000+00:00", "Size": 27040 } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "x-amz-copy-source: testbucket/test.txt" \
-X PUT http://localhost:39999/api/v1/s3/testbucket/test_copy.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:07 GMT
Content-Type: application/xml
Content-Length: 135
Server: Jetty(9.4.43.v20210629)
<CopyObjectResult>
<ETag>911df44b7ff57801ca8d74568e4ebfbe</ETag>
<LastModified>2022-05-03T14:50:07.781Z</LastModified>
</CopyObjectResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:26 GMT
Content-Type: application/xml
Content-Length: 434
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<Contents>
<Key>test_copy.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:50:07.790Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>testbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api create-bucket \ --bucket=testbucket
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets { "Buckets": [ { "Name": "testbucket", "CreationDate": "2022-05-03T11:32:34.156000+00:00" } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X PUT http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)
<ListAllMyBucketsResult>
<Buckets>
<Bucket>
<Name>testbucket</Name>
<CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
</Bucket>
</Buckets>
</ListAllMyBucketsResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api create-multipart-upload \ --bucket=testbucket --key=multipart.txt { "Bucket": "testbucket", "Key": "multipart.txt", "UploadId": "6367cf96-ea4e-4447-b931-c5bc91200375" } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X POST "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploads" HTTP/1.1 200 OK Date: Tue, 03 May 2022 23:41:26 GMT Content-Type: application/xml Content-Length: 147 Server: Jetty(9.4.43.v20210629) testbucket multipart.txt 6367cf96-ea4e-4447-b931-c5bc91200375 ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets { "Buckets": [ { "Name": "tempbucket", "CreationDate": "2022-05-03T11:55:58.134000+00:00" }, { "Name": "testbucket", "CreationDate": "2022-05-03T11:32:34.156000+00:00" } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-bucket
--bucket=tempbucket
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets { "Buckets": [ { "Name": "testbucket", "CreationDate": "2022-05-03T11:32:34.156000+00:00" } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:20 GMT
Content-Type: application/xml
Content-Length: 254
Server: Jetty(9.4.43.v20210629)
<ListAllMyBucketsResult>
<Buckets>
<Bucket>
<Name>tempbucket</Name>
<CreationDate>2022-05-03T14:43:03.651Z</CreationDate>
</Bucket>
<Bucket>
<Name>testbucket</Name>
<CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
</Bucket>
</Buckets>
</ListAllMyBucketsResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 21:43:25 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:28 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)
<ListAllMyBucketsResult>
<Buckets>
<Bucket>
<Name>testbucket</Name>
<CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
</Bucket>
</Buckets>
</ListAllMyBucketsResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \ --bucket=testbucket { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-bucket-tagging
--bucket=testbucket
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging
--bucket=testbucket { "TagSet": [] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:32:26 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:32:27 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)
<Tagging><TagSet/></Tagging>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \ --bucket=testbucket { "Contents": [ { "Key": "temp.txt", "LastModified": "2022-05-03T11:55:01.925000+00:00", "Size": 27040 }, { "Key": "test.txt", "LastModified": "2022-05-03T11:54:19.698000+00:00", "Size": 27040 } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-object
--bucket=testbucket --key=temp.txt
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2
--bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:55:01.925000+00:00", "Size": 27040 } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>temp.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:50:07.790Z</LastModified>
</Contents>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>testbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE http://localhost:39999/api/v1/s3/testbucket/temp.txt
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 22:01:56 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:01:59 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>testbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \ --bucket=tempbucket { "Contents": [ { "Key": "foo.txt", "LastModified": "2022-05-03T11:57:00.767000+00:00", "Size": 27040 }, { "Key": "temp.txt", "LastModified": "2022-05-03T11:56:11.245000+00:00", "Size": 27040 }, { "Key": "temp2.txt", "LastModified": "2022-05-03T11:56:31.414000+00:00", "Size": 27040 } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-objects
--bucket=tempbucket --delete="Objects=[{Key=temp.txt},{Key=temp2.txt}]"
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2
--bucket=tempbucket { "Contents": [ { "Key": "foo.txt", "LastModified": "2022-05-03T11:57:00.767000+00:00", "Size": 27040 } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>foo.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:59:05.906Z</LastModified>
</Contents>
<Contents>
<Key>temp.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:58:58.204Z</LastModified>
</Contents>
<Contents>
<Key>temp2.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:59:01.987Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>tempbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
$ cat delete.xml
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Object>
<Key>temp.txt</Key>
</Object>
<Object>
<Key>temp2.txt</Key>
</Object>
<Quiet>false</Quiet>
</Delete>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "Content-Type: application/xml" \
-X POST --data "@delete.xml" "http://localhost:39999/api/v1/s3/testbucket?delete"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:56:17 GMT
Content-Type: application/xml
Content-Length: 208
Server: Jetty(9.4.43.v20210629)
<DeleteResult>
<Deleted>
<Key>temp2.txt</Key>
<DeleteMarker/>
<DeleteMarkerVersionId/>
<VersionId/>
</Deleted>
<Deleted>
<Key>temp.txt</Key>
<DeleteMarker/>
<DeleteMarkerVersionId/>
<VersionId/>
</Deleted>
</DeleteResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:28:31 GMT
Content-Type: application/xml
Content-Length: 317
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>foo.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:59:05.906Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>tempbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \ --bucket=testbucket --key=test.txt { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api delete-object-tagging
--bucket=testbucket --key=test.txt
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging
--bucket=testbucket --key=test.txt { "TagSet": [] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:37:46 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:37:47 GMT
Content-Type: application/octet-stream
Content-Length: 28
Server: Jetty(9.4.43.v20210629)
<Tagging><TagSet/></Tagging>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \ --bucket=testbucket { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging" HTTP/1.1 200 OK Date: Tue, 03 May 2022 23:31:07 GMT Content-Type: application/xml Content-Length: 124 Server: Jetty(9.4.43.v20210629) key1 val1 key2 val2 ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object \ --bucket=testbucket --key=test.txt /tmp/test.txt { "LastModified": "2022-05-03T18:55:01+00:00", "ContentLength": 27040, "ETag": "0cc175b9c0f1b6a831c399e269772661", "ContentType": "application/octet-stream", "Metadata": {} }
$ stat /tmp/test.txt File: /tmp/test.txt Size: 27040 Blocks: 56 IO Block: 4096 regular file ...
Copy
</details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/testbucket/test.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:59:43 GMT
Last-Modified: Tue, 03 May 2022 21:47:36 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)
................. file contents .................
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \ --bucket=testbucket --key=test.txt { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET "http://localhost:39999/api/v1/s3/testbucket?tagging" HTTP/1.1 200 OK Date: Tue, 03 May 2022 23:31:07 GMT Content-Type: application/xml Content-Length: 124 Server: Jetty(9.4.43.v20210629) key1 val1 key2 val2 ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-bucket \ --bucket=testbucket ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ --head http://localhost:39999/api/v1/s3/testbucket HTTP/1.1 200 OK Date: Tue, 15 Nov 2022 04:49:12 GMT Content-Type: application/xml Content-Length: 0 Server: Jetty(9.4.43.v20210629) ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api head-object \ --bucket=testbucket --key=test.txt { "LastModified": "2022-05-03T18:55:01+00:00", "ContentLength": 27040, "ETag": "0cc175b9c0f1b6a831c399e269772661", "ContentType": "application/octet-stream", "Metadata": {} } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ --head http://localhost:39999/api/v1/s3/testbucket/test.txt HTTP/1.1 200 OK Date: Tue, 03 May 2022 21:54:22 GMT Last-Modified: Tue, 03 May 2022 21:47:36 GMT ETag: 0cc175b9c0f1b6a831c399e269772661 Content-Type: application/octet-stream Content-Length: 27040 Server: Jetty(9.4.43.v20210629) ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-buckets { "Buckets": [ { "Name": "testbucket", "CreationDate": "2022-05-03T11:32:34.156000+00:00" } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET http://localhost:39999/api/v1/s3/ HTTP/1.1 200 OK Date: Tue, 03 May 2022 21:35:23 GMT Content-Type: application/xml Content-Length: 161 Server: Jetty(9.4.43.v20210629) testbucket 2022-05-03T14:34:56.420Z ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects \ --bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:35:59.243000+00:00", "Size": 27040 }, { "Key": "test_copy.txt", "LastModified": "2022-05-03T11:37:16.185000+00:00", "Size": 27040 } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET http://localhost:39999/api/v1/s3/testbucket HTTP/1.1 200 OK Date: Tue, 03 May 2022 21:50:26 GMT Content-Type: application/xml Content-Length: 434 Server: Jetty(9.4.43.v20210629) false test.txt 27040 2022-05-03T14:47:36.600Z test_copy.txt 27040 2022-05-03T14:50:07.790Z false testbucket 1000 url ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint="http://localhost:39999/api/v1/s3" s3api list-multipart-uploads --bucket "testbucket" { "Uploads": [ { "UploadId": "c4cddf71-914a-4cee-b2af-8cfb7def7d04", "Key": "object", "Initiated": "2022-07-01T11:21:14.738000+00:00" }, { "UploadId": "6367cf96-ea4e-4447-b931-c5bc91200375", "Key": "object", "Initiated": "2022-07-01T11:18:25.290000+00:00" }, { "UploadId": "e111c33b-5c18-4ecd-b543-2849cdbbf22b", "Key": "object2", "Initiated": "2022-07-01T11:21:25.182000+00:00" } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET "http://localhost:39999/api/v1/s3/testbucket?uploads" HTTP/1.1 200 OK Date: Fri, 01 Jul 2022 18:23:43 GMT Content-Type: application/xml Content-Length: 499 Server: Jetty(9.4.46.v20220331) testbucket object c4cddf71-914a-4cee-b2af-8cfb7def7d04 2022-07-01T11:21:14.738Z object 6367cf96-ea4e-4447-b931-c5bc91200375 2022-07-01T11:18:25.290Z object2 e111c33b-5c18-4ecd-b543-2849cdbbf22b 2022-07-01T11:21:25.182Z ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects-v2 \ --bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:35:59.243000+00:00", "Size": 27040 }, { "Key": "test_copy.txt", "LastModified": "2022-05-03T11:37:16.185000+00:00", "Size": 27040 } ] } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET "http://localhost:39999/api/v1/s3/testbucket?list-type=2" HTTP/1.1 200 OK Date: Tue, 03 May 2022 21:56:20 GMT Content-Type: application/xml Content-Length: 438 Server: Jetty(9.4.43.v20210629) true test.txt 27040 2022-05-03T14:47:36.600Z test_copy.txt 27040 2022-05-03T14:50:07.790Z false 2 testbucket 1000 url ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \ --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 { "Parts": [ { "PartNumber": 1, "LastModified": "2022-05-03T12:56:27.775000+00:00", "ETag": "", "Size": 27040 } ], "ChecksumAlgorithm": null, "Initiator": null, "Owner": null, "StorageClass": "STANDARD" } ```
REST Clients```console $ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \ -X GET "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375" HTTP/1.1 200 OK Date: Tue, 03 May 2022 23:49:05 GMT Content-Type: application/octet-stream Content-Length: 314 Server: Jetty(9.4.43.v20210629) /testbucket multipart.txt 6367cf96-ea4e-4447-b931-c5bc91200375 STANDARD false 1 2022-05-03T16:48:56.602Z 27040 ```
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging \ --bucket=testbucket { "TagSet": [] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-bucket-tagging
--bucket=testbucket --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-bucket-tagging
--bucket=testbucket { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)
<Tagging><TagSet/></Tagging>
$ cat tags.xml
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "Content-Type: application/xml" \
-X PUT "http://localhost:39999/api/v1/s3/testbucket?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-object \ --bucket=testbucket --key=test.txt --body="${ALLUXIO_HOME}/LICENSE" { "ETag": "911df44b7ff57801ca8d74568e4ebfbe" }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-objects
--bucket=testbucket { "Contents": [ { "Key": "test.txt", "LastModified": "2022-05-03T11:35:59.243000+00:00", "Size": 27040 } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X PUT http://localhost:39999/api/v1/s3/testbucket/test.txt -T "${ALLUXIO_HOME}/LICENSE"
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:36 GMT
ETag: 911df44b7ff57801ca8d74568e4ebfbe
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET http://localhost:39999/api/v1/s3/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:44 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<Marker/>
<IsTruncated>false</IsTruncated>
<Prefix/>
<Name>testbucket</Name>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging \ --bucket=testbucket --key=test.txt { "TagSet": [] }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api put-object-tagging
--bucket=testbucket --key=test.txt --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api get-object-tagging
--bucket=testbucket --key=test.txt { "TagSet": [ { "Key": "key1", "Value": "val1" }, { "Key": "key2", "Value": "val2" } ] }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)
<Tagging><TagSet/></Tagging>
$ cat tags.xml
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "Content-Type: application/xml" \
-X PUT "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:28 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:39999/api/v1/s3/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:58 GMT
Content-Type: application/octet-stream
Content-Length: 126
Server: Jetty(9.4.43.v20210629)
<Tagging>
<TagSet>
<Tag>
<Key>key1</Key>
<Value>val1</Value>
</Tag>
<Tag>
<Key>key2</Key>
<Value>val2</Value>
</Tag>
</TagSet>
</Tagging>
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api upload-part \ --bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 --part-number=1 --body="${ALLUXIO_HOME}/LICENSE" { "ETag": "911df44b7ff57801ca8d74568e4ebfbe" }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts
--bucket=testbucket --key=multipart.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 { "Parts": [ { "PartNumber": 1, "LastModified": "2022-05-03T12:56:27.775000+00:00", "ETag": "", "Size": 27040 } ], "ChecksumAlgorithm": null, "Initiator": null, "Owner": null, "StorageClass": "STANDARD" }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X PUT -T "${ALLUXIO_HOME}/LICENSE" "http://localhost:39999/api/v1/s3/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375&partNumber=1"
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:51:19 GMT
ETag: "911df44b7ff57801ca8d74568e4ebfbe"
Content-Length: 0
Server: Jetty(9.4.43.v20210629)
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts \
--bucket=testbucket --key=multipart_copy.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
{
"Parts": [
{
"PartNumber": 1,
"LastModified": "2022-05-03T13:00:13.584000+00:00",
"ETag": "",
"Size": 27040
}
],
"ChecksumAlgorithm": null,
"Initiator": null,
"Owner": null,
"StorageClass": "STANDARD"
}
AWS CLI```console $ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api upload-part-copy \ --bucket=testbucket --key=object --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 --part-number=1 --copy-source=testbucket/object { "CopyPartResult": { "ETag": "0cc175b9c0f1b6a831c399e269772661" } }
$ aws --profile alluxio-s3 --endpoint "http://localhost:39999/api/v1/s3/" s3api list-parts
--bucket=testbucket --key=object --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375 { "Parts": [ { "PartNumber": 1, "LastModified": "2022-07-01T11:27:48.942000+00:00", "ETag": "", "Size": 1 } ], "ChecksumAlgorithm": null, "Initiator": null, "Owner": null, "StorageClass": "STANDARD" }
Copy </details>
<details><summary>REST Clients</summary>
```console
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-H "x-amz-copy-source: testbucket/object" \
-X PUT 'http://localhost:39999/api/v1/s3/testbucket/object?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375&partNumber=1'
HTTP/1.1 200 OK
Date: Fri, 01 Jul 2022 18:31:34 GMT
Content-Type: application/xml
Content-Length: 78
Server: Jetty(9.4.46.v20220331)
<CopyPartResult><ETag>0cc175b9c0f1b6a831c399e269772661</ETag></CopyPartResult>