S3 API Usage
Example Usage
S3 API Actions
AWS CLI
You can use the AWS command line interface to send S3 API requests to the Alluxio S3 API. Note that you will have to provide the --endpoint parameter to specify the location of the Alluxio S3 REST API with the server's base URI included. The endpoint format is --endpoint "http://{alluxio.worker.web.hostname}:{alluxio.worker.rest.port}/".
As a prerequisite for operations which involve the Authorization header you may need to configure AWS credentials.
See the Authorization header for details on how Alluxio uses this header
$ aws configure --profile alluxio-s3
AWS Access Key ID [None]: {user}
AWS Secret Access Key [None]: {dummy value}
Default region name [None]:
Default output format [None]:REST Clients
You can directly use any HTTP client to send S3 API requests to the Alluxio S3 API. The base URI is /; requests should be directed to http://{alluxio.worker.web.hostname}:{alluxio.worker.rest.port}/.
At the moment, access key and secret key validation does not exist for the Alluxio S3 API. Therefore the Authorization header is used purely to specify the intended user to perform a request. The header follows the AWS Signature Version 4 format.
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." ...AWS CLI
$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
--bucket=testbucket
{
"Contents": [
{
"Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/",
"LastModified": "2022-05-03T13:00:13.429000+00:00",
"Size": 0
},
{
"Key": "multipart_copy.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1",
"LastModified": "2022-05-03T13:00:13.584000+00:00",
"Size": 27040
},
{
"Key": "test.txt",
"LastModified": "2022-05-03T11:55:01.925000+00:00",
"Size": 27040
}
]
}
$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api abort-multipart-upload \
--bucket=testbucket --key=multipart_copy.txt --upload-id=6367cf96-ea4e-4447-b931-c5bc91200375
$ % aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
--bucket=testbucket
{
"Contents": [
{
"Key": "test.txt",
"LastModified": "2022-05-03T11:55:01.925000+00:00",
"Size": 27040
}
]
}REST Clients
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:29998/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:17 GMT
Content-Type: application/xml
Content-Length: 583
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Marker/>
<Prefix/>
<IsTruncated>false</IsTruncated>
<Name>testbucket</Name>
<Contents>
<Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/</Key>
<Size>0</Size>
<LastModified>2022-05-03T16:44:17.490Z</LastModified>
</Contents>
<Contents>
<Key>multipart.txt_6367cf96-ea4e-4447-b931-c5bc91200375/1</Key>
<Size>27040</Size>
<LastModified>2022-05-03T16:44:17.715Z</LastModified>
</Contents>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X DELETE "http://localhost:29998/testbucket/multipart.txt?uploadId=6367cf96-ea4e-4447-b931-c5bc91200375"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:45:30 GMT
Server: Jetty(9.4.43.v20210629)
$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
-X GET "http://localhost:29998/testbucket"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:45:36 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)
<ListBucketResult>
<version2>false</version2>
<Marker/>
<Prefix/>
<IsTruncated>false</IsTruncated>
<Name>testbucket</Name>
<Contents>
<Key>test.txt</Key>
<Size>27040</Size>
<LastModified>2022-05-03T14:47:36.600Z</LastModified>
</Contents>
<MaxKeys>1000</MaxKeys>
<EncodingType>url</EncodingType>
</ListBucketResult>Last updated