# Volcengine TOS

This guide describes how to configure [Tinder Object Storage Service (TOS)](https://www.volcengine.com/product/TOS) as Alluxio's under storage system. Tinder Object Storage Service (TOS) is a massive, secure, low-cost, easy-to-use, highly reliable, and highly available distributed cloud storage service provided by VolcEngine.

## Prerequisites

Before using TOS with Alluxio, follow the [TOS quick start guide](https://www.volcengine.com/docs/6349/74830) to sign up for TOS and create a TOS bucket.

Before you get started, please ensure you have the required information listed below:

| `<TOS_BUCKET>`            | Create a new TOS bucket or use an existing bucket                                                                                                                                                                                                                                                                                         |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `<TOS_DIRECTORY>`         | The directory you want to use in the bucket, either by creating a new directory or using an existing one                                                                                                                                                                                                                                  |
| `<TOS_ACCESS_KEY_ID>`     | <p>The <a href="https://www.volcengine.com/docs/6291/65568">Access Key ID</a> for TOS,<br>which are created and managed in the <a href="https://console.volcengine.com/iam/keymanage/">TOS AccessKey management console</a></p>                                                                                                           |
| `<TOS_ACCESS_KEY_SECRET>` | <p>The <a href="https://www.volcengine.com/docs/6291/65568">Secret Access Key</a> for TOS,<br>which are created and managed in the <a href="https://console.volcengine.com/iam/keymanage/">TOS AccessKey management console</a></p>                                                                                                       |
| `<TOS_ENDPOINT>`          | <p>The internet endpoint of the bucket, which can be found in the bucket overview page with values like <code>tos-cn-beijing.volces.com</code> and <code>tos-cn-guangzhou.volces.com</code>.<br>Available endpoints are listed in the <a href="https://www.volcengine.com/docs/6349/107356">TOS Internet Endpoints documentation</a>.</p> |
| `<TOS_REGION>`            | <p>The region where the bucket is located, such as <code>cn-beijing</code> or <code>cn-guangzhou</code>.<br>Available regions are listed in the <a href="https://www.volcengine.com/docs/6349/107356">TOS Regions documentation</a>.</p>                                                                                                  |

## Basic Setup

For the general mount mechanism and `UnderFileSystem` CR field reference, see [Underlying Storage](/ee-ai-en/ai-3.8-15.1.x/ufs.md#basic-mount-setup).

{% tabs %}
{% tab title="Kubernetes (Operator)" %}
An example `ufs.yaml` to create a TOS mount point with the operator:

```yaml
apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: alluxio-tos
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: tos://<TOS_BUCKET>/<TOS_DIRECTORY>
  mountPath: /tos
  mountOptions:
    fs.tos.accessKeyId: <TOS_ACCESS_KEY>
    fs.tos.accessKeySecret: <TOS_ACCESS_KEY_SECRET>
    fs.tos.endpoint: <TOS_ENDPOINT>
    fs.tos.region: <TOS_REGION>
```

{% endtab %}

{% tab title="Docker / Bare-Metal" %}
An example command to mount `tos://<TOS_BUCKET>/<TOS_DIRECTORY>` to `/tos` if not using the operator:

```shell
bin/alluxio mount add --path /tos/ --ufs-uri tos://<TOS_BUCKET>/<TOS_DIRECTORY> \
  --option fs.tos.accessKeyId=<TOS_ACCESS_KEY> --option fs.tos.accessKeySecret=<TOS_ACCESS_KEY_SECRET> \
  --option fs.tos.endpoint=<TOS_ENDPOINT> --option fs.tos.region=<TOS_REGION>
```

Note that if you want to mount the root of the TOS bucket, add a trailing slash after the bucket name (e.g. `tos://TOS_BUCKET/`).
{% endtab %}
{% endtabs %}

## Advanced Setup

Note that configuration options can be specified as mount options or as configuration properties in `conf/alluxio-site.properties`.\
The following sections will describe how to set configurations as properties, but they can also be set as mount options via `--option <key>=<value>`.

### Enabling HTTPS

To enable the use of the HTTPS protocol for secure communication with TOS with an additional layer of security for data transfers, configure the following setting in conf/alluxio-site.properties:

```properties
fs.tos.endpoint=https://<TOS_ENDPOINT>
alluxio.underfs.tos.secure.http.enabled=true
```

### TOS multipart upload

We use multipart-upload method to upload one file by multiple parts, every part will be uploaded in one thread. It won't generate any temporary files while uploading.

There are other parameters you can specify in `conf/alluxio-site.properties` to potentially speed up the upload.

```properties
# Timeout for uploading part when using multipart upload.
alluxio.underfs.object.store.multipart.upload.timeout
```

```properties
# Thread pool size for TOS multipart upload.
alluxio.underfs.tos.multipart.upload.threads
```

```properties
# Multipart upload partition size for TOS. The default partition size is 64MB.
alluxio.underfs.tos.multipart.upload.part.size
```

Disable TOS multipart upload, the upload method uploads one file completely from start to end in one go. you need to modify `conf/alluxio-site.properties` to include:

```properties
alluxio.underfs.tos.multipart.upload.enabled=false
```

### Setting Request Retry Policy

For retry configuration that applies to all object store UFS types, see [Request Retry Policy](/ee-ai-en/ai-3.8-15.1.x/ufs.md#request-retry-policy) in the UFS overview.

### High Concurrency Tuning

When integrating Alluxio with TOS, you can optimize performance by adjusting the following configurations:

* `alluxio.underfs.tos.retry.max`: Controls the number of retries with TOS. Default value is 3.
* `alluxio.underfs.tos.read.timeout`: Controls read timeout with TOS. Default value is 30000 milliseconds.
* `alluxio.underfs.tos.write.timeout`: Controls write timeout with TOS. Default value is 30000 milliseconds.
* `alluxio.underfs.tos.streaming.upload.partition.size`: Controls the partition size for TOS streaming upload. Default value is 64MB.
* `alluxio.underfs.tos.connect.timeout`: Controls the connection timeout with TOS. Default value is 30000 milliseconds.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://documentation.alluxio.io/ee-ai-en/ai-3.8-15.1.x/ufs/tos.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.