COSBench Benchmarks

COSBench is Intel's distributed object-storage benchmark — a Controller + Drivers cluster that executes staged XML workloads. Use it for complex, multi-stage read/write workloads, distributed across multiple driver nodes, with mixed operation types.

For when to pick COSBench over Warp or httpbench, see Choosing a Benchmark Tool. For reference throughput from a 4-driver COSBench cluster, see Reference Performance Baselines.

Prerequisites

• Operating System: CentOS 7 (kernel 3.10) or later. COSBench has known compatibility issues on Ubuntu and is not recommended.

• Pre-loaded data: For hot-read benchmarks (cache hit), ensure your test dataset is fully loaded into the Alluxio cache. Use bin/alluxio job load --path /path --submit to load data and bin/alluxio fs check-cached /path to verify.

Installation

On all COSBench controller and driver nodes:

1. Download COSBench version 0.4.2.c4 and unzip it.

2. Install dependencies:

   sudo yum install nmap-ncat curl java-1.8.0-openjdk-devel -y

3. Disable MD5 validation — edit cosbench-start.sh and add the following Java property. This is necessary for compatibility with Alluxio's S3 API:

   -Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true

4. Start the controller and all drivers from the COSBench root directory:

   sudo bash start-all.sh

Usage

Running any scenario below follows the same three steps. Pick an XML workload (see subsections below or write your own), save it under conf/, then:

# Submit
bash cli.sh submit conf/<your-file>.xml

# Monitor — open the web dashboard:
# http://<CONTROLLER_IP>:19088/controller/index.html

# Stop all services when done
sudo bash stop-all.sh

A COSBench workload is made up of these stages:

Stage	Purpose
init	Creates the test buckets
prepare	Writes the initial data for testing
main	Runs the read/write operations for a set duration
cleanup	Deletes the objects created during the test
dispose	Deletes the buckets

If you are using Alluxio mount points, you cannot create new buckets via the S3 API. Skip the init and dispose stages and use pre-existing buckets that match your mount configuration.

COSBench config syntax:

• r(1,10) — range: sequentially iterates over items 1 through 10. Used in init, prepare, cleanup, and dispose stages.

• u(1,10) — uniform random: randomly selects an item between 1 and 10. Used in main stage for realistic access patterns.

• c(64)KB — constant: fixed size of 64KB per object.

Bucket naming: COSBench appends a numeric suffix to cprefix. For example, cprefix=my-bucket;containers=r(1,1) creates a bucket named my-bucket1, not my-bucket. When using pre-existing buckets (e.g., Alluxio mount points), set cprefix so the generated names match exactly — for a bucket named my-bucket1, use cprefix=my-bucket with containers=r(1,1).

Scenario: Basic Read/Write Workload

Goal: Verify basic S3 read/write functionality with a small dataset.

Creates two buckets, writes 10 objects of 64 KB to each, runs a 30-second test with an 80/20 read-write ratio, and then cleans up.

The accesskey and secretkey below are placeholder values. Replace them with your Alluxio S3 API credentials.

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
  <storage type="s3" config="accesskey=root;secretkey=dump;endpoint=http://localhost:29998;path_style_access=true" />

  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="30">
        <operation type="read" ratio="80" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />
        <operation type="write" ratio="20" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>
  </workflow>
</workload>

Scenario: High-Concurrency Read Test

Goal: Measure maximum read IOPS/throughput under heavy concurrency. Key difference from the basic example: uses 4 distributed drivers with 128 threads each.

Prepares 10,000 small objects (100 KB) and uses four drivers, each with 128 worker threads, to read concurrently for 300 seconds.

Each <work> block is repeated per driver because COSBench requires explicit driver assignment for distributed workloads. Each block sends traffic from a separate driver node to maximize aggregate concurrency.

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
  <storage type="s3" config="accesskey=root;secretkey=dump;endpoint=http://<ip>:29998;path_style_access=true" />
  <workflow>
    <workstage name="prepare">
      <work type="prepare" workers="100" config="cprefix=ufs;containers=r(2,2);oprefix=myobjects;osuffix=.jpg;objects=r(1,10000);sizes=c(100)KB" />
    </workstage>

    <workstage name="128">
      <work name="read" workers="128" driver="driver1" runtime="300">
        <operation type="read" ratio="100" config="cprefix=ufs;containers=r(2,2);oprefix=myobjects;osuffix=.jpg;objects=u(1,10000)" />
      </work>
      <work name="read" workers="128" driver="driver2" runtime="300">
        <operation type="read" ratio="100" config="cprefix=ufs;containers=r(2,2);oprefix=myobjects;osuffix=.jpg;objects=u(1,10000)" />
      </work>
      <work name="read" workers="128" driver="driver3" runtime="300">
        <operation type="read" ratio="100" config="cprefix=ufs;containers=r(2,2);oprefix=myobjects;osuffix=.jpg;objects=u(1,10000)" />
      </work>
      <work name="read" workers="128" driver="driver4" runtime="300">
        <operation type="read" ratio="100" config="cprefix=ufs;containers=r(2,2);oprefix=myobjects;osuffix=.jpg;objects=u(1,10000)" />
      </work>
    </workstage>
  </workflow>
</workload>

Troubleshooting

• init stage fails against Alluxio — Alluxio mount points do not support bucket creation via S3 API. Skip init/dispose stages and use pre-existing buckets.

For cross-tool troubleshooting (redirect-mode behavior, kernel tuning, cache-hit verification), see Performance Tuning and Troubleshooting on the hub page.

See Also

• S3 API Benchmarks — overview, reference baselines, tool selection, cross-tool troubleshooting

• Warp Benchmarks — quick single-binary alternative for bucket-wide reads

• httpbench Benchmarks — per-worker, redirect-aware tool

• S3 API Setup and Configuration — deployment patterns, endpoint setup, load balancer configuration