The rate limiter feature in Alluxio is designed to control the flow of requests and data to ensure system stability and optimize resource utilization. It helps in managing workloads by capping the amount of data processed.
The UFS read rate limiter restricts the maximum bandwidth per second for UFS reads by a single worker. While peak bandwidth may occasionally exceed the limit, the average bandwidth will remain within the enforced boundaries. Currently the UFS which supports the read rate limiter includes HDFS, S3, OSS, COS, GCS.
Add the following configuration in alluxio-site.properties:
# Enable ufs read rate limiter
# With this configuration, the bandwidth for reading UFS in the worker will be limited to 100MB per second
alluxio.worker.data.ufs.read.rate.limit.mb=100Last updated