Alluxio
ProductsLanguageHome
AI-3.5
AI-3.5
  • Overview
  • Getting Started with K8s
    • Resource Prerequisites and Compatibility
    • Install on Kubernetes
    • Monitoring and Metrics
    • Cluster Administration
    • System Health Check & Quick Recovery
    • Collecting Cluster Information
  • Architecture
    • Alluxio Namespace and Under File System Namespaces
    • I/O Resiliency
    • Worker Management and Consistent Hashing
  • Storage Integrations
    • Amazon AWS S3
    • HDFS
    • Aliyun OSS
    • Tencent COS
    • Volcengine TOS
    • Google Cloud GCS
  • Client APIs
    • Alluxio Python Filesystem API based on FSSpec
    • FUSE based POSIX API
      • Client Writeback
    • S3 API
  • Caching Operations
    • Cache Preloading
    • Cache Filter Policy
    • Cache Eviction
      • TTL Policy
      • Priority Policy
      • Free CLI Command
  • Resource Management
    • Directory-Based Cluster Quota
    • UFS Bandwidth Limiting
  • Performance Optimizations
    • Read Throughput Via Replicas
    • Reading Large Files
    • Metadata Listing
    • Data Prefetch
    • Writing Temporary Files
  • Security
    • TLS Support
  • Performance Benchmarks
    • Fio (POSIX) Benchmark
    • COSBench (S3) Benchmark
    • MLPerf Storage Benchmark
  • Reference
    • User CLI
    • Metrics
    • S3 API Usage
    • Third Party Licenses
  • Release Notes
Powered by GitBook
On this page
  • Overview
  • UFS Read Rate Limiter
  • Configuration
  1. Resource Management

UFS Bandwidth Limiting

Overview

The rate limiter feature in Alluxio is designed to control the flow of requests and data to ensure system stability and optimize resource utilization. It helps in managing workloads by capping the amount of data processed.

UFS Read Rate Limiter

The UFS read rate limiter restricts the maximum bandwidth per second for UFS reads by a single worker. While peak bandwidth may occasionally exceed the limit, the average bandwidth will remain within the enforced boundaries. Currently the UFS which supports the read rate limiter includes HDFS, S3, OSS, COS, GCS.

Configuration

Add the following configuration in alluxio-site.properties:

# Enable ufs read rate limiter
# With this configuration, the bandwidth for reading UFS in the worker will be limited to 100MB per second
alluxio.worker.data.ufs.read.rate.limit.mb=100

Last updated 4 months ago