Multi Tenancy and Unified Management
Overview
As modern data platforms scale across multiple teams and business units, it becomes increasingly important to support multi-tenancy, ensure isolation and governance, and simplify management across federated clusters. This document outlines a reference architecture and feature set to enable secure, isolated, and manageable deployment of Alluxio clusters in a multi-tenant environment.
Key Features
1. Authentication and Authorization
Supports integration with enterprise identity providers (e.g., OKTA).
Authenticated users receive secure tokens (JWT), which are passed with each request.
Fine-grained access control is enforced by a policy engine (e.g., Open Policy Agent - OPA), which evaluates each request against customizable rules.
2. Tenant Isolation
Cache Management Isolation
Per-tenant cache configurations (TTL, quotas, eviction policies)
Independent operational controls
3. Cluster Federation
For deployments involving multiple clusters, provided to simplify operations such as:
Monitoring
License
Operation management
This ensures that scaling to tens or hundreds of clusters remains manageable.
Architecture Overview

This diagram illustrates the architecture featuring an API Gateway that centrally handles authentication and authorization across multiple Alluxio clusters. Users authenticate through the Management Console or Auth Server to obtain tokens, which the API Gateway validates before routing requests to appropriate cluster coordinators. The gateway integrates a Policy Engine (OPA) for access control decisions and supports both targeted cluster requests and multicast operations. The Management Console provides a centralized dashboard for monitoring and managing all clusters through this gateway interface.
Components
1. Management Console
Provides visual cluster management interface
Displays cross-cluster monitoring data
Authenticates users via Auth Server
Interacts with API Gateway for operations
2. API Gateway
Central entry point for all requests
Routes requests to appropriate clusters
Integrates authentication and authorization
Workflow Example: Cache TTL Update
User Authentication
User logs in through the Management Console or Auth Server (e.g. OKTA)
Receives a valid JWT token upon successful authentication
Request Submission
User sends request to API Gateway with:
The obtained JWT token
Desired TTL value
Target tenant information
Access Control Check
API Gateway validates the token
Policy Engine (OPA) verifies if user has permission to:
Modify cache settings
Access specified tenant resources
Request Execution
If authorized, API Gateway forwards request to:
Specific cluster coordinator (for single-cluster ops)
All relevant coordinators (for multi-cluster ops)
Coordinator applies changes to workers
Updated configuration is saved
Authentication and Authorization
Authentication
Authentication will initially be handled through OKTA. Upon user login and identity verification, OKTA generates a JSON Web Token (JWT) for the API gateway. The JWT is an encoded JSON object that includes user-defined claims, which can be used to associate the user with specific roles or group identifiers.
Authorization
Authorization ensures that users can only access resources they’re permitted to. The system uses Open Policy Agent (OPA) as the policy engine to enforce access control. OPA supports a declarative language called Rego for writing fine-grained, context-aware policies. Policies are configured in OPA, which exposes a decision API to evaluate whether requests should be allowed.
Multi-Tenancy
User Roles Access Control provides a way to manage user access and permissions within the Alluxio system. It allows for the creation of different user roles, each with specific access levels and permissions, ensuring that users can only access the resources they are authorized to.
Cluster Federation
Please refer to the Multi AZ High Availability section for details on how to set up and manage Alluxio clusters in a multi-AZ environment.
Licensing Management
For a large number of clusters, centralized license enforcement and propagation are supported. Please refer to License for more details.
Last updated