Multi Tenancy and Unified Management

Overview

As modern data platforms scale across multiple teams and business units, it becomes increasingly important to support multi-tenancy, ensure isolation and governance, and simplify management across federated clusters. This document outlines a reference architecture and feature set to enable secure, isolated, and manageable deployment of Alluxio clusters in a multi-tenant environment.

---

Key Features

1. Authentication and Authorization

• Supports integration with enterprise identity providers (e.g., OKTA).

• Authenticated users receive secure tokens (JWT), which are passed with each request.

• Fine-grained access control is enforced by a policy engine (e.g., Open Policy Agent - OPA), which evaluates each request against customizable rules.

2. Tenant Isolation

• Cache Management Isolation

  • Per-tenant cache configurations (TTL, quotas, eviction policies)

  • Independent operational controls

3. Cluster Federation

• For deployments involving multiple clusters, provided to simplify operations such as:

  • Monitoring

  • License

  • Operation management

This ensures that scaling to tens or hundreds of clusters remains manageable.

---

Architecture Overview

This diagram illustrates the architecture featuring an API Gateway that centrally handles authentication and authorization across multiple Alluxio clusters. Users authenticate through the Management Console or Auth Server to obtain tokens, which the API Gateway validates before routing requests to appropriate cluster coordinators. The gateway integrates a Policy Engine (OPA) for access control decisions and supports both targeted cluster requests and multicast operations. The Management Console provides a centralized dashboard for monitoring and managing all clusters through this gateway interface.

Components

1. Management Console

• Provides visual cluster management interface

• Displays cross-cluster monitoring data

• Authenticates users via Auth Server

• Interacts with API Gateway for operations

2. API Gateway

• Central entry point for all requests

• Routes requests to appropriate clusters

• Integrates authentication and authorization

Workflow Example: Cache TTL Update

1. User Authentication

• User logs in through the Management Console or Auth Server (e.g. OKTA)

• Receives a valid JWT token upon successful authentication

1. Request Submission

• User sends request to API Gateway with:

  • The obtained JWT token

  • Desired TTL value

  • Target tenant information

1. Access Control Check

• API Gateway validates the token

• Policy Engine (OPA) verifies if user has permission to:

  • Modify cache settings

  • Access specified tenant resources

1. Request Execution

• If authorized, API Gateway forwards request to:

  • Specific cluster coordinator (for single-cluster ops)

  • All relevant coordinators (for multi-cluster ops)

• Coordinator applies changes to workers

• Updated configuration is saved

---

Authentication and Authorization

Authentication

Authentication will initially be handled through OKTA. Upon user login and identity verification, OKTA generates a JSON Web Token (JWT) for the API gateway. The JWT is an encoded JSON object that includes user-defined claims, which can be used to associate the user with specific roles or group identifiers.

Authorization

Authorization ensures that users can only access resources they’re permitted to. The system uses Open Policy Agent (OPA) as the policy engine to enforce access control. OPA supports a declarative language called Rego for writing fine-grained, context-aware policies. Policies are configured in OPA, which exposes a decision API to evaluate whether requests should be allowed.

Multi-Tenancy

User Roles Access Control provides a way to manage user access and permissions within the Alluxio system. It allows for the creation of different user roles, each with specific access levels and permissions, ensuring that users can only access the resources they are authorized to.

Cluster Federation

Please refer to the Multi AZ High Availability section for details on how to set up and manage Alluxio clusters in a multi-AZ environment.

Licensing Management

For a large number of clusters, centralized license enforcement and propagation are supported. Please refer to License for more details.