Split plane architecture

RAD's layout follows a split plane architecture where components can reside in either one of two planes:

• Control plane: Components hosted by the vendor, which serves the interactive UI for users to interface with their clusters

• Application plane: Components hosted within the user's AWS account, most notably the Alluxio cluster(s)

We considered various choices in deploying a SaaS and their potential drawbacks:

• A fully hosted service, where the cluster is running on hardware managed by the vendor

  • Data security: The user must trust the vendor to protect the data traversing through the cluster and persisting in cache

  • Performance degradation due to network separation: The cluster will reside in public cloud infrastructure supported by the vendor, which may introduce unnecessary network latency between Alluxio and the storage system as well as between Alluxio and the compute framework.

• A fully private service, where all components are running on hardware managed by the user

  • Lack of supportability: Critical cluster diagnostics would not be collected and available for Alluxio engineers to immediate diagnose and provide assistance with

  • Lack of control plane enhancements: When the control plane is hosted by the vendor, it can be seamlessly upgraded to improve its management capabilities; this is not feasible when hosted by the user.

The split plane architecture allows the user to maintain full control over the cluster and allows the vendor to provide a well curated experience through the control plane.

Access credentials

Access credentials are required to enable the control plane to deploy the application plane within the user's account. The process to create a managed access in AWS creates a policy with limited permissions. In the instructions, a JSON text is copied to create the policy; this text enumerates over the permission set.

The policy is attached to a IAM role that exclusively permits the vendor's AWS account to assume. An external ID is defined in the role to prevent accidental access by another account; read more about external ID and cross account access here.

These credentials are only necessary to create and destroy clusters. They are not used while a cluster is running and may be temporarily revoked to limit exposure.