Network Attached Storage (NAS)

This guide describes the steps to configure NAS as the underlying storage system for Alluxio. NAS (Network Attached Storage) is a device or architecture that provides centralized storage services over a network (such as Ethernet), supports multiple protocols (such as NFS, SMB, AFP, etc.), and allows clients to access shared storage resources. It is often used to access remote files across a network environment and is often used for shared storage in Linux systems. For more information about NAS and NFS, refer to the Ubuntu NFS documentation.

Alluxio supports two ways to use NAS as Alluxio's UFS: 1. Mount NAS to the local file system, and Alluxio will perform operations on the files in the NAS in a similar way to the file system; 2. Use the NFS client to complete all file operations on the NAS.

NAS via Host Path

When deploying the Alluxio cluster with the operator, the hostPaths field could be set to mount the NAS path on node to a path on the worker container:

apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
spec:
  hostPaths:
    coordinator:
      /mnt/nas: /ufs/data
    worker:
      /mnt/nas: /ufs/data
    fuse:
      /mnt/nas: /ufs/data

where /mnt/nas is where NAS is mounted on the node.

An example ufs.yaml to create a mount point with the operator:

apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: alluxio-nas
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: file:///ufs/data
  mountPath: /nas

An example command to mount /mnt/nas to /nas if not using the operator:

bin/alluxio mount add --path /nas/ --ufs-uri file:///mnt/nas

NAS via NFS

Prerequisites

Before configuring NAS with Alluxio, ensure you have the following requirements:

<NAS_SERVER_IP>	IP address of the NAS server hosting the shared directory.
<SHARED_DIR>	Absolute path of the exported directory on the NAS server (e.g., /nas_share).
<CLIENT_RANGE>	Client IP range allowed to access the share (e.g., 192.168.1.0/24).
<NAS_VERSION>	NAS protocol version (Alluxio currently supports v3 only).

Basic Configuration

Use the mount table operations to add a new mount point by specifying the Alluxio path where the mount should be created and providing the path exported by the NAS server as the UFS URI. Options such as the NAS server IP address, user ID, and group ID can also be specified as part of the mount operation, as described in Configuring Mount Points.

An example ufs.yaml to create a mount point with the operator:

apiVersion: k8s-operator.alluxio.com/v1
kind: UnderFileSystem
metadata:
  name: alluxio-nas
  namespace: alx-ns
spec:
  alluxioCluster: alluxio-cluster
  path: nas://<NAS_SERVER_AUTHORITY>/<NAS_EXPORT_PATH>/
  mountPath: /nas
  mountOptions:
    fs.nas.ipAddress: <NAS_SERVER_IP>
    fs.nas.uid: "0"
    fs.nas.gid: "0"
    fs.nas.service.version: "3"

An example command to mount nas://<NAS_SERVER_AUTHORITY>/<NAS_EXPORT_PATH>/ to /nas is as follows if not using the operator:

bin/alluxio mount add --path /nas \
  --ufs-uri "nas://<NAS_SERVER_AUTHORITY>/<NAS_EXPORT_PATH>/" \
  --option fs.nas.ipAddress=<NAS_SERVER_IP> \
  --option fs.nas.uid=0 \
  --option fs.nas.gid=0 \
  --option fs.nas.service.version=3

Note: When mounting the root directory of the NAS export path, be sure to add a trailing slash (e.g., nas://<NAS_SERVER_AUTHORITY>/<NAS_EXPORT_PATH>/). NAS_SERVER_AUTHORITY is a user-defined unique field that uniquely represents a NAS server. For example, if there are two NAS servers that need to be mounted to Alluxio, NAS_SERVER_AUTHORITY can be nas_1 and nas_2 to distinguish the two different NAS servers. NAS_EXPORT_PATH is the exported path of NAS.

Advanced Configuration

When using Alluxio to mount and access NAS (Network Attached Storage), certain low-level filesystem parameters can be tuned to optimize performance and resource usage. Below are two commonly used configuration options:

1. Adjusting the buffer size for reading data from NAS

alluxio.underfs.nas.inputstream.max.buffer.size=8MB

This parameter sets the maximum buffer size used by the input stream when reading data from NAS. The default value is 8MB. The buffer caches data during file reads to reduce the number of interactions with the underlying storage. Increasing this value can typically improve sequential read performance, especially when accessing large files. However, setting this value too high may lead to increased memory usage. It is recommended to tune this based on the specific workload and the memory capacity of the node.

2. Setting the response size limit for directory listing operations (ls)

alluxio.underfs.nas.liststatus.iterable.max.byte.size.per.response=4MB

This parameter controls the maximum number of bytes each response can contain when performing ls operations (e.g., listStatus()). The default value is 4MB. This setting is mainly used to batch retrieve metadata for a large number of files, helping to avoid excessive memory consumption or network congestion caused by large responses. Reducing this value can improve system response stability, which is especially useful for directories containing a large number of sub-files or sub-directories.