CephFS
This guide describes how to configure Alluxio with CephFS as the under storage system. Alluxio supports two different implementations of under storage system for CephFS:
Prerequisites
Deploy Alluxio binary package
The Alluxio binaries must be on your machine. You can either compile Alluxio, or download the binaries locally.
Install Dependences
According to ceph packages install to install below packages:
cephfs-java
libcephfs_jni
libcephfs2
Make symbolic links
$ ln -s /usr/lib64/libcephfs_jni.so.1.0.0 /usr/lib64/libcephfs_jni.so
$ ln -s /usr/lib64/libcephfs.so.2.0.0 /usr/lib64/libcephfs.so
$ java_path=`which java | xargs readlink | sed 's#/bin/java##g'`
$ ln -s /usr/share/java/libcephfs.jar $java_path/jre/lib/ext/libcephfs.jar
Download CephFS Hadoop jar
$ curl -o $java_path/jre/lib/ext/hadoop-cephfs.jar -s https://download.ceph.com/tarballs/hadoop-cephfs.jar
Basic Setup
Configure Alluxio to use under storage systems by modifying conf/alluxio-site.properties
and conf/core-site.xml
. If them do not exist, create the configuration files from the templates
$ cp conf/alluxio-site.properties.template conf/alluxio-site.properties
$ cp conf/core-site.xml.template conf/core-site.xml
Running Alluxio Locally with CephFS
Start up Alluxio locally to see that everything works.
$ ./bin/alluxio format
$ ./bin/alluxio-start.sh local
This should start an Alluxio master and Alluxio worker. You can see the master UI at http://localhost:19999.
Last updated