Handling Images
Two sets of container images are required for deployment:
Alluxio images, provided by your Alluxio sales representative
Publicly accessible images for third party components
The images must be served by an image registry accessible to the Kubernetes cluster. For the Alluxio images, it is expected that they are uploaded to a registry managed by the user.
An image registry is a centralized location for storing and sharing your container images. It can be either public or private. Cloud provider may come with its container registry as a service. For example, Amazon Elastic Container Registry(ECR),Azure Container Registry (ACR), and Google Container Registry (GCR). Private registries may also be provided from your local system or within private networks of your organization.
Alluxio images
The following image files are provided:
alluxio-operator-3.2.1-docker.taris the docker image for all Alluxio operator componentsalluxio-enterprise-AI-3.6-12.0.2-docker.taris the docker image for Alluxio coordinator and workers
Optionally, the following images may also be provided:
alluxio-gateway-AI-3.6-12.0.2-docker.taris the docker image for Alluxio's API gatewayalluxio-dashboard-AI-3.6-12.0.2.taris the docker image for Alluxio's Management Console
This example shows how to upload the Alluxio operator images.
# load the image to local
$ docker load -i alluxio-operator-3.2.1-docker.tar
$ docker load -i alluxio-enterprise-AI-3.6-12.0.2-docker.tar
# retag the image with your private registry
$ docker tag alluxio/operator:3.2.1 <PRIVATE_REGISTRY>/alluxio-operator:3.2.1
$ docker tag alluxio/alluxio-enterprise:AI-3.6-12.0.2 <PRIVATE_REGISTRY>/alluxio-enterprise:AI-3.6-12.0.2
# push to the remote registry
$ docker push <PRIVATE_REGISTRY>/alluxio-operator:3.2.1
$ docker push <PRIVATE_REGISTRY>/alluxio-enterprise:AI-3.6-12.0.2Unable to access public image registry
In the case that the Kubernetes cluster does not have access to the public internet, the images usually supplied by public image registries will not be accessible to download. These image must also be uploaded to an image registry accessible to the Kubernetes cluster to successfully deploy Alluxio. If your network environment cannot access the public image registry, you will encounter a timeout error when pulling the images:
# Check if the operator is running properly
$ kubectl -n alluxio-operator get pod
NAME READY STATUS RESTARTS AGE
alluxio-cluster-controller-65b59f65b4-5d667 1/1 Running 0 22s
alluxio-collectinfo-controller-667b746fd6-hfzqk 1/1 Running 0 22s
alluxio-csi-controller-c85f8f759-sqc56 0/2 ContainerCreating 0 22s
alluxio-csi-nodeplugin-5pgmg 0/2 ContainerCreating 0 22s
alluxio-csi-nodeplugin-fpkcq 0/2 ContainerCreating 0 22s
alluxio-csi-nodeplugin-j9wll 0/2 ContainerCreating 0 22s
alluxio-ufs-controller-5f69bbb878-7km58 1/1 Running 0 22sYou may notice that the cluster controller, ufs controller and collectinfo controller have started successfully,
but the csi controller and csi nodeplugin remain in the ContainerCreating state.
This is due to a timeout while pulling the dependent images.
By using kubectl describe pod to view detailed information, you will see error messages similar to the following:
$ kubectl -n alluxio-operator describe pod -l app.kubernetes.io/component=csi-controller
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 10m default-scheduler Successfully assigned alluxio-operator/alluxio-csi-controller-c85f8f759-sqc56 to <nodeName>
Normal AllocIPSucceed 10m terway-daemon Alloc IP 10.0.0.27/24 took 28.443992ms
Normal Pulling 10m kubelet Pulling image "registry.xxx.com/alluxio/operator:3.2.1"
Normal Pulled 10m kubelet Successfully pulled image "registry.xxx.com/alluxio/operator:3.2.1" in 5.55s (5.55s including waiting)
Normal Created 10m kubelet Created container csi-controller
Normal Started 10m kubelet Started container csi-controller
Warning Failed 8m20s (x2 over 10m) kubelet Failed to pull image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": failed to pull and unpack image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": failed to resolve reference "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": failed to do request: Head "https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/sig-storage/csi-provisioner/manifests/v2.0.5": dial tcp 142.251.8.82:443: i/o timeout
Warning Failed 8m20s (x3 over 10m) kubelet Error: ErrImagePull
Warning Failed 7m40s (x5 over 10m) kubelet Error: ImagePullBackOff
Warning Failed 6m56s (x2 over 9m19s) kubelet Failed to pull image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": rpc error: code = DeadlineExceeded desc = failed to pull and unpack image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": failed to resolve reference "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5": failed to do request: Head "https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/sig-storage/csi-provisioner/manifests/v2.0.5": dial tcp 64.233.187.82:443: i/o timeout
Normal Pulling 5m29s (x5 over 10m) kubelet Pulling image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5"
Normal BackOff 30s (x28 over 10m) kubelet Back-off pulling image "registry.k8s.io/sig-storage/csi-provisioner:v2.0.5"Third party dependent images
operator CSI
registry.k8s.io/sig-storage/csi-node-driver-registrar
v2.0.0
csi driver registrar dependency
operator CSI
registry.k8s.io/sig-storage/csi-provisioner
v2.0.5
csi provisioner dependency
cluster ETCD
docker.io/bitnami/etcd
3.5.9-debian-11-r24
etcd dependency
cluster ETCD
docker.io/bitnami/os-shell
11-debian-11-r2
os-shell dependency
cluster monitor
grafana/grafana
10.4.5
Monitoring dashboard
cluster monitor
prom/prometheus
v2.52.0
Metrics collection
The commands to pull the Docker images and upload them to your private image registry are as follows; be sure to set platform (ex. linux/amd64) that matches the Kubernetes cluster environment.
# Pull the Docker images
$ docker pull --platform linux/amd64 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.0.0
$ docker pull --platform linux/amd64 registry.k8s.io/sig-storage/csi-provisioner:v2.0.5
$ docker pull --platform linux/amd64 docker.io/bitnami/etcd:3.5.9-debian-11-r24
$ docker pull --platform linux/amd64 docker.io/bitnami/os-shell:11-debian-11-r2
$ docker pull --platform linux/amd64 grafana/grafana:10.4.5
$ docker pull --platform linux/amd64 prom/prometheus:v2.52.0
# Tag the images with your private registry
$ docker tag registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.0.0 <PRIVATE_REGISTRY>/csi-node-driver-registrar:v2.0.0
$ docker tag registry.k8s.io/sig-storage/csi-provisioner:v2.0.5 <PRIVATE_REGISTRY>/csi-provisioner:v2.0.5
$ docker tag docker.io/bitnami/etcd:3.5.9-debian-11-r24 <PRIVATE_REGISTRY>/etcd:3.5.9-debian-11-r24
$ docker tag docker.io/bitnami/os-shell:11-debian-11-r2 <PRIVATE_REGISTRY>/os-shell:11-debian-11-r2
$ docker tag grafana/grafana:10.4.5 <PRIVATE_REGISTRY>/grafana:10.4.5
$ docker tag prom/prometheus:v2.52.0 <PRIVATE_REGISTRY>/prometheus:v2.52.0
# Push the images to your private registry
$ docker push <PRIVATE_REGISTRY>/csi-node-driver-registrar:v2.0.0
$ docker push <PRIVATE_REGISTRY>/csi-provisioner:v2.0.5
$ docker push <PRIVATE_REGISTRY>/etcd:3.5.9-debian-11-r24
$ docker push <PRIVATE_REGISTRY>/os-shell:11-debian-11-r2
$ docker push <PRIVATE_REGISTRY>/grafana:10.4.5
$ docker push <PRIVATE_REGISTRY>/prometheus:v2.52.0Updating alluxio-operator.yaml
Update the image addresses in the alluxio-operator/alluxio-operator.yaml file,
adding the provisioner and driverRegistrar image addresses:
global:
image: <PRIVATE_REGISTRY>/alluxio-operator
imageTag: 3.2.1
alluxio-csi:
controllerPlugin:
provisioner:
image: <PRIVATE_REGISTRY>/csi-provisioner:v2.0.5
nodePlugin:
driverRegistrar:
image: <PRIVATE_REGISTRY>/csi-node-driver-registrar:v2.0.0Updating alluxio-cluster.yaml
Modify the image addresses in the alluxio-operator/alluxio-cluster.yaml file accordingly.
apiVersion: k8s-operator.alluxio.com/v1
kind: AlluxioCluster
metadata:
name: alluxio-cluster
namespace: alx-ns
spec:
image: <PRIVATE_REGISTRY>/alluxio-enterprise
imageTag: AI-3.6-12.0.2
properties:
worker:
count: 2
pagestore:
size: 100Gi
etcd:
image:
registry: <PRIVATE_REGISTRY>
repository: <PRIVATE_REPOSITORY>/etcd
tag: 3.5.9-debian-11-r24
volumePermissions:
image:
registry: <PRIVATE_REGISTRY>
repository: <PRIVATE_REPOSITORY>/os-shell
tag: 11-debian-11-r2
prometheus:
image: <PRIVATE_REGISTRY>/prometheus
imageTag: v2.52.0
grafana:
image: <PRIVATE_REGISTRY>/grafana
imageTag: 10.4.5Last updated