JuiceFS CSI Driver
The JuiceFS CSI Driver implements the CSI specification for container orchestrators to manage the lifecycle of JuiceFS file system.
Prerequisites
- Kubernetes 1.14+
Installation
There are two ways to install JuiceFS CSI Driver.
1. Install via Helm
Prerequisites
- Helm 3.1.0+
Install Helm
Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes resources.
To install Helm, refer to the Helm Installation Guide and ensure that the helm
binary is in the PATH
of your shell.
Using Helm To Deploy
Prepare a YAML file
Create a configuration file, for example:
values.yaml
, copy and complete the following configuration information. Among them, thebackend
part is the information related to the JuiceFS file system, you can refer to "JuiceFS Quick Start Guide" for more information. If you are using a JuiceFS volume that has been created, you only need to fill in the two itemsname
andmetaurl
. ThemountPod
part can specify CPU/memory limits and requests of mount pod for pods using this driver. Unneeded items should be deleted, or its value should be left blank.values.yamlstorageClasses:
- name: juicefs-sc
enabled: true
reclaimPolicy: Retain
backend:
name: "<name>"
metaurl: "<meta-url>"
storage: "<storage-type>"
accessKey: "<access-key>"
secretKey: "<secret-key>"
bucket: "<bucket>"
mountPod:
resources:
limits:
cpu: "<cpu-limit>"
memory: "<memory-limit>"
requests:
cpu: "<cpu-request>"
memory: "<memory-request>"Check and update kubelet root directory
Execute the following command.
ps -ef | grep kubelet | grep root-dir
If the result is not empty, it means that the root directory (
--root-dir
) of kubelet is not the default value (/var/lib/kubelet
) and you need to setkubeletDir
to the current root directly of kubelet in the configuration filevalues.yaml
prepared in the first step.kubeletDir: <kubelet-dir>
Deploy
helm repo add juicefs-csi-driver https://juicedata.github.io/charts/
helm repo update
helm install juicefs-csi-driver juicefs-csi-driver/juicefs-csi-driver -n kube-system -f ./values.yamlCheck the deployment
Check pods are running: the deployment will launch a
StatefulSet
namedjuicefs-csi-controller
with replica1
and aDaemonSet
namedjuicefs-csi-node
, so runkubectl -n kube-system get pods -l app.kubernetes.io/name=juicefs-csi-driver
should seen+1
(wheren
is the number of worker nodes of the Kubernetes cluster) pods is running. For example:$ kubectl -n kube-system get pods -l app.kubernetes.io/name=juicefs-csi-driver
NAME READY STATUS RESTARTS AGE
juicefs-csi-controller-0 3/3 Running 0 22m
juicefs-csi-node-v9tzb 3/3 Running 0 14mCheck secret:
kubectl -n kube-system describe secret juicefs-sc-secret
will show the secret with abovebackend
fields invalues.yaml
:Name: juicefs-sc-secret
Namespace: kube-system
Labels: app.kubernetes.io/instance=juicefs-csi-driver
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=juicefs-csi-driver
app.kubernetes.io/version=0.7.0
helm.sh/chart=juicefs-csi-driver-0.1.0
Annotations: meta.helm.sh/release-name: juicefs-csi-driver
meta.helm.sh/release-namespace: default
Type: Opaque
Data
====
access-key: 0 bytes
bucket: 47 bytes
metaurl: 54 bytes
name: 4 bytes
secret-key: 0 bytes
storage: 2 bytesCheck StorageClass:
kubectl get sc juicefs-sc
will show the StorageClass like this:NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
juicefs-sc csi.juicefs.com Retain Immediate false 69m
2. Install via kubectl
Since Kubernetes will deprecate some old APIs when a new version is released, you need to choose the appropriate deployment configuration file.
Check the root directory path of kubelet
Execute the following command on any non-Master node in the Kubernetes cluster.
ps -ef | grep kubelet | grep root-dir
Deploy
If the check command returns a non-empty result, it means that the root directory (
--root-dir
) of the kubelet is not the default (/var/lib/kubelet
), so you need to update thekubeletDir
path in the CSI Driver's deployment file and deploy.# Kubernetes version >= v1.18
curl -sSL https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml | sed 's@/var/lib/kubelet@{{KUBELET_DIR}}@g' | kubectl apply -f -
# Kubernetes version < v1.18
curl -sSL https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s_before_v1_18.yaml | sed 's@/var/lib/kubelet@{{KUBELET_DIR}}@g' | kubectl apply -f -note
Please replace
{{KUBELET_DIR}}
in the above command with the actual root directory path of kubelet.If the check command returns an empty result, you can deploy directly without modifying the configuration:
# Kubernetes version >= v1.18
kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml
# Kubernetes version < v1.18
kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s_before_v1_18.yaml
Troubleshooting & FAQs
If you encounter any issue, please refer to Troubleshooting or FAQ document.
Upgrade CSI Driver
Refer to Upgrade CSI Driver document.
Examples
Before the example, you need to:
- Get yourself familiar with how to setup Kubernetes and how to use JuiceFS file system.
- Make sure JuiceFS is accessible from Kuberenetes cluster. It is recommended to create the file system inside the same region as Kubernetes cluster.
- Install JuiceFS CSI driver following the Installation steps.
Table of contents
- Static Provisioning
- Dynamic Provisioning
- Config File System Settings
- Config Mount Options
- Mount Subdirectory
- Data Encryption
- Use ReadWriteMany and ReadOnlyMany
- Config Mount Pod Resources
- Set Configuration Files and Environment Variables in Mount Pod
- Delay Deletion of Mount Pod
info
Since JuiceFS is an elastic file system it doesn't really enforce any file system capacity. The actual storage capacity value in PersistentVolume
and PersistentVolumeClaim
is not used when creating the file system. However, since the storage capacity is a required field by Kubernetes, you must specify the value and you can use any valid value e.g. 10Pi
for the capacity.
Known issues
- JuiceFS CSI Driver v0.10.0 and above does not support wildcards in
--cache-dir
mount option