Skip to main content

Static Provisioning Of JuiceFS Using in Kubernetes

This document shows how to make a static provisioned JuiceFS PersistentVolume (PV) mounted inside container.

Prerequisite

To create the CSI Driver Secret in Kubernetes, the required fields for the community edition and the cloud service edition are different, as follows:

Community edition

Take Amazon S3 as an example:

kubectl -n default create secret generic juicefs-secret \
--from-literal=name=<NAME> \
--from-literal=metaurl=redis://[:<PASSWORD>]@<HOST>:6379[/<DB>] \
--from-literal=storage=s3 \
--from-literal=bucket=https://<BUCKET>.s3.<REGION>.amazonaws.com \
--from-literal=access-key=<ACCESS_KEY> \
--from-literal=secret-key=<SECRET_KEY>
  • name: The JuiceFS file system name.
  • metaurl: Connection URL for metadata engine (e.g. Redis). Read this document for more information.
  • storage: Object storage type, such as s3, gs, oss. Read this document for the full supported list.
  • bucket: Bucket URL. Read this document to learn how to setup different object storage.
  • access-key: Access key.
  • secret-key: Secret key.

Replace fields enclosed by <> with your own environment variables. The fields enclosed [] is optional which related your deployment environment.

You should ensure:

  1. The access-key, secret-key pair has GetObject, PutObject, DeleteObject permission for the object bucket
  2. The Redis DB is clean and the password (if provided) is right

You can execute the juicefs format command to ensure the secret is OK.

./juicefs format --storage=s3 --bucket=https://<BUCKET>.s3.<REGION>.amazonaws.com \
--access-key=<ACCESS_KEY> --secret-key=<SECRET_KEY> \
redis://[:<PASSWORD>]@<HOST>:6379[/<DB>] <NAME>

Cloud service edition

kubectl -n default create secret generic juicefs-secret \
--from-literal=name=${JUICEFS_NAME} \
--from-literal=token=${JUICEFS_TOKEN} \
--from-literal=accesskey=${JUICEFS_ACCESSKEY} \
--from-literal=secretkey=${JUICEFS_SECRETKEY}
  • name: JuiceFS file system name
  • token: JuiceFS managed token. Read this document for more details.
  • accesskey: Object storage access key
  • secretkey: Object storage secret key

You should ensure accesskey and secretkey pair has GetObject, PutObject, DeleteObject permission for the object bucket.

Apply

Create PersistentVolume (PV), PersistentVolumeClaim (PVC) and sample pod

kubectl apply -f - <<EOF
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: juicefs-pv
labels:
juicefs-name: ten-pb-fs
spec:
capacity:
storage: 10Pi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
csi:
driver: csi.juicefs.com
volumeHandle: test-bucket
fsType: juicefs
nodePublishSecretRef:
name: juicefs-secret
namespace: default
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: juicefs-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
volumeMode: Filesystem
storageClassName: ""
resources:
requests:
storage: 10Pi
selector:
matchLabels:
juicefs-name: ten-pb-fs
---
apiVersion: v1
kind: Pod
metadata:
name: juicefs-app
namespace: default
spec:
containers:
- args:
- -c
- while true; do echo $(date -u) >> /data/out.txt; sleep 5; done
command:
- /bin/sh
image: centos
name: app
volumeMounts:
- mountPath: /data
name: data
resources:
requests:
cpu: 10m
volumes:
- name: data
persistentVolumeClaim:
claimName: juicefs-pvc
EOF

Check JuiceFS file system is used

After all objects are created, verify that a 10 Pi PV is created:

kubectl get pv

Verify the pod is running:

kubectl get pods

Verify that data is written onto JuiceFS file system:

kubectl exec -ti juicefs-app -- tail -f /data/out.txt

Verify the directory created as PV in JuiceFS file system by mounting it in a host:

juicefs mount -d redis://[:<PASSWORD>]@<HOST>:6379[/<DB>] /jfs