Static Provisioning Of JuiceFS Using in Kubernetes
This document shows how to make a static provisioned JuiceFS PersistentVolume (PV) mounted inside container.
Prerequisite
To create the CSI Driver Secret
in Kubernetes, the required fields for the community edition and the cloud service edition are different, as follows:
Community edition
Take Amazon S3 as an example:
apiVersion: v1
kind: Secret
metadata:
name: juicefs-secret
type: Opaque
stringData:
name: <NAME>
metaurl: redis://[:<PASSWORD>]@<HOST>:6379[/<DB>]
storage: s3
bucket: https://<BUCKET>.s3.<REGION>.amazonaws.com
access-key: <ACCESS_KEY>
secret-key: <SECRET_KEY>
name
: The JuiceFS file system name.metaurl
: Connection URL for metadata engine (e.g. Redis). Read this document for more information.storage
: Object storage type, such ass3
,gs
,oss
. Read this document for the full supported list.bucket
: Bucket URL. Read this document to learn how to setup different object storage.access-key
: Access key.secret-key
: Secret key.
Replace fields enclosed by <>
with your own environment variables. The fields enclosed []
is optional which related your deployment environment.
You should ensure:
- The
access-key
,secret-key
pair hasGetObject
,PutObject
,DeleteObject
permission for the object bucket - The Redis DB is clean and the password (if provided) is right
You can execute the juicefs format
command to ensure the Secret is OK:
juicefs format --storage=s3 --bucket=https://<BUCKET>.s3.<REGION>.amazonaws.com \
--access-key=<ACCESS_KEY> --secret-key=<SECRET_KEY> \
redis://[:<PASSWORD>]@<HOST>:6379[/<DB>] <NAME>
Cloud service edition
apiVersion: v1
kind: Secret
metadata:
name: juicefs-secret
type: Opaque
stringData:
name: ${JUICEFS_NAME}
token: ${JUICEFS_TOKEN}
access-key: ${JUICEFS_ACCESSKEY}
secret-key: ${JUICEFS_SECRETKEY}
name
: JuiceFS file system nametoken
: JuiceFS managed token. Read this document for more details.access-key
: Object storage access keysecret-key
: Object storage secret key
You should ensure access-key
and secret-key
pair has GetObject
, PutObject
, DeleteObject
permission for the object bucket.
Deploy
Create PersistentVolume (PV), PersistentVolumeClaim (PVC) and sample pod:
note
The PV volumeHandle needs to be unique within the cluster, just use the PV name.
apiVersion: v1
kind: PersistentVolume
metadata:
name: juicefs-pv
labels:
juicefs-name: ten-pb-fs
spec:
capacity:
storage: 10Pi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
csi:
driver: csi.juicefs.com
volumeHandle: juicefs-pv
fsType: juicefs
nodePublishSecretRef:
name: juicefs-secret
namespace: default
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: juicefs-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
volumeMode: Filesystem
storageClassName: ""
resources:
requests:
storage: 10Pi
selector:
matchLabels:
juicefs-name: ten-pb-fs
---
apiVersion: v1
kind: Pod
metadata:
name: juicefs-app
namespace: default
spec:
containers:
- args:
- -c
- while true; do echo $(date -u) >> /data/out.txt; sleep 5; done
command:
- /bin/sh
image: centos
name: app
volumeMounts:
- mountPath: /data
name: data
resources:
requests:
cpu: 10m
volumes:
- name: data
persistentVolumeClaim:
claimName: juicefs-pvc
Check JuiceFS file system is used
After all objects are created, verify that a 10 Pi PV is created:
kubectl get pv
Verify the pod is running:
kubectl get pods
Verify that data is written onto JuiceFS file system:
kubectl exec -ti juicefs-app -- tail -f /data/out.txt
Verify the directory created as PV in JuiceFS file system by mounting it in a host:
juicefs mount -d redis://[:<PASSWORD>]@<HOST>:6379[/<DB>] /jfs