Skip to main content

JuiceFS CSI Driver 从 v0.9.0 升级到 v0.10.0+

为了降低升级操作对业务系统的影响,JuiceFS CSI Driver 从 v0.10.0 开始将 JuiceFS 客户端与 CSI Driver 进行了分离,用户可以根据需要单独对 JuiceFS 客户端或 CSI Driver 进行升级。但从 v0.9.0 升级到 v0.10.0+ 时需要重启服务,这会导致所有的 PV 在升级过程中不可用。您可以根据实际情况,参考以下两种方案进行升级。

方案一:逐台升级

如果使用 JuiceFS 的应用不可被中断,可以采用此方案。

1. 创建新版本新增的资源

将以下的 YAML 文件保存成 csi_new_resource.yaml,然后执行 kubectl apply -f csi_new_resource.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: juicefs-csi-external-node-service-role
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- create
- update
- delete
- patch
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"
name: juicefs-csi-node-service-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: juicefs-csi-external-node-service-role
subjects:
- kind: ServiceAccount
name: juicefs-csi-node-sa
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: juicefs-csi-node-sa
namespace: kube-system
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"

2. 将 node service DaemonSet 的升级策略改成 OnDelete

kubectl -n kube-system patch ds <ds_name> -p '{"spec": {"updateStrategy": {"type": "OnDelete"}}}'

3. 升级 CSI 驱动的 node service DaemonSet

将以下的 YAML 文件保存成 ds_patch.yaml,然后执行 kubectl -n kube-system patch ds <ds_name> --patch "$(cat ds_patch.yaml)"

spec:
template:
spec:
containers:
- name: juicefs-plugin
image: juicedata/juicefs-csi-driver:v0.10.6
args:
- --endpoint=$(CSI_ENDPOINT)
- --logtostderr
- --nodeid=$(NODE_ID)
- --v=5
- --enable-manager=true
env:
- name: CSI_ENDPOINT
value: unix:/csi/csi.sock
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: JUICEFS_MOUNT_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: JUICEFS_MOUNT_PATH
value: /var/lib/juicefs/volume
- name: JUICEFS_CONFIG_PATH
value: /var/lib/juicefs/config
volumeMounts:
- mountPath: /jfs
mountPropagation: Bidirectional
name: jfs-dir
- mountPath: /root/.juicefs
mountPropagation: Bidirectional
name: jfs-root-dir
serviceAccount: juicefs-csi-node-sa
volumes:
- hostPath:
path: /var/lib/juicefs/volume
type: DirectoryOrCreate
name: jfs-dir
- hostPath:
path: /var/lib/juicefs/config
type: DirectoryOrCreate
name: jfs-root-dir

4. 逐台升级 node service

在每台节点上执行以下操作:

  1. 删除当前节点上的 CSI Driver pod:
kubectl -n kube-system delete po juicefs-csi-node-df7m7
  1. 确认新的 node service pod 已经 ready(假设节点名为 kube-node-2):
$ kubectl -n kube-system get po -o wide -l app.kubernetes.io/name=juicefs-csi-driver | grep kube-node-2
juicefs-csi-node-6bgc6 3/3 Running 0 60s 172.16.11.11 kube-node-2 <none> <none>
  1. 在当前节点上,删除使用 JuiceFS 的业务 pod 并重新创建。

  2. 确认使用 JuiceFS 的业务 pod 已经 ready,并检查是否正常工作。

5. 升级 CSI Driver controller 服务及其 role

将以下的 YAML 文件保存成 sts_patch.yaml,然后执行 kubectl -n kube-system patch sts <sts_name> --patch "$(cat sts_patch.yaml)"

spec:
template:
spec:
containers:
- name: juicefs-plugin
image: juicedata/juicefs-csi-driver:v0.10.6
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: JUICEFS_MOUNT_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: JUICEFS_MOUNT_PATH
value: /var/lib/juicefs/volume
- name: JUICEFS_CONFIG_PATH
value: /var/lib/juicefs/config
volumeMounts:
- mountPath: /jfs
mountPropagation: Bidirectional
name: jfs-dir
- mountPath: /root/.juicefs
mountPropagation: Bidirectional
name: jfs-root-dir
volumes:
- hostPath:
path: /var/lib/juicefs/volume
type: DirectoryOrCreate
name: jfs-dir
- hostPath:
path: /var/lib/juicefs/config
type: DirectoryOrCreate
name: jfs-root-dir

将以下的 YAML 文件保存成 clusterrole_patch.yaml,然后执行 kubectl patch clusterrole <role_name> --patch "$(cat clusterrole_patch.yaml)"

rules:
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- persistentvolumeclaims
verbs:
- get
- list
- watch
- update
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- get
- list
- watch
- create
- update
- patch
- apiGroups:
- storage.k8s.io
resources:
- csinodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- update
- patch
- delete

方案二:整体升级

1. 停掉所有使用 JuiceFS 的应用

2. 升级 CSI 驱动

1. 创建新版本新增的资源

将以下的 YAML 文件保存成 csi_new_resource.yaml,然后执行 kubectl apply -f csi_new_resource.yaml"

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: juicefs-csi-external-node-service-role
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- create
- update
- delete
- patch
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"
name: juicefs-csi-node-service-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: juicefs-csi-external-node-service-role
subjects:
- kind: ServiceAccount
name: juicefs-csi-node-sa
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: juicefs-csi-node-sa
namespace: kube-system
labels:
app.kubernetes.io/name: juicefs-csi-driver
app.kubernetes.io/instance: juicefs-csi-driver
app.kubernetes.io/version: "v0.10.6"

2. 升级 CSI Driver node service DaemonSet

将以下的 YAML 文件保存成 ds_patch.yaml,然后执行 kubectl -n kube-system patch ds <ds_name> --patch "$(cat ds_patch.yaml)"

spec:
updateStrategy:
type: RollingUpdate
template:
spec:
containers:
- name: juicefs-plugin
image: juicedata/juicefs-csi-driver:v0.10.6
args:
- --endpoint=$(CSI_ENDPOINT)
- --logtostderr
- --nodeid=$(NODE_ID)
- --v=5
- --enable-manager=true
env:
- name: CSI_ENDPOINT
value: unix:/csi/csi.sock
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: JUICEFS_MOUNT_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: JUICEFS_MOUNT_PATH
value: /var/lib/juicefs/volume
- name: JUICEFS_CONFIG_PATH
value: /var/lib/juicefs/config
volumeMounts:
- mountPath: /jfs
mountPropagation: Bidirectional
name: jfs-dir
- mountPath: /root/.juicefs
mountPropagation: Bidirectional
name: jfs-root-dir
serviceAccount: juicefs-csi-node-sa
volumes:
- hostPath:
path: /var/lib/juicefs/volume
type: DirectoryOrCreate
name: jfs-dir
- hostPath:
path: /var/lib/juicefs/config
type: DirectoryOrCreate
name: jfs-root-dir

确认所有 juicefs-csi-node-*** pods 都升级了。

3. 升级 CSI Driver controller 服务及其 role

将以下的 YAML 文件保存成 sts_patch.yaml,然后执行 kubectl -n kube-system patch sts <sts_name> --patch "$(cat sts_patch.yaml)"

spec:
template:
spec:
containers:
- name: juicefs-plugin
image: juicedata/juicefs-csi-driver:v0.10.6
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: JUICEFS_MOUNT_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: JUICEFS_MOUNT_PATH
value: /var/lib/juicefs/volume
- name: JUICEFS_CONFIG_PATH
value: /var/lib/juicefs/config
volumeMounts:
- mountPath: /jfs
mountPropagation: Bidirectional
name: jfs-dir
- mountPath: /root/.juicefs
mountPropagation: Bidirectional
name: jfs-root-dir
volumes:
- hostPath:
path: /var/lib/juicefs/volume
type: DirectoryOrCreate
name: jfs-dir
- hostPath:
path: /var/lib/juicefs/config
type: DirectoryOrCreate
name: jfs-root-dir

将以下的 YAML 文件保存成 clusterrole_patch.yaml,然后执行 kubectl patch clusterrole <role_name> --patch "$(cat clusterrole_patch.yaml)"

rules:
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- persistentvolumeclaims
verbs:
- get
- list
- watch
- update
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- get
- list
- watch
- create
- update
- patch
- apiGroups:
- storage.k8s.io
resources:
- csinodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- update
- patch
- delete

确认 juicefs-csi-controller-*** pod 已经升级.

另外,如果 JuiceFS CSI Driver 是使用 Helm 安装的,也可以使用 Helm 来升级。

3. 重新启动所有使用 JuiceFS 的应用