Production Recommendations
Best practices and recommended settings when going production.
PV settings
Below settings are recommended for a production environment.
- Configure more readable names for PV directory
- Enable Automatic Mount Point Recovery
- The
--writeback
option is strongly advised against, as it can easily cause data loss especially when used inside containers, if not properly managed. See "Write Cache in Client (Community Edition)" and "Write Cache in Client (Cloud Service)". - When cluster is low on resources, refer to optimization techniques in Resource Optimization.
Collect mount pod logs using EFK
Troubleshooting CSI Driver usually involves reading mount pod logs, if checking mount pod logs in real time isn't enough, consider deploying an EFK (Elasticsearch + Fluentd + Kibana) stack (or other suitable systems) in Kubernetes Cluster to collect pod logs for query. Taking EFK for example:
- Elasticsearch: index logs and provide a complete full-text search engine, which can facilitate users to retrieve the required data from the log. For installation, please refer to the official documentation.
- Fluentd: fetch container log files, filter and transform log data, and then deliver the data to the Elasticsearch cluster. For installation, please refer to the official documentation.
- Kibana: visual analysis of logs, including log search, processing, and gorgeous dashboard display, etc. For installation, please refer to the official documentation.
Mount pod is labeled app.kubernetes.io/name: juicefs-mount
. Add below config to the Fluentd configuration:
<filter kubernetes.**>
@id filter_log
@type grep
<regexp>
key $.kubernetes.labels.app_kubernetes_io/name
pattern ^juicefs-mount$
</regexp>
</filter>
And add the following parser plugin to the Fluentd configuration file:
<filter kubernetes.**>
@id filter_parser
@type parser
key_name log
reserve_data true
remove_key_name_field true
<parse>
@type multi_format
<pattern>
format json
</pattern>
<pattern>
format none
</pattern>
</parse>
</filter>