Skip to main content

Persistent Access Log

Starting with JuiceFS Enterprise Edition 5.3.8, the JuiceFS client can persist all file system operations, including reads, writes, file creations, and deletions, to an access log file. This log is useful for debugging, performance analysis, and integration with external log collection platforms such as OpenTelemetry, Filebeat, and Fluentd.

This feature is supported by the mount, gateway, and webdav commands. When you specify --access-log, JuiceFS records all file system operations in the specified file. You can analyze the resulting log using the juicefs profile command.

Starting with version 5.4.5, the rotation options were renamed to --log-rotate-size and --log-rotate-files. The previous names, --access-log-size and --access-log-files, remain supported as aliases. These options also control the automatic rotation of juicefs.log. See Client log rotation.

Parameters​

OptionDescription
--access-log=PATHPath to the access log file. When this option is specified, JuiceFS records all file system operations in the specified file. If omitted, access logs are not persisted. The log file is created with permissions set to 0666, allowing all users to read and write to it.
--log-rotate-size=SIZE Added in v5.4.5 (alias --access-log-size, default 300M)Maximum size of an individual log file before rotation is triggered. When the active log file reaches this threshold, JuiceFS rotates it. Size suffixes such as K, M, G, and T are supported (case-insensitive). For example, 2G represents 2 GiB. This option also controls the rotation threshold for juicefs.log. Set it to 0 to disable automatic rotation of juicefs.log. In high-I/O workloads, consider increasing this value to reduce rotation frequency and minimize the impact on external log collectors.
--log-rotate-files=N Added in v5.4.5 (alias --access-log-files, default 7)Number of rotated log files to retain. The minimum value is 2; a value of 1 is treated as 2. The two most recently rotated files are always kept uncompressed, giving external log collectors enough time to finish reading them. Older files (those with a sequence number greater than 2) are compressed using gzip to save disk space. When the number of rotated files exceeds N, the oldest file is deleted. This option also controls the number of rotated juicefs.log files retained.
note

The rotation options are available for the mount, gateway, and webdav commands. They also control the automatic rotation of juicefs.log. See Client log rotation for details.

Usage​

juicefs mount test-volume /mnt/jfs \
--access-log /var/log/juicefs-access.log \
--log-rotate-size 2G \
--log-rotate-files 10

Log rotation​

Taking --log-rotate-files 5 as an example, the rotation works as follows:

  1. When the active log juicefs-access.log reaches the --log-rotate-size threshold, rotation is triggered.

  2. juicefs-access.log is renamed to juicefs-access.log.1.

  3. The sequence number of each existing rotated file is incremented by 1. Files with a sequence number greater than 2 are compressed:

    • juicefs-access.log.1 → juicefs-access.log.2 (kept uncompressed)
    • juicefs-access.log.2 → compressed to juicefs-access.log.3.gz
    • juicefs-access.log.3.gz → juicefs-access.log.4.gz
    • juicefs-access.log.4.gz → juicefs-access.log.5.gz
    • juicefs-access.log.5.gz → deleted (exceeds the retention count)
  4. A new juicefs-access.log file is created for subsequent writes.

To prevent the disk from being filled up, if the active log file cannot be renamed during rotation (for example, because it is held by an external tool) and its size exceeds N × --log-rotate-size (where N is the retention count), the client truncates the file to zero bytes. This discards all of its content.

The final layout on disk:

juicefs-access.log           # active log file
juicefs-access.log.1 # most recently rotated file, uncompressed
juicefs-access.log.2 # second most recently rotated file, uncompressed
juicefs-access.log.3.gz # older rotated files, compressed
juicefs-access.log.4.gz
juicefs-access.log.5.gz

Why keep two uncompressed files?​

During rotation, older log files are compressed using gzip and eventually deleted. If external log collectors such as OpenTelemetry or Filebeat track files using a glob pattern (for example, juicefs-access.log*), they may not finish reading a file before it is compressed or deleted, resulting in log loss.

By keeping the two most recently rotated files uncompressed:

  • File names are predictable (juicefs-access.log.1and juicefs-access.log.2), allowing collectors to track them stably via a glob pattern.
  • Collectors have enough time to finish collecting.
  • Logs are not lost even when rotation happens quickly.

Disk space estimation​

Assume the following configuration: --log-rotate-size 2G and --log-rotate-files 5.

FileSize
juicefs-access.log.1About 2 GiB (uncompressed)
juicefs-access.log.2About 2 GiB (uncompressed)
juicefs-access.log.3.gz ~ juicefs-access.log.5.gzAbout 0.5-1 GiB (compressed; plain text compresses extremely well)
TotalAbout 5 GiB

Integrating with external log collectors​

OpenTelemetry Collector​

receivers:
filelog:
include:
- /var/log/juicefs-access.log
- /var/log/juicefs-access.log.?
exclude:
- /var/log/juicefs-access.log.*.gz
start_at: beginning
service:
pipelines:
logs:
receivers: [filelog]
exporters: [your_exporter]

Filebeat​

filebeat.inputs:
- type: filestream
paths:
- /var/log/juicefs-access.log
- /var/log/juicefs-access.log.?
exclude_files: ['.gz$']

Notes​

  • The glob pattern on the collector side should precisely match uncompressed files (for example juicefs-access.log.?), to avoid reading .gz files as garbled text.
  • The OpenTelemetry Filelog receiver tracks files by inode and automatically detects new inodes after rotation. However, it is recommended to set start_at: beginning to avoid missing historical data at first startup.
  • Cross-file time ordering relies on the timestamps of log entries themselves, rather than the file-level read order.
  • Collectors stop following a file after it has not been written to for a period of time (the exact timeout depends on the collector), so only the active juicefs-access.log file is continuously read.