High-Performance Find

If the JuiceFS file system contains a large number of files that need to be searched by specific criteria, the standard Linux find may become slow. Starting with v5.3.7, you can use juicefs find as a high-performance alternative, typically 1 to 2 orders of magnitude faster than find. For details, see Command Reference.

Note

juicefs find is available after upgrading the client to v5.3.7. Even if the metadata service is not upgraded to v5.3.7, the command will still work, but client-side search performance will be significantly degraded.

Background

Why juicefs find?

After deploying a large-scale distributed file system, storage governance becomes challenging:

• Large volumes of historical data consume storage space, but no one knows which files are still in use and which are no longer accessed.
• Storage teams need to periodically audit files that have not been accessed for a long time and ask application units to confirm whether they can be deleted.
• Common requests include: "List files not accessed for more than X days" or "Find files larger than Y GiB."

Solution comparison

Solution	Characteristics
Standard Linux find	Single-node traversal; cannot leverage the distributed scanning capabilities of the JuiceFS metadata service. Extremely slow for large directories.
juicefs find ✅	Server‑side filtering with predicates such as atime/size/type; returns precise results with zero external dependencies.

tip

juicefs find performs recursive searches directly inside JuiceFS, using a two-stage filtering approach (server-side metadata pre-filtering + client-side mount point evaluation) that balances performance and flexibility.

Core value

juicefs find provides a complete "discover → confirm → execute" loop for storage governance:

Locate cold data → Application team review → Create deletion tasks → Reclaim storage space

1. Discover: Use predicate combinations like -atime, -size, and -user to quickly identify cold data among massive files.
2. Confirm: Export matching results to a list with -print for application teams to review and decide what can be cleaned up.
3. Execute: After confirmation, batch-delete with -delete or use -exec to invoke external tools (for example, archiving, compression) to reclaim storage and reduce costs.

Feature overview

The juicefs find command recursively searches for files or directories in JuiceFS mount points based on conditions and supports executing actions such as printing, deleting, and executing external commands.

The command works in two stages:

1. Server-side filtering (metadata side): Uses an expression AST to quickly filter on the metadata tree, reducing the volume of data returned.
2. Client-side filtering and action execution (mount side): Performs final expression evaluation on entries returned by the server and executes actions. The default action is -print.

Command format

juicefs find [-p N] [PATH ...] [EXPRESSION]

Simple examples:

juicefs find /mnt/jfs -name "*.log"
juicefs find /mnt/jfs -type f -size +10M -print
juicefs find /mnt/jfs -name "*.tmp" -a -delete
juicefs find /mnt/jfs -type f -exec ls -l {} \;

Advanced usage examples:

# Scan with 8 concurrent threads.
juicefs find -p 8 /mnt/jfs -name "*.log"

# Find large files.
juicefs find /mnt/jfs -type f -size +1G

# Find logs modified within the last day.
juicefs find /mnt/jfs -name "*.log" -mtime -1

# Case-insensitive path matching
juicefs find /mnt/jfs -ipath "*backup*"

# Combine conditions
juicefs find /mnt/jfs -type f -a \( -name "*.go" -o -name "*.md" \)

# Time comparison: modification time newer than a specified date
juicefs find /mnt/jfs -newermt "2024-01-01"

# Time comparison: access time newer than a reference file
juicefs find /mnt/jfs -neweram /mnt/jfs/reference.txt

# Execute actions
juicefs find /mnt/jfs -type f -name "*.tmp" -delete
juicefs find /mnt/jfs -type f -name "*.log" -exec gzip {} \;

Options and syntax

Traversal options

• -p, --threads N: Number of concurrent threads (default: 10) to speed up scanning of large directories. Must appear before the path(s).
• -maxdepth N: Maximum traversal depth. Can appear anywhere in the expression.
• -mindepth N: Minimum matching depth. Can appear anywhere in the expression.

Logical operators

• AND: -a, -and (implicit AND also supported).
• OR: -o, -or.
• NOT: !, -not.
• Grouping: (...).

Expressions follow standard operator precedence (from highest to lowest):

1. Parentheses
2. NOT
3. AND
4. OR

Supported predicates

Predicate	Description	Lifecycle management use
-atime [+-]N	File was last accessed N days ago. +N matches files older than N days; -N matches files accessed within the last N days (both excluding exactly N days).	Core predicate for identifying cold data.
-mtime [+-]N	File content was last modified N days ago. Same +N/-N semantics.	Identify files that have not been updated for a long time.
-ctime [+-]N	File status was last changed N days ago (including content modification, permission/owner changes, etc.). Same +N/-N semantics.	Identify files by status change time range.
-size [+-]N[KMGT]	File size. + for greater than, - for less than. K/M/G/T use powers of 1024).	Identify large files occupying significant space.
-type f/d/l	File type (file/directory/symlink).	Distinguish files from directories.
-name/-iname	File name (supports wildcards).	Filter by extension (for example, .log, .tmp).
-path/-ipath	Path matching (supports wildcards; * matches /).	Filter by directory.
-user/-group	Filter by file owner/group. Supports both names and numeric IDs (for example, UID, GID).	Aggregate cold data by department.
-empty	Empty files or directories.	Clean up residual empty files/directories.
-perm MODE	Filter by permissions. -perm 644 matches files whose permissions are exactly 644; -perm -644 matches files with at least the specified permission bits set; -perm /644 permission has any of the specified bits.	Identify permission anomalies.
-links	Filter by number of hard links.	Identify unreferenced files.
-inum [+-]N	Filter by inode number. + for greater than; - for less than.	Locate files associated with a specific inode.
-regex/-iregex	Regex match against full path using RE2 syntax, with automatic full-path anchoring. -iregex is case-insensitive.	Complex path pattern matching.
-newerXY FILE	Match files newer than a reference file or date. X is the time field (a=access / c=change / m=modify); Y is the reference type (a/c/m for file, t for date string); -newer is shorthand for -newermm.	Compare files against a reference file or date.
-mmin/-amin/-cmin	Minute-level time filtering.	Fine-grained time control.
-nouser/-nogroup	Files with no corresponding user/group.	Identify orphaned files.

The t mode of -newerXY supports the following date formats, tried in this order:

• YYYY-MM-DD HH:MM:SS
• RFC 3339 (with timezone)
• ISO 8601 (without timezone)
• YYYY-MM-DD (date only)
• Any format accepted by the system date --date command (for example, 1 year ago, last Monday)

Supported actions

Action	Description
-print	Print the path (default action).
-print0	Print the path with a null byte delimiter (suitable for use with xargs -0).
-printf FORMAT	Print matching files using a format string. Supported placeholders include %p (full path), %f (file name), %s (file size), %M (symbolic permissions), %u/%g (owner/group name), %t/%a/%c (time), etc.
-exec CMD {} \;	Execute a command for each matching file. The -exec ... + form is not yet supported.
-delete	Delete matching files.

Typical lifecycle management scenarios

The following examples demonstrate common storage lifecycle management tasks using juicefs find:

Scenario 1: Generate a monthly storage governance report

# Find all files not accessed for over 90 days and export to a list.
juicefs find /mnt/jfs -type f -atime +90 > /tmp/cold_files_90d.txt

# Further categorize by file size.
juicefs find /mnt/jfs -type f -atime +90 -size +1G > /tmp/cold_large_1g.txt
juicefs find /mnt/jfs -type f -atime +90 -size +100M > /tmp/cold_large_100m.txt

Use case

Identify cold data, categorize it by size, and export the results for review before deletion.

Scenario 2: Find large files not accessed for over 30 days (> 1 GiB)

juicefs find /mnt/jfs -type f -atime +30 -size +1G -print

Use case

Large files consume significant space even when few in number — prioritize these for governance.

Scenario 3: Find log files modified within 7 days

juicefs find /mnt/jfs -name "*.log" -mtime -7 -print

Use case

Confirm which logs are still being written to, to avoid accidentally deleting active log files.

Scenario 4: Find empty files or directories

juicefs find /mnt/jfs -empty -print

Use case

Identify empty files/directories left behind after content deletion and reclaim inode resources.

Scenario 5: Aggregate cold data by department

# Aggregate files not accessed for over 180 days by department directory.
juicefs find /mnt/jfs/rd -type f -atime +180 -print
juicefs find /mnt/jfs/marketing -type f -atime +180 -print

# Or filter by file owner.
juicefs find /mnt/jfs -type f -atime +180 -user sales -print

Use case

Aggregate cold data by department or user to assign ownership and facilitate unified governance across application units.

Scenario 6: Batch-cleanup of old temporary files

# Step 1: Preview the temporary files to be deleted.
juicefs find /mnt/jfs -type f -name "*.tmp" -atime +7 -print

# Step 2: Execute deletion after confirming the list.
juicefs find /mnt/jfs -type f -name "*.tmp" -atime +7 -delete

Recommendation

Always preview with -print first and only execute -delete after confirming the results.

Scenario 7: Integration with external systems

# Export in null-delimited format to handle filenames with spaces.
juicefs find /mnt/jfs -type f -atime +365 -print0 > /tmp/cold_files-1y.txt

# Use xargs to batch-compress cold data older than one year (reducing storage costs).
cat /tmp/cold_files-1y.txt | xargs -0 gzip

Use case

Export search results in -print0 format for seamless integration with external tools like xargs for further processing.

Notes

• Run this command only on JuiceFS mount points. It is not intended for non-JuiceFS paths. Always preview the matching files with -print before using -delete.
• Set -maxdepth for large directories to control the scan scope.
• Predicate parsing errors are returned immediately. Start with simple expressions and add conditions incrementally.
• On older metadata service versions or with unsupported predicates, filtering can fall back to the client side.
• Directories without access permissions are silently skipped.
• Errors during traversal or execution are logged.
• atime‑based predicates require the metadata service to have atime recording enabled (disabled by default).
• For large-scale scans, use a dedicated token with a CPU usage limit (supported since v5.3.7, requiring only a metadata service upgrade) to avoid impacting normal application operations.

Troubleshooting

Issue	Possible cause
No output	Check if the path is correct, if conditions are met, or if atime is being recorded (consult the JuiceFS team for verification).
-exec execution failure	Check if the external command exists, arguments are correct, permissions are sufficient.
Error returned after execution	Common with failed delete/exec actions; check the logs for details.

Performance and load

• In practice, performance varies significantly depending on factors such as metadata distribution balance across zones, filter condition complexity, the volume of data returned, and metadata service load. Throughput can range from tens of thousands to hundreds of thousands of inodes per second per zone.
• You can increase scan speed by adjusting concurrency (-p/--threads), but higher concurrency also raises metadata service load. Avoid excessively high concurrency to prevent impacting normal application requests.
• During traversal, archived directories must be decompressed before scanning. Starting with v5.3.12 (metadata service upgrade only; no client upgrade required), scanned directory nodes are re-compressed promptly, greatly reducing the retention time of decompressed data in memory. This significantly reduces memory peaks and eliminates intermittent full-CPU usage caused by GC. Some memory growth will still occur after upgrading, proportional to the concurrency level set by -p/--threads: higher concurrency means more directory nodes in decompressed state simultaneously. On metadata service versions without this optimization, large-scale juicefs find scans may cause significant memory growth and intermittent 100% CPU usage.
• Decompression runs synchronously in the request processing thread and is subject to the requesting token's CPU quota; re-compression runs in the background, is not counted against the token’s CPU quota, and introduces negligible overhead given the small per-node size.

Using a dedicated token to control load

For large‑scale scans, create a dedicated token for the mount point and set a CPU usage limit for the metadata service (available from v5.3.7, requiring only a metadata service upgrade). Combine this with -p/--threads to control concurrency, preventing scans from impacting normal application request processing.