Merge Files
juicefs merge concatenates multiple files into one at the metadata layer. It appends the chunk references of the source files to the destination file, so the source and destination share the same chunk objects. The command does not read or copy object storage data, and it does not delete the source files.
# Merge two files into a new file
juicefs merge --output /mnt/jfs/merged /mnt/jfs/foo /mnt/jfs/bar
# Read source paths from a file, one path per line
juicefs merge --output /mnt/jfs/merged --files-from list.txt
--files-from requires JuiceFS 5.4.1 or later.
On-prem Cross-zone merge is only relevant to on-premises deployments with a multi-zone metadata service. JuiceFS Cloud Service uses a single metadata zone by default, so a merge in the Cloud Service only follows the same-zone path described below.
All source files and the destination must be in the same JuiceFS file system. The destination is created if it does not exist. If it already exists, use --overwrite or --append; otherwise, the command fails.
merge is not atomic. --overwrite removes the destination before merging. For details, see Caveats.
How it works
merge walks the source files in the given order and appends their chunk references to the destination. A source file is not modified or deleted. Deleting a source file after the merge only removes its references; the underlying objects remain until all references are released and the normal trash and garbage collection process finishes.
Because the merged file references the original chunks, its logical size is the sum of the source files, but object storage usage does not grow by the same amount. The merged size is still counted by df, du, and directory quotas.
Files imported from object storage (including import on demand) use the compatible format and cannot be merged, neither as a source nor as a destination. For the same reason, an imported file does not have a chunk table that merge can concatenate.
When new objects are written
In most cases, merge only changes metadata. There are two paths that can write new objects:
| Scenario | Writes new objects? | Notes |
|---|---|---|
| Same zone, or a single-zone file system | No | Only metadata is updated. Compaction may follow if the merge creates slices. |
| Cross-zone, source metadata can be migrated | No | Only metadata is moved to the destination zone; object data is not copied. |
| Cross-zone, source metadata is too large | May write, before merge returns | The client may compact the source file first, and then retry the migration. If it still cannot migrate, merge fails. |
| Chunk boundary misalignment, background compaction runs | Yes, asynchronously | Affected target chunks are rewritten. |
Writing to or truncating the destination after merge | Yes | This is copy-on-write behavior and is unrelated to merge itself. |
Cross-zone metadata migration
In a multi-zone deployment, each file's metadata belongs to a specific zone. When the source and destination files are in different zones, merge must migrate the source file's metadata to the destination file's zone before appending its chunk references.
For a normal source file, this is a metadata-only operation and does not copy object data. If the source file's metadata is too large (for example, the file is very large or has accumulated many slices), the client may compact the source file before retrying the migration. Compaction reads the affected slices, rewrites chunks that contain multiple slices as new chunk objects, and then the client retries. This happens before merge returns and consumes object storage read/write bandwidth.
When a source file must be migrated across zones, the default limit is about 750 GiB per file (based on 64 MiB chunks). If the source file exceeds this limit, merge fails immediately without attempting compaction. Same-zone merges are not subject to this limit, and deployments may adjust it.
If the source file still cannot be migrated after compaction, merge fails. Any new objects written by compaction are not rolled back automatically; they are released by the normal reference-counting and garbage collection process.
Chunk boundary misalignment
A JuiceFS file is divided into 64 MiB logical chunks, and object storage blocks are 4 MiB by default. If the destination file's length is not a multiple of 64 MiB, the next source file starts in the middle of a target chunk. As a result, the source file's chunk boundaries do not align with the target file's chunk boundaries.
For example, if file A is 10 MiB and file B is 130 MiB, B starts at offset 10 MiB of the merged file. The first three target chunks are 64 MiB, 64 MiB, and 12 MiB. Each target chunk now contains data from two adjacent source chunks, so the merged file has multiple slices per chunk.
The metadata service schedules background compaction for affected chunks. Compaction reads the slices of each target chunk, writes a new chunk object, and updates the metadata references. This is why object storage can show writes after a merge, even though merge itself did not copy data.
Compaction works on whole target chunks, not on individual 4 MiB blocks. Even if all slices in a target chunk are aligned to 4 MiB, compaction still rewrites the whole target chunk as a new chunk object. If the target chunk can reference a complete source chunk directly, no new object is created for that chunk.
If file A is a multiple of 64 MiB, B starts at a chunk boundary and this particular merge does not create boundary slices. However, if B already contains multiple slices, background compaction may still process those chunks.
Capacity, quotas, and performance
- Immediately after
merge: Object storage capacity does not change much; only metadata is updated. The merge operation itself is lightweight. - During compaction: Affected chunks are rewritten as new objects, while the old objects remain until their references are released. If the source files are not deleted immediately, the affected range can temporarily use close to twice its logical size.
- After deleting the source files: Old objects are reclaimed according to their references, and usage eventually returns to the logical data size.
mergedoes not permanently consume an extra copy. - Quotas and file system usage: The merged file is counted at its full logical size by
df,du, and directory quotas. - Performance: Background compaction is asynchronous and does not block
mergefrom returning. Cross-zone source compaction happens beforemergereturns and can consume read/write bandwidth synchronously.
Caveats
mergeis not atomic. Sources are submitted in batches. If a later batch fails, earlier batches have already been applied, and the destination may be left partially merged.--overwriteremoves the destination before merging. If the merge fails, the original file cannot be recovered. Merge to a new path first, verify the result, and then replace the original.- All source files and the destination must be in the same JuiceFS file system. Do not use paths from another file system.
- An existing destination requires
--overwriteor--append. Otherwise, the command fails before merging.
Best practices
- If object storage usage does not change after a
merge, the command took the normal metadata-only path. - If you see object storage writes after a
merge, first check whether background compaction is running. It can be delayed and does not necessarily mean thatmergecopied data synchronously. - For capacity-sensitive workloads, merge to a new path first, verify the result, and then delete the source files promptly.
mergeitself shares object data, so object storage usage does not double; only the logical file size and quota are counted twice until the source files are deleted. - Avoid using
--overwriteon the only copy of a file: the destination is removed before the merge starts, andmergeis not atomic. If the merge fails, you can lose the original file. - To reduce boundary slices, make every source file except the last one a multiple of 64 MiB. A source file's own slices can still trigger compaction.
- Schedule large merges during off-peak hours to avoid compaction bandwidth competing with production traffic.
- In a multi-zone deployment, if
mergereturns "not supported," the source file's metadata or chunk count likely exceeds the migration limit. Use JuiceFS 5.3.4 or later to enable automatic compaction and retry, or contact Juicedata engineers.
FAQ
Does merge delete the source files? No. The source files remain unchanged and must be deleted separately.
Why doesn't object storage usage drop immediately after deleting the source files? The source files and the merged file share the same chunk objects. Deleting a source file only removes its references; objects are reclaimed after the trash retention period and background garbage collection.
Why do I see object storage writes after merge? The common reason is chunk boundary misalignment: the merge created slices, and background compaction is rewriting the affected target chunks. Cross-zone metadata migration can also compact a large source file before retrying the migration.
Is compaction guaranteed to happen? No. If there are no slices, background jobs are disabled, or no client is available to run the job, compaction may be delayed or not run. The merged file remains readable.
Can I merge files imported from object storage? No. Imported files use the compatible format and cannot be used as a source or destination for merge.

