Trash

JuiceFS keeps deleted files in trash for a specified period of time before actual deletion from object storage. (Trash files are billable! They are kept in object storage as well.)

You may restore or delete files from trash in JuiceFS Web Console. However, if you have access to a host mount point, it is very easy to observe trash directory by simply navigating to the mount point, and then running ls .trash to see files inside trash. Note that .trash is a virtual directory which does not show even in ls -alh output, thus autocompletion is not available for this directory.

When accessing the JuiceFS host mount point via the command line, you can easily recover deleted files (see Restore data from trash). However, to empty trash and free up capacity to reduce costs, you must log into our web console for security reasons (see Empty trash).

info

Trash not only protect files against accidental deletion, as a matter of fact, all forms of data modification is protected by trash, including file content overwrites. Read trash and file fragments to learn more.

Trash is automatically enabled when users create a file system. Default expiration is 1 day for free plan, 7 days for professional plan. Users may modify this setting in JuiceFS Console.

Move files to trash

Execute the rm command in shell, target files will be moved into the Trash directory. But when it comes to deleting large amount of files, we recommend using juicefs rmr, this command deals with metadata directly (bypassing POSIX API), thus is much faster.

Being moved to Trash, these files will no longer be a part of the original directory, but the actual data is still kept in object storage. If the path of deleted files already exists in Trash, multiple versions will be kept.

Note that Trash does not apply to the following conditions:

• Trash is only valid for files, the directory will be deleted immediately. When restoring files, the original directory structure is restored, but the previous directory attributes cannot be maintained. In particular, if an empty directory is deleted, it cannot be recovered;
• If you delete a snapshot using juicefs snapshot -d <path>, the target will be deleted immediately and will not go to the trash. But when executing rm <path>, the snapshot will enter the trash. Please choose the appropriate deletion method according to your needs;
• Trash does not apply to files created by juicefs import. The metadata of the imported files will be deleted immediately, while the content in object storage is not affected.

Empty trash

Files in trash are still billed by JuiceFS and your object storage provider. To permanently delete files from trash, you must use our web console. This action is not available in a host mount point. Navigate to the trash tab in the file system page and delete files from there.

Once deletion is submitted, the file metadata will be marked as stale at the metadata service and will no longer be billable. However, the object storage blocks will remain and always be scheduled to garbage collection according to the trash expiration time when the files are deleted. Changing the expiration time after deleting files from trash will not affect object storage deletion schedule for files previously deleted.

info

As for on-prem deployments, trash behavior can be tuned to quickly delete object storage data. Contact Juicedata engineers for details.

For files already deleted, chunks in object storage are not deleted strictly according to its expiration time, instead, they'll be schedule for execution in clients, as background jobs. That's why at least one JuiceFS Client must be started (must not disable background job) for trash expiration to function correctly. All JuiceFS Client (FUSE mount, Hadoop Java SDK, and S3 Gateway) behaves roughly the same way in this regard. If no JuiceFS Client is running at all, files in trash will not be expired and garbage collected.

Restore data from trash

Restore data via console

Users can restore files from Trash in JuiceFS Console:

1. Switch to the "Trash" tab in the file system page
2. Select one or more files to be restored
3. Click "Restore" button

Restore data via command line

If you accidentally delete a large directory and there are too many files to be restored, and the console operation is inconvenient, you can also directly operate in the .trash virtual directory in the root directory of the mount point. Assume that the JuiceFS mount point is /jfs, refer to the example below.

tip

The ls output is different for trash, which looks something like:

'66637676|dir1|dir2|file.txt'

The first number is the inode, and the subsequent pipe symbol | is just a display optimization, representing the directory structure. After the recovery operation using mv demonstrated below, the original path and file name will be maintained.

Due to this pipe symbol problem, you should single quote your shell arguments when operating on trash files, or the pipe symbol will be misinterpreted by the shell, causing errors.

# Enter the root path of the mount point of the JuiceFS file system
# Note: It must be the root path of the file system. Subdirectory mount points cannot access the ".trash" virtual directory.
cd /jfs

# Please enter the command completely and do not rely on the shell's auto-completion.
# The ".trash" directory does not appear in the ls output, so autocompletion is not supported.
ls .trash

# List all files that contain xxx in their absolute path
ls '.trash/xxx'

# Recover all files that contain xxx in their absolute path
# Note that the recovery destination must be the root of the mount point and cannot be rewritten to another path.
mv '.trash/xxx' .

# Recover all files that contains tar.gz
mv '.trash/tar.gz' .

If you perform the recovery operation according to the above demonstration, you may encounter the following situations or confusions:

• The destination of the mv command must be the root path of the mount point. This is because JuiceFS only supports restoring files to the original path. Therefore, taking the above demonstration as an example, the destination must be . or /jfs , cannot be rewritten to other paths;
• If you encounter the error cannot move '.trash/xxx' to a subdirectory of itself, you need to consider performing it in another way:
  • Operate data recovery through the console, bypassing the FUSE mount point;
  • Use the find-based demonstration below to process the files that need to be recovered individually to avoid this problem.

If you need to perform recovery using more complicated conditions, use the find command:

# The ctime (change time) stands for deletion time, you can recover files deleted during a period of time.
# The following command means to restore all files with ctime greater than 2023-12-12 17:30:00 and less than 2023-12-13 23:00:00
find .trash -newerct "2023-12-12 17:30:00" ! -newerct "2023-12-13 23:00:00" -exec mv {} . \;

# Recover all tar.gz files
find .trash -name '*.tar.gz' -exec mv {} . \;

# Recover all tar.gz files under mydir
find .trash -name '*|mydir|*.tar.gz' -exec mv {} . \;

# Recover all directory that's named mydir
find .trash -wholename '*|mydir|*' -exec mv {} . \;

# Recover mydir/text.txt
find .trash -wholename '*|mydir|text.txt' -exec mv {} . \;

When recovering files from trash, know that:

• When recovering a large amount of files (over 100 thousands) via command line tools, there's possibility of misses and you should execute the recovery from our web console. If you absolutely need to use command line tools, you should compose the filter condition (demonstrated above in the code snippet) and then run repeatedly to ensure integrity.
• The parent directories of the files will be created automatically and owned by root (similar to sudo mkdir -p), which might be different from the original ownership and permissions and needs manual adjustment.
• If the path of file to be restored already exists, the restore operation will append a sequence number to the file name to avoid name collision, such as the file /a/b/c.txt in trash will be restored to /a/b/c.txt-1.
• If multiple files are selected to restore, they will be restored in the listed order. Files whose path conflict with the ones restored before them will be skipped.