Command Reference
Running juicefs
by itself and it will print all available commands. In addition, you can add -h/--help
flag after each command to get more information, e.g., juicefs mount -h
.
NAME:
juicefs - A POSIX file system built on object storage.
USAGE:
juicefs [global options] command [command options] [arguments...]
VERSION:
5.1.1 (2024-09-20 1faf43b)
COMMANDS:
help, h Shows a list of commands or help for one command
ADMIN:
auth Authorize a volume
destroy Destroy an existing volume
gc Garbage collector of objects in data storage
fsck Check consistency of a volume
dump Dump metadata into a JSON file
load Load metadata from a previously dumped JSON file
version Show version
quota Manage directory quotas
heal Heal a replicated volume
INSPECTOR:
status Show status of a volume
stats Show real time performance statistics of JuiceFS
profile Show profiling of operations completed in JuiceFS
info Show internal information of a path or inode
summary Show data and files distribution of a directory
debug, doctor Collect and display system static and runtime information
lsof List open files
SERVICE:
mount Mount a volume
umount Unmount a volume
gateway Start an S3-compatible gateway
webdav Start a WebDAV server
TOOL:
bench, benchmark Run benchmarks on a path
objbench Run benchmarks on an object storage
warmup Build cache for target directories/files
clone, snapshot Clone a file/directory without copying data
rmr Remove directories recursively
sync Sync between two storages
import Import existing files from object storage
merge merge files together without copying.
grep parallelized file pattern searcher
compact Trigger compaction of chunks
GLOBAL OPTIONS:
--verbose, --debug, -v enable debug log (default: false)
--quiet, -q show warning and errors only (default: false)
--trace enable trace log (default: false)
--no-agent disable pprof (:6060) agent (default: false)
--pyroscope value pyroscope address
--no-color disable colors (default: false)
--help, -h show help
--version, -V print version only (default: false)
Global options
Items | Description |
---|---|
-v --verbose --debug | Enable debug logs. |
-q --quiet | Show only warning and error logs. |
--trace | Enable more detailed debug logs than the --debug option. |
--no-agent | Disable pprof agent. |
--pyroscope | Config Pyroscope address, e.g. http://localhost:4040 . |
--no-color | Disable log color. |
Admin
juicefs auth
Authenticate the client against JuiceFS Console, and fetch or update configuration file for the JuiceFS Volume, and stored in $HOME/.juicefs/$VOL_NAME.conf
. This command is useful when you need to deploy JuiceFS via script, or use self-hosted object storage for onpremise deployment.
Synopsis
juicefs auth NAME [options]
juicefs auth myjfs --token xxx --access-key xxx --secret-key xxx
Options
Items | Description |
---|---|
NAME | Name of your file system. |
--token=TOKEN | Token of your file system, obtain in JuiceFS Web Console. |
--bucket=BUCKET | When a file system is first created, object storage bucket is already specified in the web console. When in need, client can override this setting by specifying this option, in order to for example, connect to object storage in special network environments. Same as the file system bucket setting in the web console, you can either specify the bare bucket name string, or the full endpoint starting with https:// or http:// . These two method have different implications:
|
--access-key=ACCESSKEY | Object storage access key, see How to get AK/SK. An interactive prompt will occur to collect this info if missing in the command line options. Leave empty if you have configured host to access object storage without credentials. |
--secret-key=SECRETKEY | Object storage secret key, see How to get AK/SK. An interactive prompt will occur to collect this info if missing in the command line options. Leave empty if host have been configured to access object storage without credentials. |
--session-token=SESSIONTOKEN | Optionally provide object storage session token. |
--storage-class=value | Storage class for the destination storage, if supported. Read relevant docs for more, e.g. S3. |
--bucket2=BUCKET2 | Optionally provide the name or endpoint for the secondary bucket For Data replication, same format and caveats as --bucket . |
--access-key2=ACCESSKEY2 | When data replication is enabled, use this parameter to specify the access key for the secondary bucket. Learn more at Cross-region data replication. |
--secret-key2=SECRETKEY2 | When data replication is enabled, use this parameter to specify the secret key for the secondary bucket. Learn more at Cross-region data replication. |
--session-token2=SESSION_TOKEN2 | the session token to access the bucket (optional) |
--passphrase=PASSPHRASE | Passphrase for encrypted key, see Data Encryption for more. |
--encrypt-keys | Encrypt the object access and secret keys locally or not (default: false). |
--subdir=SUBDIR | Specify a sub-directory for mount, you can also create ACL to restrict client access to specific sub-directories, learn more at Client Access Control |
--conf-dir=CONFDIR | Directory to store configuration files, default to ~/.juicefs ) |
--no-update | Client will by default obtain latest config from JuiceFS Web Console, set to true to disable this behavior, and then client will no longer talk to JuiceFS Console when mounting. |
juicefs gc
Deal with leaked objects, and garbage fragments produced by file overwrites.
Synopsis
juicefs gc [command options] VOLUME-NAME
# Check and print results, but don't actually delete anything
juicefs gc $VOL_NAME
# Delete leaked objects
juicefs gc $VOL_NAME --delete
Options
Items | Description |
---|---|
--delete | delete leaked objects (default: false) |
--threads=10, -p 10 | Number of concurrent threads, default to 10. |
juicefs fsck
Check consistency of file system.
Synopsis
juicefs fsck [command options] VOLUME-NAME
juicefs fsck $VOL_NAME
Options
Items | Description |
---|---|
--threads=10, -p 10 | Number of concurrent threads, default to 10. |
--repair | Fix the prefix of objects. |
juicefs dump
Dump metadata into a JSON file. Different from JuiceFS CE, JuiceFS EE require no manual metadata dump in order to make a backup, it's already handled by our engineers in the server end. Thus, this command is used mainly for debugging, or migration between CE and EE.
Synopsis
juicefs dump [command options] PATH JSON-PATH
# Export metadata to meta-dump.json
juicefs dump /jfs meta-dump.json
# Export metadata for only one subdirectory of the file system
juicefs dump /jfs/subdir meta-dump.json
Options
Items | Description |
---|---|
PATH | Mount point path. |
JSON-PATH | Export file path. If the filename ends with .gz , it will be automatically compressed. |
--gzip | Compress with gzip. |
juicefs load
Load metadata into an empty file system, often used for migrations between JuiceFS CE and EE.
Synopsis
juicefs load [command options] PATH [FILE]
# Import the metadata backup file into the file system
juicefs load /mnt/jfs meta-dump.json.gz
juicefs version
Show the JuiceFS version information. The update details of each version can be viewed on the "Release Notes".
Synopsis
juicefs version [options]
# Upgrade client version and seamless restart
juicefs version -u -r
Options
Items | Description |
---|---|
-u --upgrade | Upgrade the JuiceFS client to latest version. |
-r --restart | Execute seamless remount, if client is upgraded. |
juicefs quota
Manage directory quota. Different from Community Edition, JuiceFS Enterprise Edition manages quota directly from the web console, and users should not directly run this command from the terminal. The purpose of this command is to stay consistent with CE interface, so that CSI Driver may approach quota management using the same code.
juicefs heal
Repair data when using data replication. Execute a bidirectional synchronization on object storage for both file system.
Synopsis
juicefs heal VOL-NAME [options]
Option
Items | Description |
---|---|
--start=value | Specify object storage key range. |
--upload-limit=0 | Maximum bandwidth for upload in Mbps, default to 0 (unlimited). |
--conf-dir=/root/.juicefs | a directory for configuration (default: /root/.juicefs ) |
Inspector
juicefs status
Show status of JuiceFS.
Synopsis
juicefs status [command options] VOLUME-NAME
juicefs status $VOL_NAME
Options
Items | Description |
---|---|
--session=0 -s 0 | Show detailed information of one session, specified using a session ID (get from the SID field from the status output). |
juicefs stats
The stats
command reads the metrics of the JuiceFS client and prints the changes of each metric per second.
For more introduction and usage of this command, see "Real-time Performance Monitoring".
Synopsis
juicefs stats [options] MOUNTPOINT
Options
Items | Description |
---|---|
--schema=ufmco | Schema string that controls the output, default to “ufmco”, its meaning: u: usage f: FUSE m: meta c: blockcache o: object g: go |
--interval=1 | Update interval in seconds, default to 1. |
--verbosity=0 | Verbosity level, increase to see more information. |
juicefs profile
Performs basic profiling based on access log, and writes comprehensive statistics to command line output. Use this command to quickly identify the overall file system access pattern of your application, and assess performance.
For more introduction and usage of this command, see "Real-time Performance Monitoring".
Synopsis
juicefs profile [options]
Options
Items | Description |
---|---|
-x PATH --path=PATH | The directory path of access log, default to /jfs . |
-f FILE --file=FILE | The filename of the access log, default to .ophistory . Refer to Access Log for different types of access log. |
-g GROUP_BY --group-by=GROUP_BY | Group the output by a specific attribute, default to cmd , choose from uid , pid , cmd . |
-s SORT_BY --sort-by=SORT_BY | Sort the output by a specific column, default to total_time , choose from group , number , avg_time , total_time . |
-u FILTER_BY_UID --filter-by-uid=FILTER_BY_UID | Filter the output by one or multiple keywords on the UID column, please separate multi keywords by comma (,). |
-p FILTER_BY_PID --filter-by-pid=FILTER_BY_PID | Filter the output by one or multiple keywords on the PID column, please separate multi keywords by comma (,). |
-w WINDOW_SIZE --window-size=WINDOW_SIZE | Analysis time window, accept float value, default to 60 seconds. |
-i FLUSH_INTERVAL --flush-interval=FLUSH_INTERVAL | Flush output interval, accept float value, default to 2 seconds. |
juicefs info
Show internal information of a path or inode.
Synopsis
juicefs info [command options] PATH/INODE
# Lookup single file
juicefs info /mnt/jfs/foo
# Lookup directory
juicefs info -r /mnt/jfs/bar/
# Lookup file information through inode
cd /mnt/jfs
juicefs info -i 100
Options
Items | Description |
---|---|
PATH/INODE | Target file path or inode. |
-i --inode | Lookup file information through inode. Note that the current directory must be in the JuiceFS mount point. |
-r --recursive | Recursively obtain information about all subdirectories and files in a directory. |
--raw | Show internal raw information. |
juicefs summary
Show tree summary of target directory.
Synopsis
juicefs summary [command options] PATH
# Show with path
juicefs summary /jfs/foo
# Show max depth of 5
juicefs summary --depth 5 /jfs/foo
# Show top 20 entries
juicefs summary --entries 20 /jfs/foo
Options
Items | Description |
---|---|
--depth=2 -d 2 | Depth of tree to show, 0 means only show root, defaults to 2. |
--entries=10 -e 10 | Show top N entries, sort by size, defaults to 10. |
--csv | Print summary in csv format. |
juicefs doctor
Collect abundant debug information, including access log, client log, CPU stats, Goroutine stacktraces. When run into unexpected situations, JuiceFS team may asks you to collect information with this command, and send back the result to continue support.
We encourage you to also take a look at Troubleshooting Methods and Troubleshooting Cases, and try to solve some of the more frequently encountered problems by yourself.
Synopsis
juicefs doctor MOUNTPOINT
juicefs lsof
List recently (within 10 minutes) opened files in JuiceFS.
Synopsis
juicefs lsof PATH
Service
juicefs mount
Mount the file system. You must provide the token of the file system (check the "File System Settings" in the website console), and the key pair to access the object storage (refer to How to Get Object Storage Access Key and Secret Key for details).
If the configuration file is not generated by the auth
subcommand before mounting the file system, the client will ask you to provide the file system token and the key pair to access the object storage API separately through command line interaction.
You can mount JuiceFS as root or normal user, but due to permission issues, cache and client log locations will be different, read option descriptions below for more.
Synopsis
juicefs mount NAME MOUNTPOINT [options]
Options
Items | Description |
---|---|
NAME | The name of your file system. |
MOUNTPOINT | The mount point for your file system. |
--background -b -d | Run in the background mode, which is the default behavior. |
--foreground -f | Run in the foreground mode. Client logs will print directly to the console, often used in debugging. |
--no-syslog | Disable syslog. |
--log=PATH | Log file location, defaults to /var/log/juicefs.log (when running as root) or ~/.juicefs/log/juicefs.log (when running as normal user). |
--update-fstab | add or update entry in /etc/fstab, when applicable, see Auto-mount on boot. |
--max-space | Set a maximum space limit in GiB. When you use PAYG plans, there is no real capacity limit. You may set a limit using this argument to display a sensible value in df -h output. |
--prefix-internal | Add a .jfs prefix to all internal files, for example, .jfs.stats , .jfs.config . |
--hide-internal | Hide all internal files (for example, .accesslog , .stats ). |
--sort-dir | Sort entries within a directory by name. This does not affect the output of ls and other similar tools, as the tools perform sorting themselves. This feature is intended for systems like nfsd which paginate readdir results. JuiceFS by default does not guarantee the order of directory entries. Enabling this flag to sort by name ensures that the JuiceFS client can work with nfsd. |
--token=TOKEN | Specify client access token. |
--no-update | Client will by default obtain latest config from JuiceFS Web Console, set to true to disable this behavior, and then client will no longer talk to JuiceFS Console when mounting. |
FUSE options
Items | Description |
---|---|
--enable-xattr | Enable extended attributes (xattr) support, default to false, which is recommended for most scenarios because xattr comes with its own overhead. |
--enable-acl | Enable POSIX ACL support, see Extended ACL. |
--no-bsd-lock | Disable BSD lock support. |
--no-posix-lock | Disable POSIX lock support. |
--block-interrupt=1 | Set the number of seconds to block interruptions (default: 1). |
--readdir-cache | Set readdir caching in the kernel (requires Linux kernel 4.20+). Default is false. |
--allow-other | Allow access from other users. This option is enabled by default for root users. You need to enable this feature explicitly otherwise. |
--max-write Added in v5.1.0 | Set the maximum size for FUSE write requests (requires kernel 4.20+; capped at 1 MiB). Default is "128K". If there is no other bottleneck, increasing this value can increase large file write performance but may affect random write performance. Use cautiously. |
-o FUSE_OPTS | Other FUSE options. For example, enabling FUSE write cache through -o writeback_cache can greatly improve the performance of random writes and fragmented writes (requires Linux Kernel 3.15 or above). |
Metadata options
Items | Description |
---|---|
--subdir=SUBDIR | Specify a sub-directory for mount, you can also create ACL to restrict client access to specific sub-directories, learn more at Client Access Control |
Metadata cache options
Items | Description |
---|---|
--metacacheto=METACACHETO | Metadata cache timeout. Use 0 to disable (default: "5m"). |
--max-cached-inodes=5000000 | The maximum number of cached inodes (default: "5000000"). |
--open-cache --opencache | Open files using cached metadata, default to false, which means every open operation access metadata service to check for file changes. If file is frequently opened and scarcely changed, consider enabling --opencache to further improve read performance. For the same mount point, open cache is automatically invalidated upon file modifications. |
--attr-cache --attrcacheto=1 | File attribute cache timeout in seconds (default: "1.0s"). |
--entry-cache --entrycacheto=1 | File entry cache timeout in seconds (default: "1.0s"). |
--dir-entry-cache --direntrycacheto=1 | Directory entry cache timeout in seconds (default: "1.0s"). |
Object storage options
Items | Description |
---|---|
--get-timeout=60 | Max number of seconds to download an object, default to 60, may need to increase this in low bandwidth conditions, see Read / Write error. |
--put-timeout=60 | Max number of seconds to upload an object, default to 60, may need to increase this in low bandwidth conditions, see Read / Write error. |
--ioretries=30 --io-retries=30 | (default: 30) number of retries for network failure. Wait between retries increase linearly, but when set to over 30, wait becomes a fixed 10 seconds delay. The actual total timeout depends on specific timeout settings for each type of requests. |
--object-clients=1 Added in v5.1.0 | The number of object storage clients (default: 1). For some storage system SDKs (like Ceph), the client has a hard-coded bandwidth limit. This may cap the JuiceFS client's maximum throughput. If you cannot fully utilize host bandwidth even with a large concurrency, try to increase this value to use more object storage clients to see if you can boost performance. |
--max-uploads=20 | Upload concurrency, defaults to 20. This is already a reasonably high value for 4MiB writes, with such write pattern, increasing upload concurrency usually demands higher --buffer-size , learn more at Read/Write Buffer. But for random writes around 100K, 20 might not be enough and can cause congestion at high load, consider using a larger upload concurrency, or try to consolidate small writes in the application end. |
--max-downloads=200 | Max number of concurrent downloads, default to 200, which is usually large enough for clients and does not require further adjustments. This option controls client global download concurrency, but for single file read, a hard limit of 128 blocks is imposed to avoid uncontrolled resource usage in large file sequential scenarios. |
--max-deletes=-1 --delete-limit=-1 | Maximum QPS for deletion, default to -1 (unlimited), set to 0 to disable deletion. Use this if for example, some object storage limit API calls. |
--flush-wait=5s | The time delay between automatic flush to object storage, default to 5 seconds. When slowly writing a big file, frequent flushes will cause fragmentation, consider increasing this delay to avoid write amplification. |
--upload-limit=0 | Maximum bandwidth for upload in Mbps, default to 0 (unlimited). Apart from bandwidth control, this parameter can also be used for system load control with high throughput scenarios. |
--download-limit=0 | Maximum bandwidth for download in Mbps, default to 0 (unlimited). Apart from bandwidth control, this parameter can also be used for system load control with high throughput scenarios. |
--external | Prefer to use the external entrypoint of the object storage service. This means the traffic of JuiceFS will go through the public network instead of the internal network. |
--internal | Prefer to use the internal entrypoint of the object storage service. This means the traffic of JuiceFS will go through the internal network instead of the public network. (Applicable to some object storages that separate internal and external network domain names, e.g. Aliyun OSS, UCloud UFile, etc.) |
--rsa-key=PATH | Path to RSA private key (PEM). Passphrase will be asked or provided with JFS_RSA_PASSPHRASE environment variable. |
--flip | When data replication is enabled, use secondary bucket as primary bucket. Learn more at Cross-region data replication. |
Cache options
Read Cache to understand JuiceFS's cache design.
Items | Description |
---|---|
--buffer-size=300 | Total size of memory used for read/write buffering in MiB default to 300, learn more at Read/Write Buffer |
--prefetch=1 | When a block is randomly read by a small offset range, the whole block is scheduled for download asynchronously, this is called prefetch in JuiceFS. This options controls prefetch concurrency, which defaults to 1, and can be disabled by setting to 0. Note that:
|
--initial-readahead | The initial readahead for sequential reads in MiB. Use a larger value to boost large file sequential read performance. If this option is not specified, the default readahead window is 4096 bytes. |
--writeback | Enable client write cache, data will be written to local disk first, and then uploaded asynchronously in the background, default to false. Client write cache brings data security risks, use with caution. |
--cache-dir=CACHEDIR | Cache directory, defaults to /var/jfsCache or $HOME/.juicefs/cache depending on the current user.Separate multiple directories by : , add quotation mark if using wildcard match (* ). e.g. --cache-dir '/data*/jfsCache:/mydata*/jfsCache' .Moreover, you can specify /dev/shm to use the memory device (defaults to half of node memory), or use the string memory to use client process memory. |
--cache-size=102400 | Local cache size limit in MiB. The default value is 102400 (100GiB). When multiple directories are used as the cache directory, --cache-size limits the total size sum (split evenly).You can set it to 0 to disable disk cache altogether, but this causes an extra 100MB memory overhead to grant the client a minimum cache space. For more details, read troubleshooting resource usage. Since 5.1, you can set this value to -1 to use the disk size as cache size. |
--free-space-ratio=0.1 | Minimum free space ratio on cache disk, default to 0.1 (0.2 for 4.9 and older versions). This option doesn't support being set to 0, to achieve maximum usage, you can set a value close to 0 (such as 0.01). Also if Client write cache is enabled, this option also controls write cache size |
--cache-mode=0600 | File permission for cached blocks, default to 0600. |
--cache-partial-only | For the read cache, only data blocks smaller than one block size (default 4MiB) are cached, such as small files smaller than one block size, and data blocks smaller than one block size at the end of large files. The default value is false (that is, all read data blocks are cached).Note that this option will also affect the cache construction of Distributed Cache. Read client read cache to learn more. |
--cache-large-write Added in v5.1.0 | Cache full blocks from write. By default, JuiceFS does not cache large sequential writes (they were not read in most scenarios). You can enable this flag to cache them when needed. |
--verify-cache-checksum=extend | Checksum level for cache data. After enabled, checksum will be calculated on divided parts of the cache blocks and stored on disks, which are used for verification during reads. The following strategies are supported:
|
--cache-eviction=2-random Added in v5.0.1 | Cache eviction algorithm, supports the following strategies:
|
--cache-expire=0s Added in v5.1.0 | Cached blocks not accessed for longer than the specified duration will be automatically evicted (0 means never; default: "0s"). |
Distributed cache options
Read distributed cache for more.
Items | Description |
---|---|
--cache-group=CACHEGROUP | Cache group name for distributed cache. Nodes within the same group share cache, disabled by default. |
--subgroups=0 | the number of sub-groups in a cache group (default: 0) |
--group-network | Network interface for distributed cache. Default listening IP is the first private IP found in the network device list, if this isn't appropriate for you, use this option to specify a fixed network interface. |
--group-ip=GROUPIP | Manually set IP address listened by distributed cache, in hosts with multiple network interfaces, use this option to specify the correct NIC. Can also use CIDR prefix to automatically select IP, e.g. use 172.16.0.0 to match with 172.16.0.0/16 . |
--group-port | Listen port for distributed cache, defaults to a large random port. If the specified port doesn't work, will automatically increase by 1 until a successful bind. |
--group-weight=100 | When using hosts with different disk sizes, use this setting to adjust group member weight. Recommended value range is 10 to 1000, that's to say for example, a cache group consists of a 1TB host and a 3TB host, you can use 100 weight for the 1TB host and 300 for the 3TB host. |
--group-weight-unit=0 | If you prefer not to manually set --group-weight and instead want it to be automatically derived from --cache-size , you can specify this parameter. The unit is MiB. For example, with the default --cache-size=102400 and --group-weight-unit=1024 , where the latter means "1 weight equals 1024 MiB," the final weight will be 102400 / 1024 = 100 . |
--group-backup | Enable the group backup feature. When enabled, if a node's local cache is not hit, it forwards the request to other nodes. Once the request is fulfilled, both the original and the responding nodes attempt to cache the data locally. For more details on the principle and use cases, see Group backup |
--no-sharing | When inside a cache group, only fetch cache data from others, but never share its own cache. Use this option on ephemeral mount points (like Kubernetes Pod). Default to false. |
--fill-group-cache | Send written data blocks to the cache group as much as possible (100% caching is not guaranteed) and no longer write to the local cache. When used with --no-sharing , clients will only contribute writes smaller than block size (4MiB by default) to the cache group. Since 4.9.22, if client write cache is enabled, the staging data will be sent asynchronously to the cache group as well, no matter its size, so that cache group members have a good chance of reading data that's not yet persisted. |
--cache-group-size=0 | Unit is MiB. JuiceFS Client performs readahead and prefetch, so for files smaller than this size, client will try to schedule all its data blocks into a single node, to maximize cache utilization. |
--cache-priority=0 Added in v5.0.14 | The priority of the cache block. The available values are: 0, 1, 2, and 3. The larger the number, the higher the priority. When cache is evicted, data with lower priority will be evicted first. |
--remote-timeout=65s Added in v5.1.0 | Timeout duration for cache group requests (default: 65 seconds). |
--group-compress Added in v5.1.0 | Compress the network traffic between peers, enabled by default. You can disable it to save some CPU overhead when compression ratio for your data is low. |
Experimental options
These options are currently experimental and subject to change in the future. Incorrect usage may pose data integrity risks, if you do need to run this command, seek help from a Juicedata engineer.
Items | Description |
---|---|
--min-inodes-diff=1048576 | When the inodes difference between the current zone and the zone with the least inodes exceeds this value, the metadata will be automatically balanced. Defaults to 1048576. |
--min-dir-inodes=1048576 | When the number of directories in the current path exceeds this value, the metadata is automatically balanced. Defaults to 1048576. |
--max-space=0 | Limit the displayed total space in GiB, defaults to 0, meaning that the true total capacity is displayed. |
juicefs umount
Umount a JuiceFS file system. This command handles the platform differences, i.e. umount
in Linux and diskutil unmount
in macOS. Note that there's a similar hidden command called juicefs unmount
, this is specially reserved for Kubernetes FlexVolume and not for normal use.
If you encounter any errors while unmounting JuiceFS, read the docs for troubleshooting steps.
Synopsis
juicefs umount PATH [options]
Options
Items | Description |
---|---|
PATH | The mount point for your file system. |
-f --force | Umount it forcibly. |
juicefs gateway
Start an S3 gateway, read S3 gateway for more.
Synopsis
juicefs gateway [command options] VOLUME-NAME ADDRESS
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=12345678
juicefs gateway testVolume localhost:9000
Options
Apart from options listed below, this command shares options with juicefs mount
, be sure to refer to mount
as well.
Flag | Description |
---|---|
--console-url=URL | Web URL address, only needed in on-premise environments. |
--mountpoint=s3gateway | Specify the JuiceFS mount point on host, needed only for absolute symlink resolution, defaults to s3gateway . See S3 Gateway and symbolic link. |
--access-log=PATH | Path for file system access log, default location is the .accesslog file under the root mount point. |
--no-banner | Omit MinIO startup messages. |
--multi-buckets | Use top level directories as buckets, default to false. |
--keep-etag | Keep object ETag when uploading. |
--umask="022" | Umask used for new files, default to 022 . |
juicefs webdav
Start a WebDAV server to access JuiceFS file system.
Use the environment variable WEBDAV_USERNAME
and WEBDAV_PASSWORD
to set the username and password for WebDAV server.
Synopsis
juicefs webdav [command options] NAME ADDRESS
# Set username and password
export WEBDAV_USER=root
export WEBDAV_PASSWORD=1234
# Start WebDAV server on port 9007
juicefs webdav myjfs localhost:9007
Options
Apart from options listed below, this command shares options with juicefs mount
, be sure to refer to mount
as well.
Items | Description |
---|---|
--cert-file | certificate file for TLS |
--key-file | key file for TLS |
Tool
juicefs benchmark
Run single benchmark, including read/write/stat big/small files.
Synopsis
juicefs benchmark [options] path-to-test
Options
Items | Description |
---|---|
--dest=/jfs/benchmark | Temporary directory for benchmark test files, default to /jfs/benchmark . |
--block-size=BLOCK_SIZE | (default: 1MiB) block size in MiB |
--bigfile-file-size=BIGFILE_FILE_SIZE | (default: 1024MiB) size of big file in MiB |
--smallfile-file-size=SMALLFILE_FILE_SIZE | (default: 0.1MiB) size of small file in MiB |
--smallfile-count=100 | (default: 100) number of small files |
juicefs objbench
Run basic benchmarks on the target object storage to test if it works as expected.
Synopsis
juicefs objbench [command options] BUCKET
Options
Items | Description |
---|---|
--storage=file | Object storage type (e.g. s3 , gcs , oss , cos ) (default: file ) |
--access-key=value | Access Key for object storage (can also be set via the environment variable ACCESS_KEY ) |
--secret-key value | Secret Key for object storage (can also be set via the environment variable SECRET_KEY ) |
--session-token=value | Session token for object storage |
--block-size=4096 | size of each I/O block in KiB (default: 4096) |
--big-object-size=1024 | size of each big object in MiB (default: 1024) |
--small-object-size=128 | size of each small object in KiB (default: 128) |
--small-objects=100 | number of small objects (default: 100) |
--skip-functional-tests | skip functional tests (default: false) |
--threads=4, -p 4 | number of concurrent threads (default: 4) |
juicefs warmup
Download data to local cache in advance, to achieve better performance on application's first read. You can specify a mount point path to recursively warm-up all files under this path. You can also specify a file through the --file
option to only warm-up the files contained in it.
If the files needing warming up resides in many different directories, you should specify their names in a text file, and pass to the warmup
command using the --file
option, allowing juicefs warmup
to download concurrently, which is significantly faster than calling juicefs warmup
multiple times, each with a single file.
The warmup
command will not repeatedly download files that already exist locally (it will be skipped directly), so the displayed download speed may be higher than the actual download traffic. If the distributed cache is enabled, executing this command on any node in the cache group can achieve the same effect.
Synopsis
juicefs warmup [PATH] [options]
# Warm up all files in datadir
juicefs warmup /mnt/jfs/datadir
# Warm up selected files
echo '/jfs/f1
/jfs/f2
/jfs/f3' > /tmp/filelist.txt
juicefs warmup -f /tmp/filelist.txt
Options
Items | Description |
---|---|
--file=FILE -f FILE --listfile=FILE | file containing a list of paths (one path per line) |
--threads=50 -p 50 -c 50 --concurrent=50 | Download concurrency, default to 50. Increase concurrency to speed up, but if the download bandwidth of the object storage is low, reduce concurrency to avoid download timeout errors. If you're experience warmup slowness in distributed cache cluster, try increasing this value starting from 20 * [number of cache group members] and find a suitable concurrency. |
--cache-priority=0 Added in v5.0.14 | The priority of the cache block. The available values are: 0, 1, 2, and 3. The larger the number, the higher the priority. When cache is evicted, data with lower priority will be evicted first. |
--io-retries=1 | Max number of allowed retries, default to 1. Increase this value in poor network conditions to better handle failures. |
--max-failure=0 | Controls the command exit code according to the number of download failures, the number of allowed failures is default to 0, meaning that upon any data block warmup failure, command exits with error (exit code 1). When faced with warmup errors, check client logs for troubleshooting. |
--background -b | Run in the background. |
--evict | Evict cached blocks for given path. |
--check | check whether the data blocks for the given path are cached or not. |
juicefs snapshot
Also called juicefs clone
. This command takes a snapshot of your data by creating a mere metadata copy, without creating any new data in the object storage, thus snapshotting is very fast regardless of target file / directory size. Under JuiceFS, this command is a better alternative to cp
, moreover, for Linux clients using kernels with copy_file_range
support, then the cp
command achieves the same result as juicefs snapshot
.
Snapshot result is a metadata copy, all the files are still referencing the same object storage blocks, that's why a snapshot behaves the same in every way as its originals. When either of them go through actual file data modification, the affected data blocks will be copied on write, and become new blocks after write, while the unchanged part of the files remains the same, still referencing the original blocks.
Although object storage data is not actually copied, snapshots takes up storage space, and will result in an increase in bill, pay special attention when taking snapshots on large size directories.
Synopsis
juicefs snapshot SRC DST [-f|--force] [-c|--copy] # create a snapshot
juicefs snapshot -d DST [-f|--force] # delete a snapshot
Options
Items | Description |
---|---|
SRC | The path to the file you are creating the snapshot from. |
DST | The locate where you want to store your snapshot. |
-d --delete | Remove the snapshot. Only use this command on snapshots, deleting normal files using this command will result in permission errors. Deleted snapshots will bypass Trash completely. If you want to preserve the deleted snapshots in Trash like normal files, use juicefs rmr or simply rm -r . |
-f --force | Overwrite existing snapshot, or remove the snapshot arbitrarily. |
-c --copy | Use current user's UID, GID and umask to create the snapshot. Otherwise it will use permission information of the file you are creating snapshot. |
juicefs rmr
Remove all the files and subdirectories, similar to rm -rf
, except this command deals with metadata directly (bypassing POSIX API), thus is much faster.
It will try to remove them using current UID/GID, ignore those that can't be removed, return the number of removed files/directories, the number of left ones.
If trash is enabled, deleted files are moved into trash. Files inside trash is still billable, read more at trash.
Synopsis
juicefs rmr DIR
Options
Items | Description |
---|---|
DIR | Files to delete. |
--skip-trash | Skip the trash and delete directly, used when in urgent need to release object storage capacity. Use cautiously. |
juicefs sync
Sync data between two storage systems, read Data Migration for more.
Synopsis
juicefs sync [command options] SRC DST
# Sync object from OSS to S3
juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com
# Sync objects from S3 to JuiceFS
juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://VOL_NAME/
# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: aaa/b1
juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://VOL_NAME/
# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: a1/b1,aaa/b1
juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://VOL_NAME/
# SRC: a1/b1,a2/b2,aaa/b1,b1,b2 DST: empty sync result: a1/b1,b2
juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://VOL_NAME/
Both SRC
and DST
takes up the format [NAME://][ACCESS_KEY:SECRET_KEY[:SESSIONTOKEN]@]BUCKET[.ENDPOINT][/PREFIX]
, in which:
NAME
: Can be all JuiceFS supported storage, e.g.s3
,oss
, evenjfs
. Usingjfs://
allows you to directly transfer data from or into JuiceFS, bypassing the FUSE mount point, this saves resource under large scale data transfer.ACCESS_KEY
andSECRET_KEY
: Object storage credentials, if special characters are included, use single quote to avoid shell interpretations.SESSIONTOKEN
: Optionally provide session token for the object storage.BUCKET[.ENDPOINT]
: Object storage endpoint, note that currently MinIO only supports path style (minio://[ACCESS_KEY:SECRET_KEY[:TOKEN]@]ENDPOINT/BUCKET[/PREFIX]
).[/PREFIX]
: Optionally provide the prefix of the files to synchronize.
Apart from options listed below, this command shares options with juicefs mount
, be sure to refer to mount
as well.
Do notice that many of these shared options works exclusively with jfs
scheme, for example, the sync
command supports --writeback
as well, but if the sync destination is a normal JuiceFS FUSE mount point, it won't automatically convert itself to a --writeback
mount point simply because you are running a sync
job with --writeback
. To sum up, --writeback
only works when sync
destination is a jfs
scheme address, and this is also true for many other options that's shared with mount
.
Selection related options
Items | Description |
---|---|
--start=KEY, -s KEY, --end=KEY, -e KEY | Provide object storage key range for syncing. |
--exclude=PATTERN | Exclude keys matching PATTERN. |
--include=PATTERN | Include keys matching PATTERN, need to be used with --exclude . |
--limit=-1 | Limit the number of objects that will be processed, default to -1 which means unlimited. |
--update, -u | Update existing files if the source files' mtime is newer, default to false. |
--force-update, -f | Always update existing file, default to false. |
--existing, --ignore-non-existing | Skip creating new files on destination, default to false. |
--ignore-existing | Skip updating files that already exist on destination, default to false. |
Action related options
Items | Description |
---|---|
--dirs | Sync empty directories as well. |
--perms | Preserve permissions, default to false. |
--inplace | Modify files in place, rather than delete and replace. If destination storage system is a JuiceFS volume with Trash enabled, overwritten files by default will be kept in Trash. Use --inplace to modify files in place and prevent Trash usage. Currently only jfs:// type destination storage is supported. |
--links, -l | Copy symlinks as symlinks default to false. |
--delete-src, --deleteSrc | Delete objects that already exist in destination. Different from rsync, files won't be deleted at the first run, instead they will be deleted at the next run, after files are successfully copied to the destination. |
--delete-dst, --deleteDst | Delete extraneous objects from destination. |
--check-all | Verify the integrity of all files in source and destination, default to false. Comparison is done on byte streams, which comes at a performance cost. |
--check-new | Verify the integrity of newly copied files, default to false. Comparison is done on byte streams, which comes at a performance cost. |
--dry | Don't actually copy any file. |
--http-port=6070 | HTTP port for the pprof server to listen on, default to 6070. |
Storage related options
Items | Description |
---|---|
--threads=10, -p 10 | Number of concurrent threads, default to 10. Higher concurrency requires more resource, particularly in large file scenarios, because more big files will be loaded into the process memory. |
--list-threads=1 | Number of list threads, default to 1. Read concurrent list to learn its usage. |
--list-depth=1 | Depth of concurrent list operation, default to 1. Read concurrent list to learn its usage. |
--no-https | Do not use HTTPS, default to false. |
--storage-class=value | Storage class for the destination storage, if supported. Read relevant docs for more, e.g. S3. |
--bwlimit=0 | Limit bandwidth in Mbps default to 0 which means unlimited. |
Cluster related options
Items | Description |
---|---|
--manager=ADDR | Manager node address used in distributed syncing, this is an internal option that's used in the executed command on the worker nodes. |
--worker=ADDR,ADDR | Worker node addresses used in distributed syncing, comma separated. |
Metrics related options
项 | 说明 |
---|---|
--metrics | Address to export metrics, defaults to 127.0.0.1:9567 . |
--consul | Consul address to register, defaults to 127.0.0.1:8500 . |
juicefs import
juicefs import
scans the target object storage bucket, collects metadata of the existing files, and saves to the JuiceFS metadata engine. This gives JuiceFS access to the existing object storage files, but the file data will remain in the object storage as is. To learn more, see Import existing object storage files.
Synopsis
Caching isn't supported for external buckets. This means in order to have caching support, the JuiceFS volume and import source must be the same bucket, and execute the import command in the following format:
# URI doesn't include bucket name, caching is supported
juicefs import / /jfs/imported
juicefs import /prefix /jfs
# If URI contains bucket, caching is no longer supported
# In the below example, even if BUCKET is the same as the JuiceFS volume bucket, caching will not be available
juicefs import BUCKET/prefix /jfs
juicefs import URI DST [-v] [--name=<name>] [--mode=<mode>]
# Assuming the mount point being /jfs, and importing from the same bucket as the file system, in such case, the bucket name can be omitted
# But remember to use a prefix, to avoid importing data created by JuiceFS itself (the chunks directory), into JuiceFS, as it'll be meaningless inside JuiceFS
juicefs import /prefix /jfs/imported
# Assuming the bucket is in the same region as the file system
# Then the endpoint can be omitted to just the bucket name
juicefs import another-bucket/prefix /jfs/imported
# If the bucket is located in another region, the full endpoint must be specified
juicefs import other-bucket.s3.cn-northwest-1.amazonaws.com.cn /jfs
# Import files in mybucket by a prefix, into the specified directory within JuiceFS
juicefs import other-bucket/prefix /jfs/oss-imported
Options
Items | Description |
---|---|
URI | The object storage URI of the target, format is the BUCKET[.ENDPOINT][/PREFIX] . Must use the same type of object storage as the file system itself, and should be accessible with the same set of credentials (access key, secret key), you cannot specify separate keys within this URI. |
DST | Import destination, must be a JuiceFS mount point, and the file system must be using the same object storage as the URI . |
--mode=0440 | Mode (Unix permissions) of imported files. |
--threads=50 | Number of workers. If the object storage list produces too many items, increase this value to write to metadata faster. |
--list-threads=10 | Number of list threads. The default value is 10 . This is similar to concurrent list . |
--list-depth=2 | Depth of concurrent list operations. The default value is 1 . This is similar to concurrent list . |
--update | Only import newer files (compared using atime ). |
--force-update | Import and overwrite all different files. When some storage systems do not support automatic atime update, use this option to update the import. |
juicefs grep
Parallelized file pattern searcher.
Synopsis
juicefs grep [PATTERN] PATH ... [options]
Options
Items | Description |
---|---|
[PATTERN] | Support simple pattern and basic regular expression. |
PATH | Search path and files. |
-j JOBS --jobs=JOBS | Number of parallel jobs (default is number of CPU cores). |
-e PATTERNS --regexp PATTERNS | This option is most useful when multiple -e options are used to specify multiple patterns, or when a pattern begins with a dash (- ). |
-E extended-regexp | Interpret pattern as an extended regular expression. |
-H | Always print filename headers with output lines. |
-n --line-number | Each output line is preceded by its relative line number in the file, starting at line 1. |
-i --ignore-case | Perform case insensitive matching. By default, it is case sensitive. |
-v --invert-match | Selected lines are those not matching any of the specified patterns. |