Skip to main content

JuiceFS Java Client Parameter Reference

Add the following parameters to the core-site.xml of Hadoop.

Basic Configuration

ItemDefault ValueDescription
fs.jfs.implcom.juicefs.JuiceFileSystemSpecify the implementation of the storage type jfs://.
fs.AbstractFileSystem.jfs.implcom.juicefs.JuiceFSSpecify the implementation of AbstractFileSystem for MapReduce.
juicefs.tokenCredential for JuiceFS volume, checkout from the setting page of JuiceFS web console.
juicefs.accesskeyAccess Key for object store. It doesn’t need if the node has priviledge to access the object store.
juicefs.secretkeySecret Key for object store. It doesn’t need if the node has priviledge to access the object store.

Cache Configuration

ItemDefault ValueDescription
juicefs.cache-dir/var/jfsCacheCache directory, you could specify more than one folder, seperate by colon, or use wildcards *. Typically, the component doesn’t have permission to create these diectories, you need to create manully and give 0777 permission for components could share data in these folders.
juicefs.cache-size10240Cache capacity (unit MB), if multiple diectories are configured, this is total capacity for all cache folders.
juicefs.cache-replica1Number of cache replica.
juicefs.memory-size300Maximum memory used for read-ahead, unit MB.
juicefs.auto-create-cache-dirtrueWhether to create cache directories automatically. When set to false, it will ignore the a cache directory that doesn’t exist.
juicefs.free-space0.2Minimum free space ratio. When free space is under the ratio, it will clear the cache to free disk space, defaut 20%.
juicefs.metacachetrueEnable metadata cache.
juicefs.discover-nodes-urlSpecify the node list of cluster, refresh every 10 minutes. If you use YARN manage resource, nodes could be discovered by RESTful API (seperate by comma): http://resourmanager1:8088/ws/v1/cluster/nodes/. You could also give a path of the configuration file, e.g file:///etc/nodes or jfs://your-jfs-name/etc/nodes. The configuration file must be plain text, one line for a hostname.

Other Configuration

ItemDefault ValueDescription
juicefs.access-logThe filepath for access log (e.g /tmp/juicefs.access.log), read and write permission is required for every Hadoop component. Log file will rotate automatically, and retain last 7 files.
juicefs.debugfalseEnable DEBUG level log.
juicefs.max-uploads50Maximum number of concurrency for writing object store.
juicefs.superuserhdfsSpecify the superuser
juicefs.bucketSpecify the bucket name of object store.
juicefs.upload-limit0Speed limit for writing object store by a single process, units Byte/second.
juicefs.object-timeout5Timeout for visiting object store
juicefs.externalfalseUsing external domain to access object store
juicefs.rsaPrivKeyPathThe file path of RSA Private Key for data encryption.
juicefs.rsaPassphaseThe passphrase of RSA Key for data encryption.
juicefs.file.checksumfalseEnable checksum for copying data via Hadoop DistCp

When using multiple JuiceFS filesystems, all of above items can be specified for a single filesystem, the filesystem name JFS_NAME should to be placed in the middle of the configuration item, such as:

<property>
<name>juicefs.{JFS_NAME}.debug</name>
<value>false</value>
</property>