JuiceFS v1.0 Beta2 released - further stability improvements

2022-03-10

Juicedata

This is the second beta version of JuiceFS v1.0, with 16 community partners contributing 150+ commits 🎉. This update focuses on bug fixes and stability improvements, supplemented by a lot of documentation updates and test case optimizations, and brings the following new features.

I. New: BadgerDB as metadata engine

Contributed by community developer Qin Muyang (@diluga), adding support for BadgerDB databases to the JuiceFS plug-in metadata engine architecture, adding another member to the JuiceFS metadata storage engine family!

BadgerDB is an embedded, persistent, stand-alone Key-Value database developed in Go language, similar to SQLite, which can be used directly without installation.

# Create a file system
$ juicefs format badger://$HOME/badger-data test-volume
# Mount a file system
$ juicefs mount -d badger://$HOME/badger-data /mnt/jfs

Unlike SQLite, BadgerDB is not a single file database, the data is stored in a directory. In the above example, BadgerDB creates the badger-data database directory in the home directory, and the path to this directory is also used when mounting the file system.

II. One click to start a WebDAV server

Through the joint efforts of the open source community, we have added the WebDAV access protocol to our stable support for storage access protocols such as FUSE POSIX, S3, HDFS, CSI Driver, etc. Similar to the way S3 gateways are enabled, you can start a WebDAV server with one click using the new webdav subcommand.

$ juicefs webdav redis://127.0.0.1:6379/1 localhost:9007

You can read and write to the JuiceFS file system by accessing localhost:9007 with any client that supports the WebDAV protocol.

III. Support read-only mode connection to Redis Replicas

For scenarios where the Redis engine is stressed with reads and writes, it now supports allowing some clients to connect to replicas managed by Sentinel in read-only mode, thus reducing the load on the master node.

$ juicefs mount redis://mymaster,sentinel.local:26379?route-read=replica /mnt/jfs/ --read-only

That is, adding the parameter route-read=replica to the URL specifies that the client automatically reads metadata from Redis replicas.

Note that because Redis replica uses an asynchronous update mechanism, clients with read-only settings may sometimes read older versions of metadata.

IV. Clearer client help information

With the growing community of users, we found that making JuiceFS more accessible to users of all experience backgrounds required friendly documentation on the one hand, and enhanced command help information on the JuiceFS client itself on the other. The latter, in particular, would greatly improve the user experience if problems could be solved from the command line.

With this in mind, we have made the following improvements to the client's help information in this updated version.

Clearly classified
ADMIN: Used to manage JuiceFS volumes, such as format, destroy, gc, etc.
INSPECTOR: Used to review the internal status of JuiceFS, such as stats, profiles, etc.
SERVICE: Used to start specific services, such as mount, gateway, etc.
TOOL: Standalone tools for specific needs, such as bench, warmup, etc.
Detailed description: Specific instructions and usage examples have been added to most commands.

V. Prevention of misuse

The discussion in the community group proves one thing: even seasoned veterans can't help but "do something wrong"! Especially when creating and modifying file systems, some inadvertent missteps can overwrite old data or lead to incorrect settings.

For these operations that may involve data security, we have been improving the corresponding security check mechanism, and the following security mechanisms have been added this time.

Uniqueness check: Check and prohibit the creation of file systems with the same name on the same Bucket.
Prevent illegal parameters: When modifying file system parameters, the config command will try to determine in advance whether the new parameters are reasonable and avoid illegal parameter configurations.
Hide sensitive information: Hides the process password parameter displayed by ps when a password is required to connect to the metadata engine.