JuiceFS Cloud Service Quick Start Guide
tip
This is a quick start guide for JuiceFS Cloud Service. If you are using the JuiceFS Community Edition, please visit Community Edition Documentation.
Working Environment
- OS: JuiceFS supports Linux, macOS and Windows systems, POSIX compatibility via FUSE, please make sure you have FUSE installed before using it. most Linux distributions already have the FUSE module built in. macOS recommends installing macFUSE for FUSE support. Windows recommends installing WinFsp for FUSE support.
- Storage: With JuiceFS, all data will be stored in your own object storage, you can create the object storage Bucket manually in the cloud platform in advance, or you can let JuiceFS client automatically create the Bucket for you. either way, you need to prepare the secret key to access the API of object storage.
- Python: Make sure you have Python installed, Python 3 is recommended, Python 2.5+ is supported as a minimum.
Creating a file system
To use JuiceFS cloud service, you need to create a file system in the official website console, new users should visit JuiceFS official website registration first. Then login to your console and click the "Create File System" button.
Server Region Use to specify the service provider and region where your object storage is located. the JuiceFS client will automatically create object storage for you in the selected region when the file system is mounted, and automatically match you with the fastest metadata storage cluster.
note
If the option you want is not in the list, please refer to the prompt message to submit feedback and our engineers will support the desired platform within 72 hours and the desired region within 24 hours.
If you have prepared object storage in advance, please check "Advanced Options" and fill in the name of the Bucket you have created. In addition, you can adjust the "Block size" and "Compression algorithm" of the file system in the advanced checkbox.
When the file system is created, the page will automatically jump. Next, you can follow the instructions on the page to mount the newly created file system to your server.
In the help notes provided on the page there is a bolded Token that you need to use to authenticate when you mount this file system on your server.Each file system has a unique Token.
info
You can create any number of file systems, each associated with at least one bucket of object storage. In short, with JuiceFS, all data is stored in your own object storage.
Mounting the file system
Next you can mount the file system you created on the console to your server and use it as a local disk.
Installing the client
Any computer that needs to mount a filesystem will need to install the client first, execute the command to download and install.
curl -L https://juicefs.com/static/juicefs -o juicefs && chmod +x juicefs
note
"The Cloud Service Client" is different from the "Community Edition Client". If you need to use both, please change the name of the program to avoid name conflict. For example, you can rename the cloud service client to juicefs-cloud.
In the terminal, type . /juicefs and see that a help message is returned, indicating that the client was installed successfully.
$ ./juicefs
juicefs COMMAND [options]
COMMAND could be:
auth authorize a filesystem
mount mount a filesystem
umount umount a filesystem
info show information of file or directory
lsof list recent opened files
import import files from existing object storage
rmr remove all files/directories recursively
snapshot create or remove snapshots
grep parallelized file pattern searcher
profile analyze log (Experimental)
benchmark run benchmark
warmup build cache for target directories/files
version show the version
Mounting
The command format for mounting a file system is as follows:
$ sudo ./juicefs mount NAME-OF-JUICEFS /jfs
Please replace the NAME-OF-JUICEFS with the actual file system name.
When mounting a file system for the first time, the client will interactively ask you for a series of information. For example, to mount the file system quick to the /jfs directory, the client will ask for the Token of the file system, and the Access Key ID and Access Key Secret required to access the object storage API, respectively.
$ sudo ./juicefs mount quick /jfs
Token for quick: 4bc1xxxxxxxxx9591dec66
Access key ID for oss://juicefs-quick: LTxxxxxxxxxxxxVWL
Access key secret for oss://juicefs-quick: Xl7DpxxxxxxxxxxxxxxxDMhfUUa
OK, quick is ready at /jfs.
info
Only the secret key information such as Token is required when mounting the filesystem for the first time, after successful mounting, the secret key information will be stored in the configuration file in the $HOME/.juicefs directory. You don't need to provide them again when you mount it again, the client will read them from the configuration file automatically. The Token for the file system can be found in the official console, and the API access key for the object store can be obtained from refer to this document.
Please make sure that the Object Storage API access key has the permission to create and read/write Bucket.
After mounted, the client will run as a daemon in the background and all logs will be logged to syslog (Facility: LOG_USER).
Remember the Keys
For cases where the configuration needs to be automated by a third-party program, the relevant key information can be provided by means of command-line arguments.
sudo ./juicefs auth --token TOKEN --accesskey ACCESSKEY --secretkey SECRETKEY NAME
Alternatively, a JSON configuration file can be created directly to $HOME/.juicefs/NAME.conf, with information such as:
{"token": TOKEN, "accesskey": ACCESSKEY, "secretkey": SECRETKEY}
Auto-mount on boot
Linux
Rename and copy the cloud service client to the /sbin directory of the host:
sudo cp ./juicefs /sbin/mount.juicefs
Edit the /etc/fstab configuration file and add the following entry on a new line, replacing NAME-OF-JUICEFS with the name of your actual file system and /jfs with the actual mount point you want to use.
NAME-OF-JUICEFS /jfs juicefs _netdev 0 0
note
CentOS 6 does not automatically mount the network filesystem at boot time by default, you need to run the following command to enable automatic mounting.
sudo chkconfig --add netfs
macOS
Please replace the following with the actual information.
NAMEthe file system namePATH_TO_JUICEFSPath to the client (e.g./usr/local/bin/juicefs)MOUNTPOINTMount Points (e.g./mnt/myjfs)
Refer to unmount_filesystem to ensure that no filesystem is mounted on the target mount point.
Refer to remember_key to save the authorization information to a local file
Create the file
com.juicefs.NAME.plistunder~/Library/LaunchAgents.<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.juicefs.NAME</string>
<key>ProgramArguments</key>
<array>
<string>PATH_TO_JUICEFS</string>
<string>mount</string>
<string>NAME</string>
<string>MOUNTPOINT</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>Load the file created in the previous step, and test if the load is successful.
launchctl load ~/Library/LaunchAgents/com.juicefs.NAME.plistlaunchctl start ~/Library/LaunchAgents/com.juicefs.NAMEls MOUNTPOINTIf the mount is not successful, the following configuration can be added to the
plistfile above.<key>StandardErrorPath</key>
<string>/tmp/mycommand.err</string>
<key>StandardOutPath</key>
<string>/tmp/mycommand.out</string>Reload the updated configuration and view the output with the following command:
launchctl unload ~/Library/LaunchAgents/com.juicefs.NAME.plistlaunchctl load ~/Library/LaunchAgents/com.juicefs.NAME.plistcat /tmp/mycommand.outcat /tmp/mycommand.err
Mounting in Docker containers
JuiceFS supports being mounted by Docker or other containers, and you need to make sure that the container has access to the FUSE device before operating.
Execute the following command to enable FUSE access to the container:
sudo docker run --cap-add SYS_ADMIN --device=/dev/fuse --security-opt apparmor:unconfined
Import existing files
If you already have some files stored in the object storage before you use JuiceFS. You can use the import command of the cloud service client to import the existing files into the file system.
The command format is as follows:
sudo ./juicefs import BUCKET-NAME/PREFIX TARGET-DIR-IN-JUICEFS
For example, the following command imports all files in my-bucket starting with my-files into the /jfs/my-files folder.
sudo ./juicefs import my-bucket/my-files /jfs/my-files
info
The import operation only writes the metadata of the file to the metadata engine of the JuiceFS cloud service, and the file remains in the object storage as is.
After the import is complete, you can change the properties of these files in JuiceFS as you like, such as changing the file name or permissions, etc. These changes will not be synchronized to the object storage. Similarly, deleting "imported files" in the file system will only delete the metadata of these files, not the files in the object storage.
tip
The import operation relies on the API of the object store to access the secret key. Please make sure that the secret key has read access to the imported file.
Snapshots
A snapshot is a copy of data created of the state of a directory or file at a specific point in time, which is a very meaningful mechanism for data backup and protection. By creating a snapshot before a critical operation, subsequent operations can be restored to the state of the data at the time the snapshot was created in the event of a failure.
JuiceFS Cloud Service supports atomic snapshots, where you can make a copy of a file or directory immediately. The snapshot reuses the underlying data block and copies it only when the block is modified.
Command format for creating a snapshot:
./juicefs snapshot SRC DST
tip
Snapshots can only be created for files or folders stored in the JuiceFS file system, and they can only be saved in the file system where the source file is located.
Use the -d option to delete a snapshot:
./juicefs snapshot -d DST
Unmounting the file system
caution
Unmounting a file system involves some operational risks, and force unmounting a file system in use may result in data loss. Before performing the operation, please make sure you have backed up all your important data.
Use the umount command to unmount the file system.
sudo umount /jfs
If the command line returns umount: /jfs: target is busy., it means that the file system is in use, it is recommended to use lsof MOUNTPOINT to find and end the process of the relevant application before performing the unmount.
For Linux distributions, you can use the -l option to perform a delayed unmount (unmounting the filesystem immediately, but waiting until the device is no longer busy to clean up the resources).
sudo umount /jfs -l
For macOS systems, the force option can be used to force an unmount:
sudo diskutil unmount force /jfs
Then you can either kill the juicefs client process manually, or wait for it to exit on its own.
Delete file system
If you need to delete a file system created in the cloud service, the following steps should be strictly followed for data security reasons:
- backing up all data in the file system.
- unmounting the file system on the host.
- delete the file system in the console.
- find and delete the file system configuration file in the
$HOME/.juicefsdirectory of the host, usually named<YOUR-FS-NAME>.conf.