Skip to main content

JuiceFS Cloud Service Quick Start Guide

tip

This is a quick start guide for JuiceFS Cloud Service. If you are using the JuiceFS Community Edition, please visit Community Edition Documentation.

Concept at a glance​

JuiceFS is a distributed file system driven by a combination of object storage and database. "Object storage" is used for data storage, and "database" is used for metadata storage. The resulting file system supports shared mount use on any networked server.

JuiceFS Cloud Service provides out-of-the-box self-developed database, so you just need to prepare object storage.

System requirements​

  • Storage: With JuiceFS, all data will be stored in your own object storage, you can create the object storage Bucket manually in the cloud platform in advance, or you can let JuiceFS client automatically create the Bucket for you. either way, you need to prepare the secret key to access the API of object storage.
  • Python: Make sure you have Python installed, Python>=3.5 is recommended, Python>=2.7 is supported as a minimum.
  • FUSE: JuiceFS achieves POSIX compatibility via FUSE, make sure you have FUSE installed:

Windows caveats​

JuiceFS client for Windows is not yet available for open use, contact us if you are in need.

On the other hand, you may choose to use JuiceFS client in WSL 2, installation steps are just the same as Linux. But note that WSL 2 requires Hyper-V which is usually not available on cloud service VMs, proceed with caution.

Creating a file system​

You can create any number of file systems, each associated with an object storage bucket. Open the web console, click the "Create File System" button.

In the above form:

  • Volume Name: The name for your JuiceFS file system. Starting with a lowercase letter, can only contain lowercase letters, numbers, and hyphens (-) are allowed, and can be 3-40 characters long.
  • Region: The service provider / region of your object storage, JuiceFS Client will automatically pick the fastest Metadata Server according to your region selection.
  • Trash Expiration Time: Days to keep deleted files, set to zero to disable. Files in trash remains billable, learn more at "Trash".
  • ​Advanced Options Bucket Name: The object storage bucket used by the file system, you can fill in an already created bucket, or JuiceFS will try to create for you on the first mount (Access Key/Secret Key is required to have create bucket permission).
  • ​Advanced Options Block Size: The object storage block size, 4M is usually a better default value because many object storage services use 4M as their internal block size, thus using the same block size in JuiceFS usually yields better performance. Learn how JuiceFS split and store files in "How JuiceFS Stores Files".
  • ​Advanced Options Compression: The compression algorithm used by the file system. Compression reduces object storage usage but affects performance. If you were to store data that's already compressed (like Parquet, ORC), disable compression to avoid read amplification.

Upon creation, you will be redirected to the settings page, follow the instructions to mount the newly created file system. Note the Token in the settings page, this is unique for each file system, and is used to authenticate when you mount this file system.

Mounting the file system​

JuiceFS can be mounted on any computers connected to the Internet, and then be used like a local disk. But with production environments, machines should be in the same region as the file system's object storage, in order to achieve the best cost and performance.

Client installation​

note

JuiceFS Cloud Service Client is different from the JuiceFS Community Edition Client. If you need to use both, change the name of the program to avoid name conflict. For example, rename the cloud service client to juicefs-cloud.

For Linux and macOS, use the command below to install JuiceFS client (might need root permission). The installation path in the example command is set to /usr/local/bin by convention, you can also modify it according to the actual situation.

curl -L https://juicefs.com/static/juicefs -o /usr/local/bin/juicefs && chmod +x /usr/local/bin/juicefs

If you encounter a Permission denied error during the execution of the above command, please switch to the root user to execute.

In the terminal, run juicefs --help and expect a help message printed, indicating that the client was installed successfully. If you see command not found, check your environment variables and ensure /usr/local/bin is in PATH.

Mounting​

To mount the file system that you just created, run:

sudo juicefs mount $VOL_NAME $MOUNTPOINT

Please replace $VOL_NAME with the file system name set when "Creating a file system", and replace $MOUNTPOINT with the path of the JuiceFS mount point.

The client will interactively ask for credentials. You'll be needing the token of the file system, and the Access Key ID / Access Key Secret required to access the object storage (must have full access to the underlying bucket).

$ sudo juicefs mount myjfs /jfs
Token for myjfs: xxxxx
Access key ID for oss://juicefs-myjfs: xxxxx
Access key secret for oss://juicefs-myjfs: xxxxx
OK, myjfs is ready at /jfs.
info

Credentials are required when mounting for the first time, upon a successful mount, relevant information will be stored in the $HOME/.juicefs/$VOL_NAME, so you won't need to provide them again on the next mount.

Now that JuiceFS has been mounted on the system, the client will run as a daemon in the background, logs will be written to syslog (Facility: LOG_USER) or /var/log/juicefs.log, refer to "Fault diagnosis and analysis" for specifics.

Upgrade & Seamless remount​

It's recommended to continuously upgrade JuiceFS Client in order to get the latest improvements and fixes, upgrading is done by:

juicefs version --upgrade

A remount is required in order for the upgrade to take effect, if you're already using JuiceFS>=4.6, use --restart to achieve upgrade and seamless remount at the same time, using this one-liner:

juicefs version --upgrade --restart

The mount command itself supports seamless remount as well (needs Python>=3.3), simply run juicefs mount in a supported Python environment:

sudo juicefs mount myjfs /jfs
# Ensure Python>=3.3 is used to run JuiceFS
sudo ps -ef | grep juicefs
# Use different options to test seamless remount
sudo juicefs mount myjfs /jfs --attrcacheto=3
# Newly added options should appear in mount command
sudo ps -ef | grep juicefs

For older versions of Python, upgrading needs manual remount, see Unmounting the file system.

Auto-mount on boot​

After confirming that the mount is successful and can be used normally, you can refer to this section to set the automatic mount on boot.

Linux​

Add an entry in file system table (/etc/fstab) so that JuiceFS is automatically mounted on boot. JuiceFS v4.8 and above supports --update-fstab to make this easier:

# Needs root permission to modify /etc/fstab
$ sudo juicefs mount --update-fstab $VOL_NAME $MOUNTPONT
$ grep $VOL_NAME /etc/fstab
<VOL_NAME> <MOUNTPONT> juicefs _netdev 0 0
note

CentOS 6 does not automatically mount the network file system at boot time by default, you need to run the following command to enable automatic mounting.

sudo chkconfig --add netfs

If you'd like to control this process by hand, note that:

  • A soft link needs to be created from /sbin/mount.juicefs to the juicefs executable, the /sbin/mount.juicefs command is called when the operating system parses the fstab.

  • All parameters provided in your juicefs mount command must also be included in the fstab options to take effect. Remember to remove the prefixing hyphen(s), and add their values with =, for example:

    $ sudo juicefs mount --update-fstab $VOL_NAME $MOUNTPONT -b --max-uploads=1 --prefetch 2 --writeback myjfs /jfs -o max_read=3
    # -o stands for FUSE options, and is handled differently
    $ grep $VOL_NAME /etc/fstab
    <VOL_NAME>  <MOUNTPONT> juicefs _netdev,background,max-uploads=1,max_read=3,prefetch=2,writeback 0 0

If services depending on JuiceFS have also been configured to start automatically on boot, you may need to adjust the startup order. Using Docker as an example, set up JuiceFS dependency by the following systemd unit file:

/etc/systemd/system/docker.service.d/override.conf
[Unit]
# systemd unit for JuiceFS mount is usually called jfs.mount
# find out using: systemctl list-units | grep mount
After=network-online.target firewalld.service containerd.service jfs.mount

macOS​

  1. Create ~/Library/LaunchAgents/com.juicefs.VOL_NAME.plist as follows, replace capitalized words to their true values.

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
            <key>Label</key>
            <string>com.juicefs.VOL_NAME</string>
            <key>ProgramArguments</key>
            <array>
                    <string>/usr/local/bin/juicefs</string>
                    <string>mount</string>
                    <string>VOL_NAME</string>
                    <string>MOUNTPOINT</string>
            </array>
            <key>RunAtLoad</key>
            <true/>
    </dict>
    </plist>

  2. Run the following command and see if mount succeeds.

    launchctl load ~/Library/LaunchAgents/com.juicefs.VOL_NAME.plist
    launchctl start ~/Library/LaunchAgents/com.juicefs.VOL_NAME
    ls $MOUNTPOINT

  3. If the mount is not successful, add this piece to the above plist file:

    <key>StandardErrorPath</key>
    <string>/tmp/mycommand.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/mycommand.out</string>

    Reload to check JuiceFS logs:

    launchctl unload ~/Library/LaunchAgents/com.juicefs.VOL_NAME.plist
    launchctl load ~/Library/LaunchAgents/com.juicefs.VOL_NAME.plist
    cat /tmp/mycommand.out
    cat /tmp/mycommand.err

Import existing files​

To import files into JuiceFS, simply mount JuiceFS Client, and then copy data using tools like cp or rsync. If the data set is too large, we also recommend using Juicesync to further improve import speed.

Unmounting the file system​

caution

Force unmounting a file system in use may result in data loss. Make sure you have backed up all your important data.

Use the umount command to unmount the file system.

umount /jfs

If the command returns the target is busy error, it means that the file system is in use, use lsof MOUNTPOINT to find and kill the process of the relevant application, and try again.

For Linux, you can use the -l option to perform a delayed unmount (unmounting the file system immediately, but waiting until the device is no longer busy to clean up the resources).

umount -l /jfs

For macOS, use diskutil to force an unmount:

diskutil unmount force /jfs

Also with macOS, you can find JuiceFS in Finder - Locations - (Your computer), and then press the eject key to umount.

You can kill the juicefs client process manually, if you don't, it'll exit on its own.

Delete file system​

If you need to delete a file system created in the cloud service, the following steps should be strictly followed for data security:

  1. Backup all data in the file system
  2. Unmount the file system on all host
  3. Delete the file system in the console
  4. Delete file system related configuration file: rm $HOME/.juicefs/$VOL_NAME.conf
Last updated on
Follow us on Twitter
Let's chat on Slack