Skip to main content

Guidelines to deploy S3 Gateway for JuiceFS Cloud

JuiceFS Cloud provides S3 compatible RESTful APIs for file system access. Currently clients like s3cmd, AWS CLI and MinIO Client (mc) and S3 SDKs for various programming languages are capable for accessing files stored in JuiceFS.

Download Client

S3 Gateway for JuiceFS Cloud requires dedicated clients. Please choose the appropriate version based on your operating system and architecture.

sudo wget https://s.juicefs.com/static/Linux/s3-gateway -o /usr/local/bin/s3-gateway && sudo chmod +x /usr/local/bin/s3-gateway
sudo wget https://s.juicefs.com/static/Darwin/s3-gateway -o /usr/local/bin/s3-gateway && sudo chmod +x /usr/local/bin/s3-gateway

Command Syntax

s3-gateway [global options] command [command options] [arguments...]

Here are available [global options].

OptionDescription
--conf valueconfig in JSON (default: "jfs.conf")
--log valuepath of logging
--address valuebind to a specific ADDRESS:PORT, ADDRESS can be an IP or hostname (default: ":9000")
--umask valueumask for new file in octal (default: "022")
--debugenable debug log
--quietdisable startup information
--anonymoushide sensitive information from logging
--jsonoutput server logs and startup information in json format
--multi-bucketsuse top level of directories as buckets
--keep-etagkeep the etag information of the object
--help, -hshow help
--version, -vprint the version

Deploy Gateway

Before we go, please ensure that you have a file system created in JuiceFS Console, and have installed the appropriate client for JuiceFS Cloud. See Getting Started with JuiceFS Cloud for more details.

We assume that a file system named 'myjfs' has been created.

1. mount the file system

Mount myjfs to the $HOME/myjfs directory of the current user:

juicefs mount myjfs ~/myjfs

2. create a configuration file

Create a directory for configuration files, for example juicefs-s3, and enter this directory:

mkdir juice-s3 && cd juice-s3

Dump the content of file ~/myjfs/.jfsconfig into configuration file jfs.conf:

cat ~/myjfs/.jfsconfig > jfs.conf

3. set credentials

Set Access Key and Secret Key into environment variables, for example:

export MINIO_ACCESS_KEY="admin"
export MINIO_SECRET_KEY="password"

4. launch S3 gateway

s3-gateway

You will see the following outputs if your gateway starts successfully:

2022/03/28 14:39:42.567808 juicefs[57203] <INFO>: connected to meta 47.103.48.104:9308
2022/03/28 14:39:42.574222 juicefs[57203] <INFO>: Cache: /home/herald/.juicefs/cache/jfs8 capacity: 102400 MB
Endpoint: http://192.168.1.128:9000
RootUser: admin
RootPass: password

Browser Access:
http://192.168.1.128:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
$ mc alias set myJuiceFS http://192.168.1.128:9000 admin password

Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide

S3 gateway listens 9000 on default, and it is configurable by setting option --address.

For example, set the port for listening only:

s3-gateway --address :9090

Or, set both address and port:

s3-gateway --address 192.168.192.88:9090

Access Gateway

Once S3 Gateway for JuiceFS Cloud is launched successfully, you can directly access the URL you set in your browser, i.e., http://192.168.1.128:9000 in the example above. Open the file explorer for web, and enter the Access Key and Secret Key you set previously to login.

In addition, clients like AWS CLI and MinIO MC and S3 SDKs for various programming languages are also available for accessing S3 Gateway for JuiceFS Cloud.

Common Issues

1. ls doesn't list .jfsconfig file

.jfsconfig is a virtual file which locates in the root of JuiceFS mount point, that's why you cannot see it in the output of ls. However, you can still print its content by cat. Assuming your JuiceFS is mounted to ~/myjfs, you can see its content by:

cat ~/myjfs/.jfsconfig

2. ERROR Unable to validate credentials inherited from the shell environment: Invalid credentials

Seeing this error means the S3 gateway access credentials are not properly set. See 3. set credentials.

3. panic: runtime error: invalid memory address or nil pointer dereference

Seeing this error means the configuration file of the S3 gateway is not correct. Please check if the jfs.conf file exists and it has the same content as the .jfsconfig file in the root of the mount point.

If a non-default jfs.conf is used, you need to set option --conf to specify your configuration file path.