Cross-region data replication
JuiceFS supports data replication across the cloud and can replicate data asynchronously to object storages in another region or other cloud service.
For the convenience of description, we will refer to the server region specified when creating the file system as the "main region". To enable data replication, you need to specify a target region for the file system. It can be the same or different cloud service provider as the main region.
How it works
Taking the main region write and the target region read as an example, the data copy works as shown below:
Access to the same metadata service from both regions is readable and writable.
When data is written, the data is preferentially written to the object storage of the current region. After successful, the data is considered to be completed and then asynchronously copied to the remote object storage.
When reading data, it will preferentially reading from the object storage of the current region. If it does not exist (not yet synchronized), it will be read from the object storage in the remote region (in most cases, it needs to cross the public network, performance will be affected).
The data flow of write to target region and read from main region is as shown in the following figure:
Due to network conditions, access speeds are generally faster in the main region and slower in the target region (you can enable metadata caching via --metacache to improve read data performance).
In the volume settings page, you can enable data replication, select a target cloud service and region, and save the settings, then re-mount the file system through the client to take effect.
When mounting in the main region, please use the default configuration. When mounting in the target region, you need to add the
--flip parameter to indicate that the object storage of the target region is written first, and then asynchronously copy to the object storage of main region.
Because both zones use the same metadata service, they will see exactly the same status. The data in the object storage uses a unique key, so no need to worry about data consistency.
Cross-region data replication is currently available to all users for free. When enabled, no additional metadata is generated, and JuiceFS charges the same amount as before.