FAQ

My question is not answered in the documentations

Try searching using different keywords (top right corner), if that doesn't help, reach the Juicedata team using Intercom by clicking the bottom right button.

Why is file system data size different from object storage usage?

Data written to JuiceFS is eventually uploaded to object storage. However, due to JuiceFS' architecture, there is typically a difference between the file system's reported data size and the actual storage usage in the object storage bucket. While this difference is generally less than 5%, certain features can increase it. If compression is enabled for the file system, or juicefs snapshot is heavily used (same goes for hard links), you may see a larger difference.

• The most common reason for a size difference is the Trash feature. When Trash is enabled, deleted files are retained for a configured period before being removed by a background job garbage collection process.

• If you use juicefs snapshot to clone files or directories, only the metadata is copied, while the same underlying object storage blocks are referenced. When a lot of snapshots are created, JuiceFS usage can be significantly higher than object storage buckets. Important notes regarding the snapshot feature:

  • For Linux clients using kernels with copy_file_range support, the cp command achieves the same result as juicefs snapshot.
  • ​On-prem For a multi-zone architecture, when juicefs snapshot tries to copy file across different metadata zones, only in this special case, JuiceFS also copies object storage data asynchronously, which increases object storage usage.

• If you create hard links within the file system, the file system usage statistics increase. However, since the links reference the same set of object storage blocks, object storage bill is not affected.

• Random writes and overwrites produce more slices than other write patterns. Consequently, the occupied space in object storage can be greater than or equal to the actual data size. When files become fragmented, slices (the term in JuiceFS that stands for data fragments) are compacted and re-uploaded to object storage via a background job. When files become fragmented, slices (the term in JuiceFS that stands for data fragments) are compacted and re-uploaded to object storage via a background job. However, the slices will continue to occupy space in the object storage even if they are already overwritten and become stale data. By default, they are retained for 24 hours and then cleaned up after expiration. If the background job cannot keep up, stale object storage data accumulates and causes higher object storage usage.

• When there is object storage service outage, DELETE requests will fail and be retried repeatedlybefore eventually abandoning the deletion requests. This can cause object storage leakage. If you noticed significantly higher usage on the object storage, run juicefs gc to scan and inspect. Submit the scan results to Juicedata engineers to jointly determine the cause and whether cleanup is necessary.

• If a file is deleted while it is still open, although it becomes invisible in the file system, some processes are still using it. To ensure normal access for those processes, the object storage data are not deleted until all processes have closed the file.

• If compression is enabled, the objects stored in object storage might be smaller than the actual file size (depending on the compression ratio of different file types).

• If you use juicefs import or the import on demand feature, this is a special mode that does not involve compaction and data overwrites.

  If the whole bucket is imported, the JuiceFS file system size equals the bucket size. If only a subset of files are imported, JuiceFS only holds partial of the full data. In short, JuiceFS charges based on the actual amount of data imported into the file system.

• Billing size may vary with different storage classes across different service providers, For example, the minimum billable size for Alibaba Cloud OSS IA storage is 64KB. If a file is smaller than 64KB, it will be calculated as 64KB.

• ​On-prem For self-hosted object storage services, for example MinIO, actual data usage is affected by storage class settings.

Which public cloud and region is supported?

Currently, all regions from Amazon Web Service (AWS), Google Cloud Platform (GCP), Microsoft Azure, Alibaba Cloud, Tencent Cloud, Huawei Cloud, UCloud, QingCloud, Qiniu, Baidu Cloud, Kingsoft Cloud, Netease Cloud, JD Cloud, and DigitalOcean are supported (many region may not open for public).

Any public cloud which supports Linux instance and object storage, could be easily supported by JuiceFS. If a public cloud or a region is not listed in web console, please contact us.

JuiceFS also supports self-hosted deployment in the enterprise's own public cloud VPC and data center, contact us if you're in need.