MLPerf Storage Benchmark Results for JuiceFS

Testing approach

MLPerf® Storage is a universal AI benchmark suite that simulates real AI workload access to storage systems through multiple clients. It replicates storage loads in large‑scale distributed training clusters to comprehensively evaluate the practical performance of storage systems in AI training tasks.

The MLPerf Storage v2.0 suite includes training workloads covering the most representative I/O patterns in deep learning training, plus a checkpointing workload:

Task	Model	Sample size	I/O pattern
Medical image segmentation	3D U-Net	146 MiB	Sequential/concurrent reads of large‑volume 3D data
Image classification	ResNet-50	150 KiB	Highly concurrent random reads of massive small images
Cosmology prediction	CosmoFlow	2 MiB	Highly concurrent reads of vast numbers of small files
Checkpointing	Llama-3	GBs to TBs	Concurrent sequential writes of large files

The benchmark suite also requires submitted results to meet the following GPU utilization thresholds while maximizing the number of GPUs supported:

• The 3D U‑Net workload: 90% GPU utilization
• The ResNet-50 workload: 90% GPU utilization
• The CosmoFlow workload: 70% GPU utilization

The key differentiator is the maximum number of GPUs a storage system can support, which is essentially determined by the system's maximum aggregate bandwidth.

Testing tool

The tests were performed with MLPerf Storage v2.0 benchmark suite. All results were officially submitted to MLCommons (the global authoritative AI engineering consortium who developed MLPerf Storage and other AI benchmark suites) and published in the August 2025 benchmark round.

JuiceFS participated in the shared file system category, along with other Ethernet‑based solutions (Alluxio, Oracle, etc.) as well as InfiniBand‑based storage appliances (DDN, HPE, Ubix, etc.). These InfiniBand‑based solutions deliver extremely high bandwidth and performance, but their hardware specifications and overall cost are also very high.

Testing environment

The JuiceFS system topology in this test consisted of three layers, which is a common JuiceFS deployment architecture for large‑scale AI training or other workloads that require high‑bandwidth and low‑latency access to large datasets:

• The client layer: 10 client nodes, all running on the same GCP instance type with identical bandwidth configurations per node.
• The cache cluster layer: 10 cache nodes using cloud disks combined with local memory as cache to accelerate data access.
• The cold data storage layer: Data is stored in Google Cloud Storage (GCS). Before training started, cold data was warmed up from GCS to the cache cluster.

Notes

MLPerf Storage v2.0 allows data warm-up before the official test. In the JuiceFS setup, cold data was pre‑loaded from object storage (GCS) into the distributed cache cluster to ensure stable high‑bandwidth and low‑latency access during training. This practice is recommended when benchmarking large‑scale AI workloads to avoid interference from object storage latency and to ensure fairness among different storage solutions.

The 10‑node scale officially submitted to MLPerf reflects the maximum configuration permitted by the benchmark suite's rules for this test scenario. This external constraint does not represent a performance ceiling for JuiceFS. With more nodes, the distributed cache and the metadata engine scale out to provide linearly increasing bandwidth, supporting significantly larger GPU clusters.

Testing results

The 3D U‑Net workload

In this workload, JuiceFS delivered strong results across all key metrics:

• Supported a training scale of 40 H100 GPUs across 10 nodes
• Delivered a data read bandwidth of 108 GiB/s
• Achieved 86.6% network bandwidth utilization
• Maintained 92.7% GPU utilization

Among all Ethernet‑based storage systems, JuiceFS delivered the highest bandwidth and highest network utilization, indicating not only higher total bandwidth but also more efficient use of network and hardware resources.

The CosmoFlow workload

In this small‑file, high‑concurrency workload, JuiceFS performed reliably:

• Supported a training scale of 100 H100 GPUs across 10 nodes
• Maintained 75% GPU utilization

This workload places extreme demands on metadata performance and read latency stability. JuiceFS continued to lead among comparable systems, successfully maintaining GPU utilization above the required 70% threshold despite the high latency sensitivity of massive concurrent small‑file access.

The ResNet‑50 workload

For the ResNet‑50 workload involving high‑concurrency random reads of small images, JuiceFS delivered strong results:

• Supported a training scale of 500 H100 GPUs
• Delivered a data read bandwidth of 90 GiB/s
• Achieved 72% network bandwidth utilization
• Maintained 95% GPU utilization

In this test, JuiceFS supported the largest number of H100 GPUs (500) among all Ethernet‑based solutions and achieved the highest network bandwidth utilization (72%), far exceeding the ~40% level of other vendors. This demonstrates JuiceFS' ability to fully leverage software optimizations to efficiently utilize hardware resources under high‑concurrency random I/O scenarios.

Summary

JuiceFS demonstrated stable low latency and high resource utilization across different AI training workloads in the MLPerf Storage v2.0 tests. As a user‑space, cloud‑native distributed file system, it utilizes a distributed caching layer and a high‑performance metadata engine to provide a unified storage system for large‑scale AI training clusters, achieving high throughput and low latency while leveraging object storage for cost advantages.

Key takeaways:

• Bandwidth utilization: JuiceFS achieved 86.6% network bandwidth utilization in the 3D U‑Net workload, the highest among Ethernet‑based systems.
• Scalability: JuiceFS supported up to 500 H100 GPUs in the ResNet‑50 workload, the largest scale among comparable systems.
• Metadata performance: The high‑performance metadata engine with integrated client‑side caching delivers very high IOPS and extremely low latency, critical for small‑file workloads like CosmoFlow.
• Elastic caching layer: The distributed cache cluster supports elastic scaling, providing up to 1.2 TB/s aggregate bandwidth and sub‑millisecond access latency.

For large‑scale AI training that does not rely on expensive proprietary hardware, JuiceFS presents a viable, cost‑effective option.

References

• MLCommons is an artificial intelligence engineering consortium, built on a philosophy of open collaboration to improve AI.
• The MLPerf Storage benchmark measures how fast storage systems can supply training data when a model is being trained.
• To learn more details about our journey working with this benchmark, read the blog post: MLPerf Storage v2.0: JuiceFS Leads in Bandwidth Utilization and Scalability for AI Training.