Keeping GPUs Fed: How Meta's AI Storage Architecture Aligns with JuiceFS

Introduction

Recently, I came across an eye-opening blog post and presentation from Meta's engineering team detailing their AI storage blueprint at scale. Meta operates hundreds of exabyte-scale storage clusters serving everything from Facebook and Instagram to Meta AI.

What caught my attention wasn't just the scale—it was the architecture. After years of wrestling with AI's brutal latency and throughput demands, Meta has converged on a clear pattern: data-metadata separation, rich client design, and comprehensive caching.

Here's the fascinating part: this blueprint isn't unique to Meta. JuiceFS shares these same first principles of design. The hyperscaler's new construction closely resembles the cloud-native architecture that JuiceFS has advocated for years.

In this post, I'll analyze both architectures, compare their core components, dive into their cross-region capabilities, and explain why this convergence in architecture makes the most sense for a distributed file system.

Why storage matters for AI workloads

Before diving into the architectures, it's worth understanding why storage has become such a critical component for AI infrastructure.

Meta puts it succinctly: "If AI is the brain, storage is the memory: capability and speed are highly dependent on the size of memory and speed of retrieval." Yet while AI compute performance has roughly tripled every two years, storage and interconnect performance growth have been far more modest. This disparity has made storage bottlenecks one of the primary contributors to GPU stalls for AI workloads, which is directly impacting both expenditures and time to market.

The math is brutal. During model training, hundreds of thousands of GPUs iterate over vast datasets across multiple epochs. GPUs synchronize their state periodically. If a single GPU stalls due to slow storage, it slows down the entire training job. In Meta's presentation, if GPUs were idle for about 20% of the time, "that 20% stall means about tens of millions of dollars loss every hour" for a data center the size of Manhattan.

Beyond the direct dollar cost, there is an even more consequential dimension: opportunity cost. The AI industry is moving at an unprecedented pace. Major model releases, which took roughly 104 weeks between 2020 and 2022, have now been compressed to approximately 4-week cycles by 2026. Every week of delay in shipping a frontier model translates to lost market share, diminished competitive advantage, and missed revenue opportunities.

In short, storage is what makes the difference between a model shipping in weeks versus months and between a billion-dollar data center running at 80% versus 100% efficiency.

Meta's AI storage blueprint: the evolved architecture

After iterating on their storage infrastructure, Meta has arrived at a definitive architectural blueprint to solve the GPU stall problem at its root while continuing to support existing external and internal products. The training stack has been gradually migrating to the BLOB-storage interface. The motivation is more about performance: AI workloads demand predictable, bounded pMax latency. Traditional layered metadata lookups could add hundreds of milliseconds per request, stalling GPUs during training. By moving to a BLOB-centric interface, Meta leverages flash drives to deliver the low, predictable latencies required to keep thousands of GPUs productive.

The three-component architecture

The diagram below illustrates the request flow for Meta's BLOB storage getObject API and shows the system's three major components.

• The Tectonic data storage: At the foundation sits Tectonic, a regional, multi-tenant storage fabric providing high durability and availability through erasure coding. It supports tiering across HDDs and flash drives, intelligently managing hot, cold, and warm data placement.

• The BLOB server and metadata layer: Operating on top of Tectonic, the BLOB layer handles API requests, stores metadata, and exposes a global, infinitely scalable storage fabric with policies allowing users to trade off between durability and availability, abstracting away the complexity of the underlying block storage.

• The rich client SDK: Meta eliminated the data-plane proxy and built a rich client SDK with a Tectonic BlockClient embedded within it, enabling direct data reading from Tectonic storage servers to clients. The client also implements hedged reads and dynamic concurrency control to reduce tail latencies.

To maximize cache efficiency, Meta also leverages spare memory on GPU hosts as a distributed data cache by integrating peers from the Owl subsystem directly into the BLOB-storage client SDK, which yields an average cache hit rate of 80%.

Cross-region data access with tiered caching

Meta's cross-region approach treats storage as a global data lake, but with tiered caches accelerating access across geographical boundaries.

The results speak for themselves: average ingestion time dropped by 93%, and worst-case ingestion time dropped by 97%. This is what it looks like when storage stops being the bottleneck and starts enabling AI research velocity.

JuiceFS: cloud-native, AI-native from day one

For readers already familiar with JuiceFS, the parallels to Meta's design are immediately apparent. The architecture Meta converged upon after years of iteration shares the same core principles that have guided JuiceFS from day one.

JuiceFS was architected from the ground up on the data-metadata separation principle, which also consists of three core components:

• Object storage: JuiceFS uses object storage as the reliable, scalable backend for actual data storage. It supports all major public cloud offerings (AWS S3, Alibaba OSS, Google Cloud Storage, etc.) as well as self-hosted solutions like Ceph and MinIO. This layer ensures that AI datasets, which can be as large as petabytes or exabytes, are safe, accessible, and can grow almost without limit.

• Metadata engine: The metadata engine stores all file system metadata, including file names, sizes, permissions, directory structures, and the mapping between files and their underlying data blocks. JuiceFS supports multiple engine options, including Redis, MySQL, PostgreSQL, TiKV, and many others. For extremely large-scale deployments, JuiceFS Enterprise Edition offers a proprietary Raft-based distributed metadata engine that provides high availability and strong consistency across multiple nodes.

• JuiceFS client: The client handles all file I/O logic, including data slicing, merging, caching, and POSIX semantics enforcement. It communicates with both the metadata engine and object storage directly, coordinating reads and writes intelligently.

Cross-region strategy: mirrors and replication

The JuiceFS Enterprise Edition also provides a native approach to support cross-region and cross-cloud data access:

• Mirror file systems: Deploy a full metadata cluster and optionally an object storage replica in a mirror region. Metadata syncs automatically from the source region with approximately 1-second latency under normal network conditions.

• Async data replication: JuiceFS supports cross-region and cross-cloud asynchronous replication with a one-to-many topology. Data written in the primary region is replicated to multiple mirror regions in the background, enabling global datasets without manual copying.

• Distributed cache: JuiceFS clients on mounting hosts can form a distributed cache group using a consistent hashing ring, sharing cached data blocks with one another. This is particularly effective for model training, where datasets are repeatedly accessed across GPU nodes.

• Smart read strategy: Clients preferentially read from the local region. If the data isn't fully synced yet, they seamlessly fall back to the source region.

This design resonates with Meta's global data lake + tiered cache approach but offers a more flexible, cloud-native model that works across public clouds, private data centers, and hybrid environments.

Architectural similarities

On the surface, Meta mainly exposes a BLOB-style interface, while JuiceFS primarily provides a POSIX file system. Yet beneath these protocol differences, the architectural parallels are noteworthy. Both systems converged on the same three-component design with comprehensive caching and cross-region capabilities:

A concrete example: explicit cache hydration

One interesting example is how both systems handle proactive cache hydration, which is the practice of loading data into the cache before it's actually needed.

Meta exposes a prefetch() API through the BLOB-storage SDK, while JuiceFS provides a warmup subcommand. Both serve the same purpose: allowing the client side to signal to the storage system exactly which data will be needed in the near future so it can be preloaded into cache before the GPU finishes its current work.

This is especially critical in distributed training, where thousands of GPUs may simultaneously read the same dataset. Without cache hydration, the initial requests would have to read from the underlying storage (Tectonic for BLOB and object store for JuiceFS) with higher latency. With prefetch() or juicefs warmup, data is staged in the distributed cache before the GPUs need it. In essence, both systems recognize the limitations of network-based storage and passive caching, and they both provide explicit prefetching as an important tool to compensate for what the storage layer alone cannot guarantee.

Conclusion

Meta's AI storage blueprint is a testament to the architectural patterns that emerge when a hyperscaler evolves its data infrastructure to hundreds of exabytes. After years of iteration, they arrived at a design that's similar to what JuiceFS has championed from day one: data-metadata separation, a rich client that handles I/O logic directly, tiered caching, and cross-region capabilities.

The same architectural pattern is also available to teams of any size. JuiceFS brought it into the big data era and now brings it into the AI age, with the flexibility of multi-cloud cross-region strategies, distributed caching, and a rich ecosystem of access protocols.

Hopefully, storage is no longer the bottleneck in your AI training stack. At least, it doesn't have to be.

If you have any feedback on this article or ideas to share, we invite you to participate in the discussions on GitHub and join our community on Discord.