pNFS-Lattice Implementation - PHase 1

Design Intent

Lattice is intended to evolve into a scale-out metadata and coordination platform where:

  • protocol personalities plug into Lattice Core
  • metadata authority is shared and versioned
  • protocol state can be shared across MDS instances
  • static and ephemeral MDS instances can participate safely
  • data-server control remains explicit and direct
  • peripheral data services remain modular rather than defining the architecture

Lattice therefore starts with a working pNFS MDS, but its design target is broader: a modern distributed metadata platform for the evolving pNFS world and beyond.

Phase 1

Phase 1 delivers the first working Lattice implementation. It intentionally separates the current implementation substrate from the long-term architectural model.

  • Active protocol personality. NFSv4.2/pNFS.
  • Shared catalog substrate. RonDB provides the Phase 1 shared substrate for the MD Catalog Authority.
  • Runtime protocol state. Current Lattice MDS instances manage protocol runtime state locally using sharded in-memory structures.
  • Hot-tree scale-out model. Phase 1 uses referral-based sharding for hot namespace regions so active metadata service can be distributed across multiple MDS instances.
  • Scale validation. The architecture has been designed toward clusters of up to 72 metadata servers and has been tested as of April '26 at 16 MDS instances.
  • Forward design target. A future shared Protocol State Plane remains a deliberate community development area.
Catalog Substrate: RonDB is used in Phase 1 as the shared substrate for theMD Catalog Authority and large-scale testing of Lattice Core.

The Phase 1 model is intentionally aggressive. It has been essential in proving the Lattice Core design at meaningful scale, while also making clear that Lattice's longer-term goal is significantly more advanced and is aimed at the evolving demands of AI, HPC, and future large-scale metadata services.

Alongside the core Lattice work, PEAK:AIO has also developed and tested advanced supporting modules including resilvering, weighted data distribution, failover, and tiering. These have been valuable in exercising and strengthening Lattice Core during development. However, they are not being presented as part of the initial open core itself. The long-term intent is to evolve such capabilities into dedicated production-grade modules with clearer boundaries, stronger packaging, and independent lifecycle management.

Scalable MDS Goal

Lattice is not designed as a single large MDS. The goal is a cluster of MDS runtimes that can scale from small deployments to larger fleets while preserving correctness, failover, and metadata authority. The design therefore treats the following as core concerns rather than optional extras:

  • namespace partitioning across MDS instances as an active scale-out mechanism
  • cross-MDS consistency for operations such as rename and hardlink
  • shared metadata authority
  • failover and takeover
  • migration and elastic growth
  • future support for more ephemeral MDS instances

The long-term architectural target is a system in which static and ephemeral MDS instances can safely participate through a shared Protocol State Plane and a shared MD Catalog Authority, while namespace partitioning remains the primary active scale-out mechanism.

Lattice Core

Lattice Core is the pluggable user-space MDS runtime that forms the operational heart of Lattice. In the current userspace implementation it is delivered with a working NFSv4.2/pNFS personality and provides the active execution environment for a real scale-out metadata service. Responsibilities include:

  • Protocol execution. Request ingress, dispatch, COMPOUND processing, XDR encode and decode, and RPC handling.
  • Runtime state handling. Session, open/lock, and grace management.
  • Metadata execution. Namespace orchestration, catalogue dispatch, and durable metadata update paths.
  • pNFS control. Layout control, placement, and DS selection.
  • Cluster participation. Cluster-aware metadata workflows, routing, and scale-out coordination.

Lattice Core is therefore much more than an RPC endpoint. It is the runtime that receives protocol work, resolves it into authoritative metadata actions, coordinates with the wider Lattice cluster, and drives the behavior of each live MDS instance.

MD Catalog Authority

Architectural metadata authority for namespace, identity, attributes, placement, reverse indexes, and workflow state across Lattice instances.

The MD Catalog Authority is the authoritative metadata truth of the system. It provides durable, versioned truth for namespace, identity, attributes, placement, reverse indexes, workflow state, stripe-map and DS-related metadata, and related durable coordination records. In Phase 1, Lattice uses RonDB as the shared catalog substrate. This is important because it gives Lattice Core a real distributed metadata authority for development, community co-development, and large-scale testing.

The broader architectural concept is larger than any single backend choice. Lattice requires a shared authoritative metadata plane. Phase 1 uses RonDB to exercise that plane in practice.

Protocol State Plane

Shared authority for NFSv4.2/pNFS client, session, replay, lease, stateid, layout, fencing, and failover state across Lattice instances

The Protocol State Plane is a core part of the Lattice design, but it is not yet implemented as a fully shared subsystem in Phase 1.

Today, Lattice MDS instances still manage runtime protocol state locally using standard sharded in-memory structures. That includes state associated with sessions, opens, locks, delegations, replay handling, and related runtime protocol semantics. This is sufficient for Phase 1 and allows the current user-space MDS to function correctly today. Moving forward, the shared Protocol State Plane is a major design target for stronger shared failover and takeover semantics, cleaner scale-out state management, more elastic MDS participation, and future support for more ephemeral MDS instances.

Why Direct MDS-to-DS RPC Matters

Direct MDS-to-DS RPC for registry, health, provisioning, placement coordination, and future persistent data services across the storage backbone.

Lattice's design expects direct MDS-to-DS RPC for control-plane operations. This keeps DS-side control in the metadata and control plane rather than pushing that responsibility into clients or middleware. In practice, it supports looser coupling to DS implementations, cleaner control of DS-side provisioning and metadata-related operations, better separation between metadata authority and data-server execution, and a more scalable architecture than forcing the MDS to behave like a traditional client for every DS management task. In short, Lattice's intent is that the MDS controls DS-facing operations directly and explicitly.