Mitigating NVLink Stalls and Linux Page Cache Saturation During Distributed Checkpointing Using Async NVRAM Writeback

In frontier transformer training topologies scaling across tens of thousands of GPUs, checkpoint serialization is a primary operational source of Model FLOPs Utilization (MFU) degradation. Standard training regimes operating on 100,000 Hopper-class accelerator clusters require continuous persistence of distributed optimizer states (such as Adam FP32/FP8 momentums), master weights, and pipeline activations. Under a 3D parallelism architecture—combining Tensor Parallelism (TP), Pipeline Parallelism (PP), and Data Parallelism via ZeRO-3 or Fully Sharded Data Parallel (FSDP)—a 1-trillion parameter model represents multiple petabytes of aggregate checkpoint data that must be serialized every few hundred steps to mitigate frequent Mean Time Between Failures (MTBF). When distributed worker processes serialize checkpoints using conventional POSIX write() or framework-level torch.save() abstractions, the tensors transit from GPU High Bandwidth Memory (HBM3) across NVLink and PCIe Gen5 into host DRAM. If this data path interacts with the Linux virtual memory subsystem via buffered I/O, it rapidly saturates the Linux page cache. This triggers kernel-level synchronous dirty-page throttling, halting host worker threads. Because distributed collective operations (NCCL) require nanosecond-level host synchronization across nodes, a dirty-page writeback stall on even a single host cascades into an all-stop condition across the entire GPU fabric.

The Mechanics of Kernel Dirty Page Throttling During State Serialization

When tensors are staged from GPU HBM to the host operating system, the runtime executes cudaMemcpy (or uses an IPC handle) into virtual memory allocated via malloc() or mmap(). Writing these buffers to non-volatile storage via buffered file descriptors places pages directly into the Linux page cache as dirty memory (PAGE_CACHE_DIRTY). The Linux kernel governs the accumulation of dirty pages using several tunable watermarks, defined in /proc/sys/vm/:

vm.dirty_background_ratio (or vm.dirty_background_bytes): The threshold at which kernel background flusher threads (kworker with writeback context bdi_writeback) wake up and asynchronously flush pages to block devices.
vm.dirty_ratio (or vm.dirty_bytes): The strict ceiling. When unwritten dirty pages exceed this limit, any task executing an I/O write is forced out of the processor runqueue into synchronous direct writeback.

In a dual-socket host provisioning eight H100 SXM5 accelerators, system DRAM is typically 1.5 TB to 2.0 TB. A typical default kernel setting might specify vm.dirty_ratio = 20 and vm.dirty_background_ratio = 10. Under a 2 TB DRAM configuration, up to 400 GB of dirty pages can accumulate before hard throttling occurs, with background writeback initiating at 200 GB. During a checkpoint interval, each GPU dumps roughly 40 GB to 100 GB of sharded optimizer states within seconds. Aggregate host I/O throughput into memory reaches hundreds of gigabytes per second over PCIe Gen5 x16 buses (64 GB/s theoretical per slot). The local storage subsystem—whether a local RAID array of NVMe U.2 SSDs or a network-attached distributed file system (Ceph, Lustre, GPFS) mounted over InfiniBand or RoCEv2—rarely sustains ingestion at this speed. The timeline of kernel failure proceeds along this deterministic path:

Dirty memory quickly exceeds dirty_background_bytes. Kernel flusher threads spin up to issue write requests to block storage device queues.
Ingestion bandwidth from the GPU far outstrips the physical block layer's queue depth and drain rate. Dirty pages reach dirty_bytes.
The kernel transitions calling threads directly into balance_dirty_pages_ratelimited() inside mm/page-writeback.c.
The user process executing the checkpoint write is placed into an uninterruptible sleep state (TASK_UNINTERRUPTIBLE, state D), forcing the thread to synchronously flush dirty pages to the block device until the dirty count drops below the hysteresis threshold.

TEXT

[Training Process: PID 48211]
   | (cudaStreamSynchronize / Host Runtime)
   | 
   +---> write() / pwrite64() to Local/Shared Filesystem
           |
           +---> mm/filemap.c: generic_perform_write()
                   |
                   +---> mm/page-writeback.c: balance_dirty_pages_ratelimited()
                           |
                           * DIRTY BYTES EXCEEDS THRESHOLD *
                           |
                           +---> schedule() [State: TASK_UNINTERRUPTIBLE (D)]
                                   |
                                   v
                    Thread Stalled on I/O Completion (Block/SAN)

While the CPU thread is descheduled inside balance_dirty_pages_ratelimited(), it cannot service CUDA runtime command queues, acknowledge NCCL ring synchronization primitives, or issue subsequent work to the hardware execution streams. The GPU sits completely idle.

NVLink Interconnect Contention and Collective Synchronization Skew

Frontier training relies on continuous tensor exchanges across NVLink 4 meshes inside the node (900 GB/s bidirectional per GPU) and across high-speed InfiniBand/Ethernet NICs (e.g., eight 400 Gbps ConnectX-7 adapters per node) inter-node. Inter-GPU communication is synchronized using NCCL (NVIDIA Collective Communications Library). Distributed checkpoint schemes typically operate in one of two ways:

Stop-the-World Synchronous Checkpointing: The main training loop halts all forward and backward passes. All ranks write states concurrently. If host A is throttled by Linux dirty page writeback for 45 seconds while host B clears its writes in 10 seconds, host B—and all other 12,499 hosts in a 100k GPU cluster—must sit at a global synchronization barrier (MPI_Barrier or torch.distributed.barrier). Cluster MFU drops proportionally to the slowest straggler.
Naive Non-Blocking Background Checkpointing: The application spawns a host-side worker thread to flush states while the GPU immediately resumes computation. This creates memory bus and interconnect starvation.

In the non-blocking case, asynchronous background transfers often trigger bus saturation. When checkpoint data is pulled from GPU HBM to host memory during active training, the bulk memory transfers traverse the same PCIe Gen5 switches or host root complexes utilized by GPUDirect RDMA transfers between the NICs and GPUs. Furthermore, if the model architecture utilizes ZeRO-3 (FSDP), layer weights are re-gathered across nodes via NCCL AllGather operations during the forward pass and released immediately afterward. If the host CPU is stalled in TASK_UNINTERRUPTIBLE due to page cache writeback, it fails to execute the kernel launch commands necessary to orchestrate NCCL communication steps on time. The high-speed NVLink pipelines experience pipeline bubbles, starving the Tensor Cores.

Instrumentation of Kernel Stalls and Page Allocation Latency

To diagnose whether node-level performance jitter stems from Linux dirty writeback rather than networking hardware drops or CUDA kernel regressions, engineers must measure kernel function latency directly. The following bpftrace program traces the duration of time spent by training processes blocked inside balance_dirty_pages:

#!/usr/bin/env bpftrace

#include <linux/sched.h>

kprobe:balance_dirty_pages
{
    if (comm == "python3" || comm == "torch_worker") {
        @start[tid] = nsecs;
    }
}

kretprobe:balance_dirty_pages
/@start[tid]/
{
    $duration_us = (nsecs - @start[tid]) / 1000;
    @stall_duration_us = hist($duration_us);
    if ($duration_us > 10000) {
        printf("Process %d (%s) blocked in dirty writeback for %u ms
", 
               pid, comm, $duration_us / 1000);
    }
    delete(@start[tid]);
}

interval:s:5
{
    print(@stall_duration_us);
    clear(@stall_duration_us);
}

Executing this trace against a host undergoing synchronous state persistence exposes deep latency distributions. On unoptimized systems, invocations of balance_dirty_pages() regularly hold the CPU thread in an uninterruptible sleep state for anywhere between 800 milliseconds and 12 seconds, dependent on the underlying I/O subsystem queue limits. Simultaneously, host metrics expose high Pressure Stall Information (PSI). The contents of /proc/pressure/memory indicate intense stalls:

TEXT

some avg10=78.21 avg60=45.12 avg300=12.04 total=18492044
full avg10=64.18 avg60=31.89 avg300=8.10 total=14201993

An I/O stall ratio (full) exceeding 60% over a 10-second window indicates that all CPU threads in the cgroup/system are locked waiting on I/O completions, entirely breaking the real-time execution timing demanded by NCCL algorithms. We can cross-reference dirty memory consumption via /proc/vmstat:

BASH

watch -n 0.5 "awk '/nr_dirty/ {print \1, \2} /nr_writeback/ {print \1, \2}' /proc/vmstat"

Under unthrottled checkpoint dumping, nr_dirty climbs rapidly until it plateaus at the dirty limit, at which point nr_writeback remains pegged at the maximum IOPS throughput of the target drive controller while execution pipelines freeze.

Bypassing the Kernel Page Cache with Direct I/O and Pinned NVRAM Staging

To resolve this structural bottleneck, checkpoint serialization must be split into two disjoint phases: Microsecond HBM-to-Host Staging and Asynchronous Host-to-Non-Volatile Storage Ingestion.

TEXT

+-------------------------------------------------------------------------+
| GPU HBM3 (Accelerator Memory)                                           |
+-------------------------------------------------------------------------+
      | 
      | cudaMemcpyAsync (Pinned Host DRAM / CXL NVRAM via PCIe Gen5)
      v
+-------------------------------------------------------------------------+
| Host-Pinned Bounce Buffer (Bypasses Linux Page Cache Entirely)          |
+-------------------------------------------------------------------------+
      | 
      | io_uring with O_DIRECT (Unaligned pages bypassed, direct NVMe write) 
      v
+-------------------------------------------------------------------------+
| Local Storage Array: NVMe Write Buffers / Persistent Fast-Tier          |
+-------------------------------------------------------------------------+
      | 
      | User-space Low-Priority Background Flusher (kthreads uninhibited)
      v
+-------------------------------------------------------------------------+
| Remote Distributed Storage Fabric (Ceph / GPFS / S3 Target)             |
+-------------------------------------------------------------------------+

By executing direct memory copies from GPU HBM into pinned, non-pageable host memory (cudaHostAlloc or cudaMallocHost), the transfer executes completely via hardware DMA engines without touching the OS virtual page allocator. Once stored in host-pinned DRAM or host-attached Non-Volatile Dual In-line Memory Modules (NVDIMM/CXL NVRAM), the training process can resume forward and backward passes immediately. To persist this pinned buffer to persistent media without triggering the page cache mechanisms outlined above, the persistence daemon must leverage Direct I/O (O_DIRECT) coordinated through modern asynchronous I/O frameworks such as Linux io_uring.

Implementation: Zero-Copy Asynchronous NVRAM/NVMe Staging Engine

The following C code provides an optimized I/O engine that uses io_uring with O_DIRECT to write checkpoint state directly to block devices, entirely circumventing the kernel dirty writeback mechanism:

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <liburing.h>
#include <sys/stat.h>
#include <cuda_runtime.h>

#define QUEUE_DEPTH 64
#define BLOCK_SIZE (2 * 1024 * 1024) // 2MB Direct I/O Alignment

struct checkpoint_io_request {
    struct iovec iov;
    size_t offset;
};

int init_direct_writer(const char *filepath) {
    int fd = open(filepath, O_CREAT | O_WRONLY | O_TRUNC | O_DIRECT, 0644);
    if (fd < 0) {
        perror("open O_DIRECT");
        return -1;
    }
    return fd;
}

void stage_gpu_checkpoint_async(void *d_tensor_data, void *h_pinned_buffer, 
                                size_t size, cudaStream_t stream) {
    // Phase 1: Rapid HBM to Pinned Host DRAM Offload
    cudaMemcpyAsync(h_pinned_buffer, d_tensor_data, size, cudaMemcpyDeviceToHost, stream);
}

int flush_to_nvme_uring(int fd, void *h_pinned_buffer, size_t total_size) {
    struct io_uring ring;
    struct io_uring_sqe *sqe;
    struct io_uring_cqe *cqe;
    size_t bytes_submitted = 0;
    size_t bytes_completed = 0;

    if (io_uring_queue_init(QUEUE_DEPTH, &ring, 0) < 0) {
        perror("io_uring_queue_init");
        return -1;
    }

    while (bytes_completed < total_size) {
        // Submit write blocks up to QUEUE_DEPTH
        while (bytes_submitted < total_size && (bytes_submitted - bytes_completed) < (QUEUE_DEPTH * BLOCK_SIZE)) {
            size_t chunk_size = total_size - bytes_submitted;
            if (chunk_size > BLOCK_SIZE) chunk_size = BLOCK_SIZE;

            sqe = io_uring_get_sqe(&ring);
            if (!sqe) break;

            io_uring_prep_write(sqe, fd, (char *)h_pinned_buffer + bytes_submitted, 
                                chunk_size, bytes_submitted);
            
            bytes_submitted += chunk_size;
        }

        io_uring_submit(&ring);

        // Harvest at least one completion event
        int ret = io_uring_wait_cqe(&ring, &cqe);
        if (ret < 0) {
            perror("io_uring_wait_cqe");
            break;
        }

        if (cqe->res < 0) {
            fprintf(stderr, "Async write failed: %s
", strerror(-cqe->res));
            io_uring_cqe_seen(&ring, cqe);
            break;
        }

        bytes_completed += cqe->res;
        io_uring_cqe_seen(&ring, cqe);
    }

    io_uring_queue_exit(&ring);
    return (bytes_completed == total_size) ? 0 : -1;
}

Using this implementation, the data is pulled directly from the physical RAM frames reserved for the pinned buffer and pushed to the NVMe controller via the block device interface. The OS page cache does not allocate a single struct page for the transfer. As a result, the kernel's internal counter nr_dirty remains at zero, completely neutralizing the path to balance_dirty_pages_ratelimited().

Linux Kernel Tuning for Residual Checkpoint Traffic

While high-throughput state data should bypass the page cache entirely via O_DIRECT, residual checkpoint data (metadata, configuration graphs, optimizer master states from small non-tensor structures) may still route through POSIX interfaces. To ensure the kernel never allows these residual writes to interrupt compute tasks, host sysctls must be explicitly bounded.

Sysctl Property	Unoptimized Default Setting	Deterministic Real-Time Setting	Architectural Mechanism
`vm.dirty_background_ratio`	10%	0 (Disabled)	Prevents variable percentage calculations on dense RAM nodes.
`vm.dirty_background_bytes`	0	1073741824 (1 GB)	Triggers background writeback early; forces SSD pipelines to run constantly at low saturation.
`vm.dirty_ratio`	20%	0 (Disabled)	Disables relative thresholding which permits hundreds of gigabytes of uncommitted pages.
`vm.dirty_bytes`	0	4294967296 (4 GB)	Imposes a strict hard limit on dirty data accumulation before invoking thread writeback throttling.
`vm.dirty_expire_centisecs`	3000 (30 seconds)	500 (5 seconds)	Decreases the maximum duration dirty pages can age before writeback marking occurs.
`vm.dirty_writeback_centisecs`	500 (5 seconds)	100 (1 second)	Forces the kernel flusher wake-up period to occur with sub-second frequency to avoid spikes.
`kernel.sched_rt_runtime_us`	950000	1000000	Allows high-priority real-time worker threads to fully utilize cores without 50ms OS starvation throttles.

To apply these settings deterministically on compute nodes:

INI

# /etc/sysctl.d/99-checkpoint-latency.conf
vm.dirty_background_ratio = 0
vm.dirty_background_bytes = 1073741824
vm.dirty_ratio = 0
vm.dirty_bytes = 4294967296
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.vfs_cache_pressure = 50

Applying these controls forces writeback flushing to occur almost immediately upon page dirtification, flattening the I/O spikes into a continuous, low-amplitude stream that stays well below the threshold required to invoke global process throttling.

Impact on Model FLOPs Utilization (MFU) at Cluster Scale

The mathematical impact of checkpoint stalls on frontier training efficiency scales exponentially with node count. MFU is derived from the sustained model compute efficiency over total wall-clock time:

TEXT

             T_compute * Theoretical_Peak_FLOPS
MFU = ------------------------------------------------
        T_wall_clock * Hardware_Target_Peak_FLOPS

Where T_wall_clock includes compute time, pipeline bubbles, and synchronization delays:

TEXT

T_wall_clock = T_compute + T_comm_overhead + T_checkpoint_stall + T_straggler_recovery

In a 100,000 H100 GPU cluster configured for a FP8 training regime with target performance of ~198 ExaFLOPs/sec, training a 1-trillion parameter architecture requires roughly 4 days per model run under perfect theoretical scaling. Because distributed failures occur at an MTBF between 2 and 6 hours across a fabric of this scale, checkpoints are taken every 30 to 45 minutes. The real-world performance profile between standard synchronous checkpointing and decoupled asynchronous NVRAM staging exhibits massive divergences across cluster scale.

Architectural Parameter	Synchronous Page-Cache Buffered Path	Decoupled Host-Pinned Async NVRAM / O_DIRECT	Impact on Infrastructure Mechanics
HBM Execution Freeze Duration	45 to 120 seconds per event	1.8 to 2.4 seconds per event	Execution resumes as soon as HBM->Host DRAM DMA pipeline resolves.
Page Cache Stall Status	Frequent `balance_dirty_pages` invocation	Fully bypassed; Zero dirty pages allocated	Host runtime CPU threads never enter `TASK_UNINTERRUPTIBLE`.
Inter-Node Straggler Window	High (correlated with storage queue variance)	Zero (deterministic local transfer)	Eliminates node-level tail latency during global synchronization.
Sustained Aggregate Cluster MFU	29.5% to 32.1%	38.6% to 41.2%	~9% absolute MFU recovery represents tens of millions of dollars in compute efficiency.
PCIe Bus Contention Profile	Bursty saturations conflicting with NCCL	Regulated DMA via dedicated stream priorities	Prevents inter-GPU RDMA packet loss and packet retransmission loops.

By systematically eradicating page cache interaction during tensor offloading, large-scale training setups avoid OS-induced tail latency events. Staging states across host-pinned memory tiers and leveraging asynchronous Direct I/O write paths transforms distributed checkpointing from an intrusive whole-cluster halt into an unnoticeable, continuously running background pipeline.