What are you looking for ?
VergeIO
RAIDON

SC26: Vast Data Solves Longstanding I/O Contention Challenge Without Interrupting Running Jobs

Offering Job-Level QoS to bring real-time, job-level performance control to large-scale computing environments, enabling administrators to prioritize critical workloads without disruption

Vast Data, an AI operating system company, announced Job-Level Quality of Service (QoS), a new capability coming to the Vast AI operating system that enables schedulers and administrators to dynamically control how individual computing jobs consume shared I/O resources, without stopping or restarting them.For decades, administrators of large-scale computing environments have faced a familiar problem: a single I/O-intensive job can slow down dozens of others, with few options to intervene without disrupting ongoing work. On the world’s largest computing systems, a critical simulation can be delayed by an unrelated analysis workload, even when ample computing power is available. For research institutions and AI infrastructure providers, this competition creates unpredictable performance, operational disruption, and inefficient use of expensive infrastructure.

Job-Level QoS addresses this challenge by extending the job identity defined by workload managers such as Slurm, PBS Pro, and LSF into the Vast data layer. This allows organizations to treat I/O performance as an allocatable resource alongside compute, enforcing individual bandwidth and IO/s policies on a job-by-job basis and adjusting allocations in real time while workloads continue running. Administrators can protect priority workloads and make more productive use of shared infrastructure without interrupting the research and applications depending on it.

“Anyone who’s used a HPC knows the frustration of unpredictable performance when another job consumes more than its share of shared resources,” said Glenn Lockwood, principal technical strategist, Vast Data. “The scheduler knows exactly which jobs are running, but that understanding has historically stopped at the storage system. With Job-Level QoS, we’re closing that gap. The storage system itself can now manage the I/O performance it delivers to individual jobs competing for resources, allowing resource managers and schedulers to prioritize bandwidth and IO/s without having to cancel jobs or change how applications run.”

The capability introduces several new controls within the Vast AI OS:

  • Enforce QoS at the individual job level: The Vast NFS client carries configurable job identifiers with I/O requests, enabling Vast to apply independent bandwidth and IOPS policies to multiple jobs sharing a single compute node, or manage the collective I/O of a job spanning multiple nodes under a single policy
  • Adjust performance without interruption: Administrators can modify read and write performance policies in real time, without restarting jobs, modifying applications, or remounting storage
  • See which jobs are consuming resources: The optional, open-source VNFS Collector provides job-level visibility into bandwidth, IOPS, and errors, with telemetry available through Prometheus, Grafana, or the Vast DataBase
  • Extend controls beyond batch: The underlying architecture also supports Kubernetes and other orchestrators, enabling workload-level performance management across AI training, inference, and scientific computing environments

At SC26, Vast will demonstrate Job-Level QoS in action, showing how administrators can identify competing jobs, dynamically adjust bandwidth allocations, and enforce independent performance policies across shared infrastructure while workloads continue running uninterrupted.

Job-Level QoS is planned for availability with Vast AI OS 5.6 in late January or early February 2027, requiring Vast NFS 4.8 or later. Attendees at SC26 can see the capability demonstrated at the event.

Read also :
Articles_bottom
AIC