Fig 1: Conceptual visualization of the Decoupled Rollout-as-a-Service (RaaS) Infrastructure

Training a multi-turn AI agent requires doing two very different things at the same time. You need to run the agent through thousands of real-world task sequences, generating what researchers call rollout trajectories, and you need to run the reinforcement learning update loop that adjusts the model weights based on what the agent did right or wrong. The problem is that these two workloads have almost nothing in common. Rollout is input/output-intensive. Training is graphics processing unit-intensive. Coupling them inside the same system forces a compromise that ends up serving neither well.

That is the structural problem Nvidia's ProRL Agent is designed to fix. Released in late March 2026 and integrated into the NeMo Gym open-source framework, ProRL Agent separates the rollout lifecycle from the training loop by exposing it as an independent application programming interface service. The paper describes this as a rollout-as-a-service architecture. Existing frameworks such as SkyRL and VeRL-Tool couple these workloads together. ProRL Agent does not.

What the Architecture Actually Does

The system breaks the reinforcement learning training cycle into three asynchronous stages: initialize, run, and evaluate. Each stage communicates through a unified hypertext transfer protocol interface. This means the rollout work, where the agent interacts with tools, browsers, code environments, or whatever external system you are training it on, happens independently of the gradient updates happening on the graphics processing unit cluster.

A few technical choices are worth noting because they point to where the real operational friction has been. The system uses token-in/token-out communication between components, which prevents re-tokenization drift. It uses Singularity-based rootless sandboxing so the rollout environments can run safely in shared high-performance computing settings where you typically cannot run containers as root. Shell command latency in tool execution dropped from 0.78 seconds to 0.42 seconds under the new tool backends. These are not headline numbers, but they matter at scale when you are running thousands of rollout trajectories per training iteration.

The architectural bet is that rollout and training are different enough problems that they should live in different systems. Once you accept that, a lot of other decisions follow.

What the Benchmark Numbers Are Actually Saying

Nvidia validated ProRL Agent on the SWE-Bench Verified benchmark, which measures how well an agent can resolve real-world software engineering tasks. A Qwen3-8B model trained with ProRL Agent went from a baseline score of 9.6 percent to 18.0 percent. A 14B model moved from 15.4 percent to 23.6 percent. These gains came from improved training infrastructure, not from a larger model or a different training dataset.

The more significant claim in the paper is near-linear throughput scaling across compute nodes. Linear scaling in distributed training is hard to achieve. When you add more nodes, coordination overhead typically eats into the gains. The claim that rollout throughput scales near-linearly suggests the decoupled architecture is removing a real bottleneck, not just reorganizing the same work.

For context, ProRL Agent sits in the same family as earlier Nvidia reinforcement learning work. ProRL and ProRL v2 were focused on extending reinforcement learning training duration for single-turn reasoning tasks, with strong results on mathematics and coding benchmarks. The Agent variant extends that philosophy into the multi-turn agentic setting, where the agent has to operate across long sequences of actions and observations rather than answering a single question.

The Broader Context: NeMoClaw and the Agentic Stack

ProRL Agent is open-sourced and lives inside NeMo Gym, which is Nvidia's environment library for reinforcement learning training of agents. This positions it alongside the NemoClaw security agent stack that Nvidia announced at the GTC 2026 conference, and alongside the broader NeMo framework for large language model training and post-training work. Nvidia is building out a coherent set of open-source tools that span the full agent development cycle, from post-training to deployment to the reinforcement learning loops that improve agents after they are in production.

The openJiuwen community released a separate agent framework called JiuwenClaw on the same day. JiuwenClaw addresses a different constraint, specifically the contextual amnesia problem in long-horizon task agents, using a hierarchical memory system and an autonomous skill evolution loop to let the agent self-refine based on failed executions. The two releases together reflect a broader architectural rethink happening across the field: chat-centric AI is giving way to execution-centric AI, and the infrastructure underneath needs to change accordingly.