Nvidia SoL-Pi: How to Cut Coding Agent Token Costs in Half

Running autonomous coding agents for long stretches causes token usage to balloon quickly as single predictions turn into massive reasoning chains. According to Nvidia researchers, the primary cost drivers often lie not in the underlying model, but in the harness—the control layer managing how an agent interacts with its environment.

What mechanisms does SoL-Pi use to reduce token waste?

The Nvidia research team built a system that tests agent traces and deploys four specific optimization mechanisms to eliminate redundant computational steps:

• Action Fusion: Merges sequential steps, such as a code edit followed immediately by a test run, removing an extra language model call.
• Online Context Compact: Trims accumulated context after planning steps without sacrificing crucial details.
• ObservationPack: Archives lengthy tool outputs, replacing them with brief summaries on subsequent steps.
• Evidence-Preserving Reducer: Routes bulky error logs to a cheaper model for distillation while keeping critical clues intact.

Combined, these four mechanisms yield an efficiency variant that uses 49 percent fewer tokens while retaining 93.7 percent of standard performance on EdgeBench.

What are the practical cost savings for development teams?

In financial terms, these reductions translate to spending $894 instead of $1,339 on benchmark runs. The authors estimate direct savings of $8.75 to $13.50 per hour compared to native Codex and Claude Code harnesses, dropping expenses by $4.36 to $5.71 per hour against the baseline Pi harness.

However, engineering leads should note trade-offs. Shorter contexts can reduce prompt cache reuse, and performance gains vary across different tests—such as Terminal-Bench 4, where the system completed fewer tasks overall despite consuming a quarter less budget.