NVIDIA Nemotron-3-Super-120B-A12B: Technical Deep Dive

NVIDIA AI-Q Deep Research Agent: A Technical Deep Dive

Executive Summary
NVIDIA AI-Q is an open, modular multi-agent research system that achieved first place on both DeepResearch Bench I (55.95) and DeepResearch Bench II (54.50) as of March 2026. It is built on a planner–researcher–orchestrator architecture powered by a custom fine-tuned Nemotron-3-Super-120B-A12B model, NVIDIA NeMo Agent Toolkit, and LangChain DeepAgents. The system demonstrates that a fully open and configurable stack can outperform closed alternatives on both holistic report-quality and fine-grained factual/analytical benchmarks. Key innovations include evidence-grounded planning, specialist parallel researchers, principle-based trajectory filtering, and an optional ensemble + report refiner layer.

Technical Architecture

The AI-Q deep researcher follows a multi-agent orchestration pattern with three primary roles:

• Orchestrator: Manages the overall research loop, decides when to terminate, and coordinates hand-offs between planning and execution phases.
• Planner: Analyzes the information landscape, decomposes the research query, and produces an evidence-grounded research plan that explicitly identifies required evidence types and analytical lenses.
• Researcher: Executes the plan by dispatching multiple specialist sub-agents in parallel. Each specialist focuses on a distinct analytical perspective (e.g., technical, economic, regulatory, historical). The researcher aggregates evidence and performs initial synthesis.

Each agent can be powered by a different LLM, enabling heterogeneous model usage within a single workflow. An optional ensemble runs multiple independent researcher pipelines in parallel and merges their outputs via another LLM call. A final report refiner stage further polishes the merged report for coherence, citation quality, and readability.

The system is implemented using:

• NVIDIA NeMo Agent Toolkit for config-driven agent composition, tool registration, function calling, and evaluation harnesses.
• LangChain DeepAgents for the multi-phase planner–researcher–orchestrator graph and sub-agent middleware.
• Tooling includes Tavily (web search) and Serper (academic paper search), with mandatory citation tracking.

All components are designed to be portable and self-hostable, aligning with NVIDIA’s emphasis on open, enterprise-ownable AI systems.

Performance Analysis

AI-Q achieved the following top scores:

DeepResearch Bench I evaluates the final report against a reference report using multi-dimensional scoring that rewards synthesis quality and narrative polish. DeepResearch Bench II uses a much more granular evaluation with 70+ fine-grained binary checks per task, emphasizing factual correctness, proper information retrieval, analytical rigor, and clear presentation.

By leading on both, AI-Q demonstrates strength in both surface-level report quality and underlying retrieval + reasoning correctness — a rare combination. Earlier open-source baselines (e.g., LangChain-Open-Deep-Research with GPT-4.1 + Tavily) achieved only 6th place overall, highlighting the performance gap closed by AI-Q’s specialized fine-tuning and architectural refinements.

Model and Training Details

The core intelligence comes from a fine-tuned NVIDIA Nemotron-3-Super-120B-A12B model (a 120B-parameter MoE with 12B active parameters). This model was selected for its strong alignment with multi-step agentic reasoning, tool use, and citation-grounded generation.

Trajectory Generation Process:

• Seed questions: ~17k from OpenScholar, ~21k from ResearchQA, ~2.5k from Fathom-DeepResearch-SFT.
• Generated ~80k full trajectories using an open-sourced GPT-OSS-120B model as the teacher.
• Each trajectory captures complete planner, researcher, and orchestrator behavior, including real API calls to Tavily and Serper.

Filtering:

• Many trajectories failed due to timeout or tool-call limits.
• Successful trajectories were further filtered using a principle-based judge model to retain only high-quality examples.
• Final supervised fine-tuning (SFT) dataset: approximately 67k high-quality trajectories.

This targeted fine-tuning significantly improved long-horizon tool calling, multi-step planning reliability, and synthesis quality compared to base Nemotron models.

Technical Implications

1. Open vs Closed Research Agents: AI-Q proves that open, modular stacks can reach state-of-the-art performance on complex research benchmarks. This reduces vendor lock-in for enterprises needing auditable, customizable research agents.
2. Modularity and Heterogeneity: The architecture’s support for different LLMs per agent role enables cost/performance optimization (e.g., smaller models for simple retrieval specialists, larger models for synthesis).
3. Reproducibility: Because the full stack (NeMo Agent Toolkit + fine-tuned Nemotron weights + LangChain graphs) is open and configurable, other teams can reproduce, extend, or specialize the system for domain-specific research (legal, medical, scientific, etc.).
4. Tool Integration Standards: The emphasis on citation-backed outputs and real search API integration sets a practical standard for production research agents.

Limitations and Trade-offs

• Compute Cost: Running the full ensemble + refiner configuration is significantly more expensive than single-pass inference. The benchmark-topping configuration likely used the maximum optional components.
• Latency: Multi-agent orchestration with parallel specialists and ensemble merging introduces non-trivial latency compared to simpler RAG or single-LLM approaches.
• Data Quality Dependency: Performance is heavily tied to the quality of the ~67k filtered trajectories. The principle-based filtering step, while effective, is itself dependent on judge model quality.
• Search API Reliance: The system depends on external commercial search APIs (Tavily, Serper). Self-hosted search indexes or alternative open search backends would require additional engineering.
• Trajectory Scale: 67k trajectories is modest compared to some frontier agent fine-tuning efforts; further gains may require substantially larger high-quality datasets.

Expert Perspective

The significance of AI-Q lies less in raw benchmark numbers and more in the demonstration that a carefully engineered open-source blueprint can compete with (and surpass) closed systems on demanding agentic research tasks. The combination of evidence-grounded planning, specialist parallel researchers, and targeted SFT on real search trajectories represents a reproducible recipe for building capable research agents.

From an engineering standpoint, the most valuable contribution may be the open NeMo Agent Toolkit patterns and the public validation that principle-based trajectory filtering works well for long-horizon agent tasks. This approach is likely to influence future open agent development far beyond the specific Nemotron model.

The dual-benchmark leadership is particularly noteworthy because the two benches measure somewhat orthogonal qualities (polished narrative vs. granular factual rigor). Achieving both simultaneously validates the architectural choices rather than just benchmark hacking on a single metric.

Technical FAQ

How does AI-Q’s architecture differ from standard LangChain or LlamaIndex research agents?

AI-Q uses a dedicated three-agent loop (orchestrator + planner + researcher) with explicit evidence-grounded planning and parallel specialist sub-agents, rather than simple sequential tool-calling or single-agent ReAct loops. The optional ensemble and report refiner stages further differentiate it from typical open-source implementations.

What is the exact model size and fine-tuning dataset used?

The core model is the Nemotron-3-Super-120B-A12B (120B total parameters, 12B active). It was fine-tuned on approximately 67k high-quality SFT trajectories derived from ~80k generated trajectories across OpenScholar, ResearchQA, and Fathom-DeepResearch-SFT datasets, filtered by a principle-based judge.

Can the system run without NVIDIA hardware or NIM inference?

Yes. While optimized for NVIDIA NIM and Build, the NeMo Agent Toolkit and LangChain components are framework-agnostic. The fine-tuned weights can be served via any compatible inference engine (vLLM, Hugging Face Text Generation Inference, etc.), though performance characteristics may vary.

How much of the performance comes from the fine-tuned model versus the architecture?

Both are critical. The blog post emphasizes that the custom fine-tuning on real search-and-synthesis trajectories was “a major factor,” particularly for long-horizon reliability and citation quality. However, earlier open baselines using frontier closed models performed worse, suggesting the combination of architecture + specialized model is what enabled the top position.

Sources

• How NVIDIA AI-Q Reached #1 on DeepResearch Bench I and II
• NVIDIA AI-Q Tops DeepResearch Bench Research Leaderboard
• Measuring Open-Source Llama Nemotron Models on DeepResearch Bench
• NVIDIA's New Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI
• NVIDIA AI-Q Achieves Top Score for Open, Portable AI Researcher
• DeepResearch Bench GitHub Repository

All technical specifications, pricing, and benchmark data in this article are sourced directly from official announcements. Competitor comparisons use publicly available data at time of publication. We update our coverage as new information becomes available.