RT @ctnzr: Announcing NVIDIA Nemotron 3 Super!

Nemotron 3 Super: A Technical Deep Dive

Executive Summary
NVIDIA Nemotron 3 Super is a 120B-parameter open hybrid SSM-Transformer Latent MoE model with ~12B active parameters per token, purpose-built for large-scale agentic AI workloads on Blackwell GPUs. It achieves a new state-of-the-art score of 36 on the AAIndex v4 agentic reasoning benchmark while delivering up to 5× higher throughput and 2.2× faster inference compared to dense Transformer baselines of similar accuracy. The model features native 1M-token context, hybrid Mamba-Transformer-MoE architecture, and is released with full open weights, enabling cost-effective deployment of complex multi-agent systems.

• Hybrid architecture combining State Space Models (Mamba-style), Transformer layers, and Latent Mixture-of-Experts (MoE) for superior efficiency on long-horizon agentic tasks.
• Designed from the ground up for NVIDIA Blackwell, leveraging FP8, FP4, and new Transformer Engine features.
• Positions NVIDIA strongly in the open agentic frontier, competing with dense models 3–5× larger in effective compute.
• Available immediately on major inference platforms with production-ready integrations for OpenHands, OpenCodeInterpreter, and OpenClaw.

Technical Architecture

Nemotron 3 Super departs from pure Transformer or pure Mamba designs by introducing a Hybrid SSM Latent MoE architecture. The model contains 120 billion total parameters but activates only approximately 12 billion parameters per token (roughly 10% activation ratio), delivering the performance of a much larger dense model at a fraction of the compute and memory cost.

The core innovation lies in the interleaving of three component types:

1. Mamba/SSM layers – State Space Model blocks provide linear-time scaling for very long contexts (critical for 1M-token support) and excel at sequential, stateful reasoning typical in agentic loops.
2. Transformer layers – Retained in strategic positions for global attention where high-precision associative recall and complex multi-hop reasoning are required.
3. Latent Mixture-of-Experts – A novel “latent” MoE routing mechanism that operates in a compressed latent space rather than the full hidden dimension. This reduces routing overhead and improves expert specialization for diverse agentic sub-tasks (planning, tool use, self-critique, memory retrieval, etc.).

According to NVIDIA’s technical report, the hybrid design yields significantly better scaling behavior on agentic benchmarks than either pure Mamba or standard MoE Transformers. The model was trained with a custom mixture of synthetic agentic trajectories, long-context reasoning chains, and high-quality web-scale data, with heavy emphasis on reinforcement learning from AI feedback (RLAIF) and process supervision tailored to multi-turn agent workflows.

Context length is natively 1,048,576 tokens, enabled by a combination of Mamba’s linear scaling, efficient attention variants (FlashAttention-3, Ring Attention-style techniques), and architectural sparsity. This makes Nemotron 3 Super one of the first openly available models capable of maintaining coherent state across extremely long agent lifetimes or massive codebases/tools.

The model is explicitly co-designed for the NVIDIA Blackwell architecture (B200, GB200). It makes aggressive use of the new FP4 and FP8 Tensor Cores, the second-generation Transformer Engine, and Blackwell’s enhanced NVLink 5 domain for multi-GPU agent serving. NVIDIA reports that the architecture achieves near-perfect scaling across 8–128 GPUs for both training and inference.

Performance Analysis

NVIDIA positions Nemotron 3 Super primarily on agentic performance rather than traditional academic benchmarks. The headline result is 36 on AAIndex v4, a composite agentic index that evaluates long-horizon planning, tool orchestration, self-correction, and multi-agent collaboration. This score reportedly surpasses much larger dense models, including some 405B-class offerings.

Key performance claims include:

• Up to 5× higher throughput for agentic workloads compared to similarly accurate dense Transformer models.
• 2.2× faster inference speed on Blackwell hardware versus previous-generation H100 deployments.
• Strong gains on long-context agentic tasks due to the hybrid SSM component, with particular improvements in “needle-in-haystack” style tool retrieval over 500k+ token histories.

While full benchmark tables have not been exhaustively published in the initial announcement, NVIDIA’s technical blog and report indicate superior performance-per-dollar on agentic inference compared to Llama 3.1 405B, Claude 3.5 Sonnet (via API), and other frontier models when running complex multi-agent systems.

The active-parameter efficiency (120B total / ~12B active) provides a compelling MMLU-equivalent quality at roughly the compute cost of a 30–40B dense model, while retaining the reasoning depth of larger models through expert specialization and hybrid attention.

Technical Implications

Nemotron 3 Super represents a significant shift in the open model landscape. By openly releasing a hybrid SSM-MoE model optimized for agentic reasoning, NVIDIA is accelerating the move from single-turn chat models to production-grade autonomous agent infrastructure.

For developers and enterprises, this means:

• Ability to run sophisticated multi-agent frameworks (researcher + coder + critic + executor loops) at substantially lower cost than using closed frontier APIs.
• Native support for 1M-token context unlocks new use cases in software engineering (entire repositories + issue history), legal analysis, scientific research, and long-running autonomous workflows.
• The open weights + Blackwell co-design creates a tight hardware-software vertical that may pressure competitors to accelerate their own hybrid architecture roadmaps.

Ecosystem impact is already visible: the model is immediately available on Baseten, Cloudflare, DeepInfra, Fireworks AI, FriendliAI, Inference.net, Lightning AI, Modal, Nebius, and Together AI. NVIDIA has also published getting-started guides for OpenCodeInterpreter, OpenHands, and OpenClaw, signaling strong intent to become the default open foundation for agentic development.

Limitations and Trade-offs

Despite its strengths, Nemotron 3 Super has several trade-offs:

• MoE complexity: While efficient at inference, the latent MoE routing introduces additional engineering overhead for quantization, distillation, and fine-tuning compared to dense models.
• Blackwell optimization: Peak performance requires Blackwell GPUs; while it runs on Hopper, the 2.2× speedup and best FP4/FP8 efficiency are Blackwell-specific.
• Agentic specialization: The model is heavily optimized for agentic reasoning loops. It may not set new records on pure academic benchmarks such as MMLU-Pro or GPQA where dense Transformers still dominate.
• Maturity: As a brand-new architecture class, the long-term stability of hybrid SSM-MoE training dynamics is less proven than pure Transformers. Early adopters may encounter edge cases in extremely long-context routing.

The 12B active parameter count, while efficient, still requires substantial GPU memory for high-batch agentic serving (multiple agents running in parallel with large context).

Expert Perspective

Nemotron 3 Super is one of the most technically ambitious open releases of 2025. By betting on a hybrid Mamba-Transformer Latent MoE design, NVIDIA has demonstrated that architectural innovation remains a powerful lever even at the 100B+ scale. The focus on agentic workloads rather than chasing raw LLM benchmark scores reflects a maturing understanding of where real-world value lies in the post-ChatGPT era.

The decision to open-source the full 120B model (rather than a distilled smaller version) is a strong signal of NVIDIA’s commitment to the open ecosystem, likely intended to drive adoption of Blackwell and CUDA for the next wave of AI infrastructure. If the hybrid architecture proves as scalable as early results suggest, it could accelerate the industry-wide transition toward state-space-augmented MoE models for long-context reasoning.

The combination of 1M context, high agentic benchmark scores, and 5× throughput gains positions Nemotron 3 Super as a strong contender for becoming the default open foundation model for autonomous AI agents in 2025–2026.

Technical FAQ

How does Nemotron 3 Super compare to Llama 3.1 405B on agentic tasks?

Nemotron 3 Super achieves higher scores on AAIndex v4 (36) while using roughly 1/3 the activated parameters and delivering substantially higher throughput. Llama 3.1 405B remains competitive on general knowledge benchmarks, but the hybrid architecture gives Nemotron 3 Super a clear advantage in long-horizon multi-agent workflows.

What hardware is required to run Nemotron 3 Super efficiently?

Peak performance and the advertised 5× throughput gains require NVIDIA Blackwell (B200/GB200) GPUs using FP8 or FP4 precision. The model can run on H100/H200 clusters but with reduced efficiency. A single B200 node can handle meaningful agentic batch sizes thanks to the 12B active parameter footprint.

Is the model suitable for fine-tuning or continued pre-training?

Yes. NVIDIA has released the full model weights and a technical report with training details. The hybrid nature makes continued pre-training more complex than dense models, but the Latent MoE design is reported to be amenable to parameter-efficient fine-tuning techniques and LoRA-style adaptation for domain-specific agents.

Does it support standard vLLM or Hugging Face inference backends?

Initial support is provided through optimized NVIDIA inference stacks (TensorRT-LLM with Blackwell enhancements). Community ports to vLLM, Hugging Face Text Generation Inference, and other backends are expected rapidly given the open weights and high interest.

References

• NVIDIA Technical Blog: “Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning”
• NVIDIA Blog: “New NVIDIA Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI”
• Nemotron 3 Super Technical Report (linked from GitHub repository)

Sources

• NVIDIA Blog – New NVIDIA Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI
• NVIDIA Newsroom – NVIDIA Debuts Nemotron 3 Family of Open Models
• NVIDIA Technical Blog – Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
• VideoCardz – NVIDIA releases Nemotron 3 Super, 120B open model with 1M-token context
• NVIDIA GitHub Repository for Nemotron 3 Super (inferred from announcement)