Paste the prompt above into Cursor/Claude and let it generate the skeleton

Title:
How to Build an Agentic Coding Agent with NVIDIA Nemotron 3 Super

Why this matters for builders
Nemotron 3 Super is NVIDIA’s open 40B-parameter hybrid Mamba-Transformer Mixture-of-Experts model that lets you run long-context, high-reasoning agentic workflows with dramatically lower memory and higher throughput than equivalent dense transformers. It interleaves Mamba-2 layers for efficient sequence processing, Transformer layers for deep attention when needed, and MoE routing that activates only ~4B parameters per token. This architecture directly solves the “15× token explosion” problem multi-agent systems create when they repeatedly resend history, tool outputs, and reasoning traces.

When to use it

• You are building autonomous coding agents that must maintain 32k–128k context across multi-step tasks
• You need strong performance on HumanEval, GPQA, and agentic benchmarks while staying under 40 GB VRAM on a single H100/A100
• You want to experiment with hybrid architectures without waiting for closed-source providers
• Your product involves repeated tool calling, code generation, and self-critique loops where cost and latency matter
• You are already comfortable editing code and using Cursor, Continue.dev, or Aider with local/open models

The full process

1. Define the goal (30 minutes)

Write a one-paragraph spec before touching any code.

Example goal:
“Build a persistent coding agent that can own a full GitHub repository. Given a natural-language ticket, it will explore the codebase, create a plan, write and test code changes, run CI, and open a PR — all while keeping the entire conversation + file context inside a 32k–128k window without blowing up memory or token cost.”

Success metrics:

• Completes 3 out of 5 SWE-bench lite tasks autonomously
• Average context length > 24k tokens without OOM
• Runs comfortably on one H100 80 GB or two A100 40 GB

2. Shape the prompt/spec for your AI coding assistant

Use this starter prompt when you ask Cursor or Claude to scaffold the project:

You are an expert Python architect. We are building an agentic coding system powered by NVIDIA Nemotron 3 Super (40B hybrid Mamba-Transformer MoE).

Requirements:
- Use vLLM or Hugging Face Transformers with the official Nemotron-3-Super-40B checkpoint from Hugging Face
- Implement a ReAct-style loop with tool use (grep, read_file, edit_file, run_tests, git_commit)
- Maintain long-term memory via a sliding 32k–128k context window; prefer Mamba-2 layers for history compression
- Add self-critique and plan-revise steps before every code edit
- Expose a FastAPI endpoint so the agent can be called from a web UI or other agents
- Include proper quantization (FP8 or INT4) to fit in 40–48 GB VRAM

Deliver:
1. Full project structure
2. requirements.txt with exact versions
3. agent.py with the main loop
4. tools.py with secure sandboxed implementations
5. vllm_server.py launch script tuned for Nemotron 3 Super

3. Scaffold the project

Run the following commands:

mkdir nemotron-coding-agent && cd nemotron-coding-agent
git init

Key files you should end up with:

• agent.py – main orchestration loop
• tools.py – sandboxed file & shell tools
• memory.py – context manager that optionally summarizes older turns with a smaller Nemotron-3-Nano model
• vllm_server.py – launches the 40B MoE with proper tensor parallelism and FP8

4. Implement carefully (the hybrid advantage)

Configure vLLM for Nemotron 3 Super. Example launch script:

# vllm_server.py
from vllm import LLM, SamplingParams

model = LLM(
    model="nvidia/Nemotron-3-Super-40B",
    tensor_parallel_size=2,          # for two GPUs
    quantization="fp8",              # or "int4" if you need even lower memory
    max_model_len=131072,            # 128k context
    enforce_eager=False,
    gpu_memory_utilization=0.92
)

sampling_params = SamplingParams(
    temperature=0.3,
    top_p=0.95,
    max_tokens=4096,
    stop=["<|endoftext|>"]
)

# Then in your agent loop:
response = model.generate(prompt, sampling_params)

Because only ~4B active parameters are used per forward pass, you can keep the full 128k context active far longer than with a 40B dense model.

5. Validate with concrete tests

Create a validation checklist:

• Run the model with a 64k token prompt (codebase + history) and confirm no OOM
• Execute 5 SWE-bench lite tasks end-to-end
• Measure tokens per task vs baseline Llama-3.1-70B (expect ~40-60% reduction)
• Add a self-consistency check: run the same ticket 3 times and compare final patch quality
• Stress test with 30-turn conversations to ensure Mamba layers are actually helping long-range coherence

Use the official NVIDIA benchmark numbers as reference:
Nemotron 3 Super achieves leading scores on agentic reasoning tasks while delivering 2–3× higher throughput than comparable dense models at 40B scale.

6. Ship it safely

Production hardening steps:

1. Run the vLLM server behind a gateway with request-level rate limiting and prompt guardrails
2. Use NVIDIA NeMo Guardrails or Llama-Guard to filter malicious tool-use attempts
3. Store conversation state in a vector database (Chroma or PGVector) so you can reload context without resending everything
4. Add human-in-the-loop approval before git push or opening PRs
5. Containerize with the official NGC container: nvcr.io/nvidia/nemo:24.10 or the latest vLLM + Nemotron image

Pitfalls and guardrails

What if the model hallucinates file paths or function names?

Nemotron 3 Super is strong but not perfect. Force it to use tools first:
First use the grep or read_file tool to locate the exact symbol before suggesting any edit.

What if context blows up after 15 turns?

Implement a summarization side-channel using Nemotron-3-Nano (much smaller) to condense turns older than 8k tokens. Keep only the summary + last 4 turns in the main 128k window.

What if vLLM complains about unsupported architecture?

Double-check you are using a vLLM version released after December 2025 that includes Mamba-2 and hybrid MoE support. If in doubt, fall back to the Hugging Face transformers library with device_map="auto" and load_in_4bit=True for initial experimentation.

What if performance is slower than expected on single GPU?

The model was designed for multi-GPU tensor parallelism. A single H100 will still work but you should use tensor_parallel_size=1 + pipeline_parallel_size=1 and accept ~30–40% lower throughput.

What to do next

• Replace the ReAct loop with a more advanced framework such as LangGraph or CrewAI once you have baseline working
• Fine-tune a small LoRA on your company’s internal codebase using the Nemotron-3-Super-40B base
• Add multi-agent collaboration: let one instance of Nemotron 3 Super act as “architect” and another as “implementer”
• Publish your agent as an open-source template on GitHub so others can swap in Nemotron 3 Ultra when it drops

Sources

• Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning, NVIDIA Technical Blog, December 2025
• NVIDIA Nemotron 3: Efficient and Open Intelligence (arXiv:2512.20856)
• Inside NVIDIA Nemotron 3: Techniques, Tools, and Data That Make It Efficient and Accurate, NVIDIA Technical Blog
• Official model cards and Hugging Face repository for Nemotron-3-Super-40B

(Word count: 1,187)

This guide gives you a complete, repeatable process to turn the Nemotron 3 Super announcement into a working, production-ready agentic coding system. Ship something real.