NVIDIA Nemotron 3 Super: The Open AI Model That Just Beat GPT on Coding (March 2026)

On March 11, 2026, during NVIDIA’s GTC conference, the company released something that quietly rewrote the leaderboard for open AI models. NVIDIA Nemotron 3 Super is a 120-billion-parameter hybrid model that scores 60.47% on SWE-Bench Verified — the most rigorous coding benchmark in AI — beating GPT-OSS-120B’s 41.90% by nearly 20 points. It also delivers 2.2 times the inference throughput at the same time.

For developers and enterprises that have been waiting for an open-weight model that genuinely competes with closed frontier models on real-world coding tasks, this is that announcement.

What Is NVIDIA Nemotron 3 Super?

Nemotron 3 Super is NVIDIA’s first open model built specifically for agentic AI workloads — the kind of multi-step, multi-tool tasks where an AI needs to maintain context across an entire codebase, reason over long documents, and take sequences of actions to complete complex goals.

The core specs:

• 120 billion total parameters, 12 billion active per token (MoE architecture)
• 1 million token context window — enough to fit an entire mid-size codebase in a single prompt
• Training across 10+ reinforcement learning environments
• Support for 20 languages and 43 programming languages
• A novel LatentMoE architecture that activates 4x more experts at the same computational cost

The model is available for free on OpenRouter and Hugging Face, and for enterprise deployment through NVIDIA NIM containers.

The Architecture That Makes It Different

Nemotron 3 Super is NVIDIA’s first model to combine three distinct architectural paradigms into a single system, each addressing a different limitation of existing approaches.

Mamba-2 State Space Layers

The majority of sequence processing is handled by Mamba-2 layers — state space models (SSMs) that offer linear-time complexity with respect to sequence length. Traditional attention layers scale quadratically as sequences get longer, making long-context reasoning computationally expensive. Mamba-2’s linear scaling is what makes the 1M token context window practical and fast, not just theoretically possible.

Standard Transformer Layers

Interspersed throughout are standard transformer layers for attention-based reasoning. The combination — SSMs for efficient sequence processing, attention for precise reasoning — gives Nemotron 3 Super the best properties of both architectures without the worst costs of either.

LatentMoE: NVIDIA’s Novel Expert Routing

This is NVIDIA’s most significant architectural innovation in Nemotron 3 Super. LatentMoE compresses input tokens into a latent space before routing them to experts. This compression allows the system to activate four times more experts at the same computational cost as traditional MoE routing. More experts per token means more specialized knowledge brought to bear on each computation — without a corresponding compute increase.

What this means in practice: Nemotron 3 Super gets more inputs from specialized sub-networks on each computation, resulting in richer, more accurate outputs on complex tasks — especially coding and multi-step reasoning — without making the model slower or more expensive to run.

Multi-Token Prediction (MTP)

Nemotron 3 Super uses Multi-Token Prediction for speculative decoding. Instead of predicting one token at a time, MTP predicts multiple future tokens simultaneously and verifies them in one pass. On SPEED-Bench, this achieves an average acceptance length of 3.45 tokens per verification step compared to 2.70 for DeepSeek-R1, translating to up to 3x wall-clock speedup without needing a separate draft model.