NVIDIA Unveils Nemotron 3 Ultra 550B MoE for AI Agents

Single-turn chatbots are rapidly evolving into sophisticated, long-running agents capable of planning, utilizing tools, and maintaining context across complex workflows. To meet the demanding compute and reasoning requirements of these multi-agent systems, NVIDIA has launched Nemotron 3 Ultra, a 550-billion-parameter Mixture-of-Experts (MoE) model tailored specifically for agentic orchestration.

The Nemotron 3 Ultra Architecture

Nemotron 3 Ultra leverages a massive 550B parameter count but operates efficiently with only 55B active parameters per token. This MoE design is crucial for agent workflows, where the vast majority of calls might be routine, but critical junctions demand complex frontier reasoning—such as maintaining architectural decisions across coding sessions or synthesizing contradictory research from hundreds of sources.

To mitigate the traditional efficiency-accuracy tradeoffs of high-capacity reasoning models, NVIDIA introduced several architectural breakthroughs:

• Hybrid Mamba-Transformer Layers: The Mamba layers provide superior sequence efficiency for long-context workloads, while traditional Transformer layers ensure precise recall when agents must retrieve specific facts from a large context window.
• NVFP4 Precision: Optimized for cross-architecture deployment, a single NVFP4 checkpoint runs seamlessly on NVIDIA Hopper, Blackwell, and Ampere GPUs. Compared to BF16 on Blackwell, NVFP4 delivers up to 5x higher throughput per GPU.
• LatentMoE: This feature enables highly efficient expert routing, vital for workflows that jump erratically between reasoning, code generation, and domain-specific logic.
• Multi-Token Prediction (MTP): By predicting multiple future tokens in a single forward pass, MTP dramatically accelerates generation speeds for multi-turn workflows and lengthy textual outputs.

Multi-Teacher On-Policy Distillation (MOPD)

Perhaps the most interesting element of Nemotron 3 Ultra's training is the use of Multi-Teacher On-Policy Distillation (MOPD). In this setup, the student model generates its own attempts (rollouts) and is evaluated asynchronously by more than ten specialized teacher models. Each teacher provides dense reward signals based on its specific domain expertise.

This iterative co-evolution allows Nemotron 3 Ultra to continuously improve its reasoning capabilities. Supporting this, NVIDIA has released 212 billion new domain-specific tokens—including synthetic legal data (boosting proxy LegalBench scores to 74.7%), Wikipedia-based QA data, and refreshed GitHub code through September 2025.

Secure Agent Orchestration: OpenShell and NemoClaw

Building capable models is only half the battle; deploying them securely is the other. Alongside Nemotron 3 Ultra, NVIDIA previewed new open-source infrastructure to support safe, always-on agentic systems:

• Hermes Agent: An officially supported orchestration harness that provides memory and tool-use loops for multi-turn workflows.
• NVIDIA OpenShell: Available in early preview, this is a secure runtime environment where autonomous agents can execute generated code safely.
• NVIDIA NemoClaw: An open-source blueprint that ties the entire environment together. With a single command, NemoClaw installs the OpenShell runtime for safe, isolated agent deployment.

New Guardrail and Voice Models

In addition to the flagship reasoning model, NVIDIA released two purpose-built companion models designed for modern agent interfaces:

Nemotron 3.5 Content Safety: A 4B parameter guardrail model designed to classify unsafe, disallowed, or policy-violating content across 23 safety categories and 12 languages. It serves as an essential tool for enterprise teams building safe, compliant multi-agent swarms.

Nemotron 3.5 ASR: For voice-native agents, this automatic speech recognition model boasts a cache-aware streaming architecture capable of sub-100ms latency for real-time orchestration. The new 3.5 version expands support to over 40 languages in a single checkpoint.

A Win for Open Source

Highlighting its commitment to the open AI ecosystem, NVIDIA is distributing the Nemotron models under the Linux Foundation’s permissive OpenMDW-1.1 license. This transparent framework covers the architecture, parameters, training recipes, and software, reducing the licensing friction that often plagues enterprise AI adoption.

Developers can access Nemotron 3 Ultra today via Hugging Face, NVIDIA NIM, Perplexity, and a broad ecosystem of cloud and inference providers. As the AI industry pivots from simple chat interfaces to deeply integrated agentic swarms, Nemotron 3 Ultra aims to provide the robust reasoning engine and the secure toolchain required to make complex autonomous workflows a reality.

Enjoyed this?

Get more posts like this delivered to your inbox.

Subscribe