SkillOpt: Microsoft Research's Optimizer That Trains Agent Skills, Not Model Weights

AIDeveloper44 Team

A New Optimization Paradigm for the Agent Era

Fine-tuning large language models is expensive, slow, and locks you into a specific model version. SkillOpt, from Microsoft Research Asia, proposes a different approach: instead of updating model weights, treat a compact natural-language skill document as the trainable parameter, and optimize it through agent experience.

The Core Idea: Text-Space Optimization

SkillOpt frames agent improvement as an optimization problem in text space, deliberately mirroring the structure of deep learning:

• Learning rate → bounded edit budget (prevents destructive rewrites)
• Forward pass → rollout: the frozen agent executes tasks with the current skill
• Backward pass → reflect: the optimizer analyzes success/failure minibatches
• Weight update → bounded skill edits (add, delete, replace operations)
• Validation gate → held-out selection: edits are only accepted if performance improves

The target model, backend, and harness stay completely frozen. Only the skill document changes.

52/52: Best Results Across Every Setting

The benchmark results are striking. SkillOpt achieves best or tied-best performance in 52 out of 52 settings — every combination of model and benchmark, and every harness-benchmark pair. This covers:

• 7 target models including GPT-5.5, GPT-5.4, GPT-5.4-mini, GPT-5.4-nano, GPT-5.2, Qwen3.5-4B, and Qwen3.6-35B-A3B
• 6 benchmarks: SearchQA, SpreadsheetBench, OfficeQA, DocVQA, LiveMathBench, ALFWorld
• Three execution harnesses: direct model calls, Codex, and Claude Code

Average gains range from +9.1% to +24.9% depending on the model, with standout improvements like +49.4% on DocVQA with GPT-5.4-nano and +50.7% on ALFWorld with Qwen3.5-4B.

The Training Loop in Detail

SkillOpt's loop has four phases per epoch:

1. Rollout — the frozen target model executes tasks with the current skill, recording tool calls, verifier feedback, and scores
2. Reflect — failures and successes are analyzed separately by an optimizer model to surface reusable procedures and recurring failure patterns
3. Edit — candidate add/delete/replace operations are merged and ranked under a bounded edit budget
4. Gate — the candidate skill replaces the current one only if held-out selection performance improves

A rejected-edit buffer prevents the optimizer from proposing the same harmful edits repeatedly. A slow update mechanism allows broader meta-skill updates when multiple epochs agree on a direction.

Reusable Skill Artifacts That Transfer

SkillOpt exports a single best_skill.md file. The paper tests transfer across three dimensions:

• Cross-model: a GPT-5.4 LiveMath skill transferred to GPT-5.4-nano gained +15.2%
• Cross-harness: a Codex-trained SpreadsheetBench skill transferred into Claude Code gained +31.8%
• Self-optimizer: GPT-5.4-nano used as its own optimizer improved SpreadsheetBench by +10.4% over baseline

The deployed target model consumes only the final skill file — no optimizer memory, no weight changes, no infrastructure overhead.

Why This Matters

SkillOpt points toward a future where improving an agent's capabilities doesn't require access to training infrastructure or model weights. Organizations running frontier models via API can continuously improve task performance by evolving skill documents — a far more accessible and auditable process than fine-tuning.

Code and paper available at aka.ms/skillopt and arXiv 2605.23904.

Enjoyed this?

Get more posts like this delivered to your inbox.

Subscribe