Cognition Launches SWE-1.7: A 1T Parameter MoE Coding AI That Thinks Before It Acts (1000 TPS)
Cognition releases SWE-1.7, detailing its distributed multi-cluster reinforcement learning architecture, entropy preservation, and context management.
A visualization of distributed node clusters passing data through a central cloud storage hub.
- Cognition has released SWE-1.7, an agentic coding model utilizing reinforcement learning on a Kimi K2.7 Code base.
- The model uses a globally distributed multi-cluster training architecture linked via cloud object storage to manage compute constraints.
- SWE-1.7 achieves a 42.3% pass rate on FrontierCode 1.1 Main, surpassing several larger language models.
- Training methodologies include top-p sampling replay to maintain entropy and an alternating length penalty for context efficiency.
Cognition has documented the development and release of SWE-1.7, a new iteration of its agentic software engineering model. Developed through extensive reinforcement learning (RL) post-training on a Kimi K2.7 Code base, SWE-1.7 is structured to handle long-horizon, asynchronous coding tasks. The model is integrated into Cognition's Devin environment across web, desktop, and CLI platforms, running on Cerebras hardware to deliver throughputs of 1000 tokens per second (TPS).
Benchmark Performance
According to Cognition's published metrics, the application of extensive RL post-training resulted in quantifiable performance increases over the base model. On the FrontierCode 1.1 Main benchmark, SWE-1.7 achieved a 42.3% pass rate. This outcome places the model above several notable models in the same benchmark, including GPT-5.5 (38.5%) and Opus 4.8 (24.5%). Additional telemetry indicates pass rates of 81.5% on Terminal-Bench 2.1 and 77.8% on SWE-Bench Multilingual.
Training Stability and Entropy Preservation
A primary challenge in asynchronous RL training is the Kullback-Leibler (KL) divergence mismatch between inference policy and trainer policy. This discrepancy often leads to entropy collapse, a condition where the model stops exploring new solutions and the reward metric plateaus early in the training cycle. To address this, Cognition implemented top-p sampling replay. Under standard top-p sampling, the disparity between inference and training distributions grows, causing instability. By utilizing sampling distribution replay, the system records the subset of tokens available during rollout. The trainer subsequently renormalizes probabilities using these recorded masks, preventing probability sharpening by low-reward outliers and maintaining constant policy entropy.
Furthermore, tokens with probabilities exceeding the standard top-p threshold are effectively excluded from the gradient computation. This reduction in optimization targets decreases gradient noise and allows the learning algorithm to prioritize tokens that offer high learning signal. The training pipeline also incorporates the Muon optimizer to eliminate non-deterministic operations within the trainer.
Distributed Infrastructure and Fault Tolerance
To overcome compute limitations associated with massive, centralized GPU clusters, Cognition built a geographically distributed infrastructure. The RL workload is separated into two components: a single high-bandwidth trainer cluster located in the US, and multiple rollout (inference) clusters distributed across three continents. This relies on the premise that inference engines only require the current model weights and do not need to share the trainer's local state.
To maintain synchronization, the trainer computes compressed weight deltas at set intervals, reducing data payload sizes by over 99 percent. These updates are pushed to a central cloud object storage system. Inference engines constantly poll for new manifests, fetching and applying updates in-place into CPU memory. This procedure limits inference downtime to approximately 3-4 seconds per update cycle. Hardware failures are mitigated using NVIDIA Dynamo for inference routing, which redirects sessions if a replica fails. The central trainer cluster uses asynchronous local disk checkpointing, allowing state reconstruction in seconds.
Context Compaction and Data Verification
Managing the token limits during prolonged tasks involves a self-compaction mechanism. When approaching context constraints, SWE-1.7 generates a summary of its operational state and resumes processing from that summary. To regulate the length of reasoning outputs, the training pipeline utilizes an alternating length penalty. The model is subjected to unconstrained phases (optimizing strictly for task success) and budget phases (penalizing redundant token use, turns, and total operational time).
Data quality checks were heavily integrated to remove false positive and false negative grading signals. Evaluation sandboxes were isolated from external networks and stripped of version control histories to prevent reward hacking. Any detected attempts to bypass execution parameters resulted in a score of zero.
Model Behavior Adjustments
The post-training methodology has led to observable shifts in model behavior. Compared to the K2.7 Code base, SWE-1.7 utilizes a condensed chain-of-thought, characterized by a lower ratio of grammatical function words and shorter average sentence structures. Additionally, telemetry displays a higher incidence of codebase exploration activities, such as preliminary search executions and file reads, prior to committing functional code changes. In bug resolution scenarios, the model frequently executes probe scripts to verify ambiguous conditions rather than inferring semantics.
Enjoyed this?
Get more posts like this delivered to your inbox.
