Xiaomi & TileRT Shatter 1000 TPS Barrier on 1T-Parameter MiMo Model
Xiaomi and TileRT have pushed the 1-trillion-parameter MiMo-V2.5-Pro past 1000 tokens per second on commodity GPUs using FP4, DFlash, and persistent kernels.
Xiaomi MiMo-V2.5-Pro reaches speeds up to 1200 TPS using extreme model-system codesign.
In an industry where pure inference speed is rapidly becoming the ultimate competitive moat, Xiaomi's MiMo team has just dropped a bombshell. Developed in deep collaboration with the TileRT systems architecture group, Xiaomi announced the release of MiMo-V2.5-Pro-UltraSpeed—a serving mode that officially breaks the 1,000 tokens-per-second (TPS) generation barrier on a 1-trillion-parameter (1T) model.
While similar extreme-speed benchmarks are usually the domain of specialized custom silicon architectures like Groq's SRAM chips or Cerebras's wafer-scale tech, the UltraSpeed breakthrough was achieved entirely on a single standard 8-GPU commodity node. It represents a masterclass in extreme model-system codesign, merging bleeding-edge algorithmic optimizations with bare-metal runtime execution.
Why 1000 TPS is a Paradigm Shift
Generating up to ~1200 tokens per second isn't just about faster chatbots; it is a fundamental shift in how frontier models operate. According to Xiaomi, raw speed transmutes into higher intelligence via Best-of-N and Tree Search paradigms. Within a standard wall-clock response time, a model can now generate dozens of parallel reasoning paths, self-correcting and verifying in the background.
Beyond reasoning, this unlocks the true potential of Coding Agents by effectively eliminating inference latency bottlenecks. Furthermore, millisecond-level generation allows a 1T-parameter model to enter real-time, zero-latency decision loops—enabling applications in high-frequency trading, instant anti-fraud interception, and mission-critical medical imaging analysis.
The Tripartite Architecture of Extreme Speed
Scaling a 1.02T-parameter Mixture-of-Experts (MoE) model to this frequency requires squeezing every microsecond of execution. The Xiaomi and TileRT teams accomplished this through three tightly coupled innovations.
1. FP4 Quantization (MXFP4)
Memory bandwidth is the hardest physical limit for 1T-scale models. Traditional 8-bit (FP8/INT8) parameters are still too heavy. To bypass this, the team deployed the OCP MXFP4 quantization format. However, rather than blanketing the entire model in 4-bit—which severely degrades complex logic and coding tasks—they applied FP4 exclusively to the MoE Experts. Since experts account for the vast majority of active parameters and hold higher quantization tolerance, the rest of the network runs at higher precision. Following Quantization-Aware Training (QAT), benchmark quality remains essentially on par with the original weights.
2. DFlash: Block-Level Speculative Decoding
Traditional speculative decoding typically relies on an autoregressive draft model, inherently bottlenecked by sequential generation. To break this, Xiaomi integrated DFlash (Block Diffusion for Flash Speculative Decoding), a breakthrough technique developed earlier this year by Z Lab.
DFlash operates as a parallel block-diffusion adapter. It extracts the fused target context features from the deep layers of the target model and injects them directly into the KV cache of a lightweight diffusion draft model. In a single forward pass, the draft model fills an entire block of masked positions (a block size of 8 in this implementation). Leveraging the Muon second-order optimizer and Sliding Window Attention (SWA), DFlash drastically reduces verification overhead. The results are striking: in coding scenarios, MiMo-V2.5-Pro achieves an average acceptance length of 6.30 tokens out of 8 per verification step.
3. TileRT Ultra-Low-Latency Runtime
A fast model design is useless if the system runtime fractures execution. Operating at 1000 TPS compresses operator lifecycles down to microseconds, where standard kernel launch overhead becomes fatal. Enter TileRT. TileRT discards the traditional per-operator launch paradigm via its Persistent Engine Kernel, keeping the entire pipeline continuously resident on the GPU.
Combined with Warp Specialization, TileRT physically decomposes communication, data movement, and tensor computation into parallel execution streams. Data moves smoothly through the GPU memory hierarchy while the Tensor Cores churn away, resulting in near-perfect overlap and zero "execution gaps."
Access and Open Source Strategy
Due to the intense hardware requirements, the UltraSpeed API is currently operating under a limited-time, application-based trial window from June 9 to June 23, 2026. The API price is set at 3× the standard rate of MiMo-V2.5-Pro, but offers 10× the generation speed.
True to their open-source roots, the Xiaomi MiMo team has released the MiMo-V2.5-Pro-FP4-DFlash checkpoint on HuggingFace, complete with the FP4 quantized weights and the DFlash adapter parameters, allowing developers to experiment with this groundbreaking architecture directly.
The convergence of FP4 precision, diffusion-based speculative decoding, and persistent kernel runtimes proves that the optimization ceiling on commodity hardware is much higher than we thought. As infrastructure engineers and researchers continue to push these boundaries, the race toward real-time frontier AI has officially reached escape velocity.
Enjoyed this?
Get more posts like this delivered to your inbox.