Back to News
ZyphraFebruary 5, 2026

Online Vector Quantized Attention (OVQ)

Open Source
LLM
ML

Explore Online Vector Quantized Attention (OVQ)

Visit the official website to learn more and get started

### TL;DR

OVQ is a novel sequence mixing layer developed by Zyphra that aims to balance memory-compute costs and long-context capabilities more effectively than standard sequence mixing layers. It utilizes a sparse memory update to significantly increase memory state size while maintaining efficient training and inference characteristics.

Key Insights & Metrics

Pricing
N/A
Cost structure
Version
1.0
Current release version
Hardware
N/A
Compute requirements
Category
Open Source
Licensing model
Region
United States
Primary region

Key Features

  • Linear compute and constant memory complexity
  • Sparse memory update for increased memory capacity
  • Competitive performance on long-context tasks
  • Efficient training and inference
  • Theoretical basis grounded in Gaussian mixture regression

Related Releases

Protenix

Protenix is an open-source, trainable PyTorch implementation of AlphaFold 3, designed for high-accuracy biomolecular structure prediction. It aims to advance accessible and extensible research tools for the computational biology community. ([github.com](https://github.com/bytedance/Protenix?utm_source=openai))

ByteDanceNov 5
Open

SkyRL tx

SkyRL tx is an open-source library that implements a backend for the Tinker API, enabling users to set up their own Tinker-like services on personal hardware. It supports end-to-end reinforcement learning (RL) and offers significantly faster sampling. The library is designed to be modular, allowing easy prototyping of new training algorithms, environments, and execution plans without compromising usability or speed.

NovaSky AINov 3
Open

Step-Audio-R1

Step-Audio-R1 is an advanced audio language model developed by StepFun AI, designed to enhance audio reasoning capabilities by grounding its reasoning in acoustic features. It introduces Modality-Grounded Reasoning Distillation (MGRD), an iterative training framework that shifts the model's reasoning from textual abstractions to acoustic properties, effectively addressing the 'inverted scaling' problem where performance degrades with longer reasoning. This model has demonstrated superior performance across various audio understanding and reasoning benchmarks, surpassing models like Gemini 2.5 Pro and achieving results comparable to Gemini 3 Pro.

StepFun AINov 29
Open

Z-Image

Z-Image is an efficient image generation foundation model developed by Tongyi-MAI, designed to produce high-quality, diverse, and stylistically versatile images. It serves as a robust base for creators, researchers, and developers seeking advanced image generation capabilities.

Tongyi-MAINov 27
Open

Discussion

0
Upvotes
0
Downvotes
0 reviews

Sign in to leave a review

Reviews

No reviews yet. Be the first to review!

🚀 Join the AI dev community — follow us everywhere

© 2026 MARKTECHPOST AI MEDIA INC. All rights reserved.Terms & ConditionsPrivacy Policy
Beta Mode