Back to News
Allen Institute for AI (Ai2)•April 23, 2026
OlmPool
Paid
Open Source
Language Models
Long Context
AI Research
Architecture
### TL;DR
OlmPool is a controlled suite of 26 fully open 7B language models from Ai2, each released with 38 pretraining checkpoints. It reveals how four architectural choices — QK normalization, grouped-query attention, sliding window attention, and pretraining context length — compound to degrade long-context extension performance by up to 47%.
Key Insights & Metrics
Pricing
Free (Open Source, Apache 2.0)
Cost structure
Version
1.0
Current release version
Hardware
Open Source (self-hosted)
Compute requirements
Category
Paid
Licensing model
Region
United States
Primary region
Key Features
- 26 fully open 7B models with 38 checkpoints each for long-context architecture research
- Identifies 4 architectural factors that compound to hurt long-context by up to 47%
- Full weights, code, and training data open-sourced under Apache 2.0
Discussion
0
Upvotes
0
Downvotes
0 reviews
Sign in to leave a review
Reviews
No reviews yet. Be the first to review!