Google Research Releases TimesFM-3 for Multivariate Forecasting
Google Research has launched TimesFM-3, a 300M parameter zero-shot foundation model designed for complex multivariate time-series forecasting tasks.
TimesFM-3 introduces specialized Variate Attention to model the complex interdependencies between multiple time-series variables.
- Google has released TimesFM-3, a 0.3B parameter foundation model optimized for multivariate time-series forecasting.
- The architecture utilizes a Stacked Mixing Transformer with specialized Variate Attention and CPM Iterative RevIN mechanisms.
- Released under a non-commercial license, the model is designed for zero-shot performance across diverse datasets without fine-tuning.
Overview of TimesFM-3
Google Research has announced the release of TimesFM-3, the latest iteration of its Time Series Foundation Model series. This version marks a significant evolution in the project by focusing on multivariate forecasting—a task that involves predicting future values for multiple interdependent variables simultaneously. Traditional time-series models often struggle to capture the complex cross-variable correlations found in real-world data, such as economic indicators, weather patterns, or retail inventory levels. TimesFM-3 aims to address these challenges through a zero-shot approach, meaning it can generate accurate forecasts for new datasets without requiring domain-specific fine-tuning.
Architectural Innovations
The technical foundation of TimesFM-3 is built upon a decoder-only transformer architecture, as detailed in the research documentation. The model contains approximately 300 million parameters and is composed of 20 transformer layers. Each layer features a model dimension of 1280 with 16 attention heads. According to the Hugging Face model card, the architecture is specifically described as a Stacked Mixing Transformer.
Variate Attention and Normalization
One of the primary differentiators in TimesFM-3 is the inclusion of Variate Attention. In multivariate forecasting, the model must not only understand the temporal patterns within a single variable (temporal attention) but also the relationships between different variables (variate attention). This mechanism allows the model to weight the influence of one time series on another, improving accuracy in complex systems where variables are coupled. Furthermore, the model incorporates "CPM Iterative RevIN." Reversible Instance Normalization (RevIN) is a technique used to mitigate the effects of distribution shifts over time, which is a common hurdle in time-series analysis where the mean and variance of data may change significantly between the training and inference phases.
Patching Strategy
TimesFM-3 employs a specific patching strategy to process sequential data. It uses a context patch length of 32 and a forecast horizon patch length of 64. By breaking continuous time-series data into discrete patches, the transformer can more efficiently process long-range dependencies and generate extended forecasts. This approach is consistent with recent trends in time-series research that adapt natural language processing tokenization techniques for numerical data.
Training Data and Methodology
To achieve robust zero-shot capabilities, TimesFM-3 was trained on a massive and diverse corpus of time-series data. The training set includes GiftEvalPretrain, specifically curated to exclude datasets that overlap with the fev-bench evaluation suite to ensure unbiased testing. Key public datasets used in the pretraining phase include Wikipedia Pageviews (with a data cutoff of November 2023) and Google Trends top queries (with a cutoff at the end of 2022). These datasets provide the model with exposure to human-driven temporal patterns, seasonality, and global trends.
In addition to real-world data, Google Research utilized synthetic and augmented data to fill gaps in the training distribution. This multi-source training regimen allows the model to learn a generalized representation of time-series dynamics, ranging from high-frequency fluctuations to long-term cyclical trends.
Practical Application and Availability
TimesFM-3 is released under the TimesFM Non-Commercial License v1.0. The official PyTorch weights and configurations have been made available on the Hugging Face platform, facilitating experimentation by the research community. The model is designed to output various quantiles, with the median forecast located at index 4 of its output array, allowing users to assess the uncertainty and range of potential outcomes rather than just a single point estimate.
The move toward foundation models in time-series forecasting reflects a broader shift in AI research. By leveraging the scaling laws observed in large language models, Google aims to provide a "base" model that reduces the computational overhead for organizations that previously had to train individual models for every unique forecasting task. With its multivariate capabilities, TimesFM-3 is positioned as a tool for complex analytical environments where variables do not exist in isolation.
Enjoyed this?
Get more posts like this delivered to your inbox.
