← all releases
Perplexity AI·Sep 2, 2026·rolling·open source
Lily
Lily is a high-performance, lightweight inference engine written in Rust and optimized specifically for running the Qwen3.6-35B-A3B model on Apple Silicon. It utilizes custom Metal kernels to bypass standard frameworks like PyTorch or MLX, providing a minimal OpenAI-compatible API for greedy text generation.
featured on blogInference EngineApple SiliconRustMetal
on the blogPerplexity Releases Lily for Local Qwen 35B Inference on Apple Silicon 5 min
overview
Lily is a high-performance, lightweight inference engine written in Rust and optimized specifically for running the Qwen3.6-35B-A3B model on Apple Silicon. It utilizes custom Metal kernels to bypass standard frameworks like PyTorch or MLX, providing a minimal OpenAI-compatible API for greedy text generation.
key features
- 01Custom Metal kernels for Apple Silicon optimization
- 02Minimal OpenAI-compatible chat completions API
- 03Fixed-block attention and GQA packing for high-speed decoding
- 04Session prefix cache for efficient prompt reuse
- 05Strict, greedy-only decoding for predictable performance
related productsbrowse all →