Rotary GPU Runs Large MoE Models on Consumer Laptops
TL;DR. Researchers developed Rotary GPU, enabling a large Mixture-of-Experts model to execute locally on a laptop with limited VRAM. - The Qwen3.6-35B-A3B-class MoE model was successfully run on an RTX 4060 Laptop GPU with 8 GB VRAM. - The system maintained 6.3 GB VRAM usage while generating over 20 tokens per second. - This exploratory work aims to improve accessibility of large AI models beyond datacenter infrastructure.
- Rotary GPU allows large Mixture-of-Experts models to run on consumer hardware with limited VRAM.
- A Qwen3.6-35B-A3B-class MoE model was tested on an NVIDIA RTX 4060 Laptop GPU (8GB VRAM).
- The system achieved a decode throughput of 21.06 tokens/second while using approximately 6.3GB of VRAM.
- The research explores increasing deployment accessibility for large models in resource-constrained environments.