mKernel Library Fuses GPU Communication for AI Workloads
TL;DR. UC Berkeley researchers released mKernel, a CUDA kernel library that fuses multi-GPU and multi-node communication with compute for AI. - mKernel reduces GPU communication overhead, a major bottleneck accounting for up to 47% of AI model execution time. - It integrates NVLink, RDMA, and compute into a single, GPU-driven persistent kernel. - The library enables fine-grained overlap of compute and communication, outperforming host-driven methods.
- GPU communication causes significant bottlenecks in production AI workloads and training times.
- mKernel is a CUDA kernel library developed by UC Berkeley's UCCL project, fusing NVLink, RDMA, and compute.
- It addresses limitations of host-driven communication by shifting control to the GPU, enabling finer-grained overlap.
- The library supports both intra-node NVLink and inter-node RDMA within a single persistent kernel.
- mKernel aims to improve the performance and efficiency of large-scale AI training, especially for MoE models.