From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon
Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing.
Key Takeaways
- Hardware is changing rapidly - not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads.
Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago.
- Newer hardware ecosystems (Apple Silicon, custom AI accelerators, and others) are growing fast but lack this depth.
In this work we ask whether that expertise can be transferred automatically.
- 97x speedup compared to the native MLX Attention kernel, and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel; we report the numbers, and how much of the gain comes from the translation layer, in the sections below.
Although we focus on MLX kernels for Apple Silicon, the method is not specific to MLX and applies to any ecosystem where CUDA expertise is transferable.
- MLX runs models correctly but often leaves significant performance on the table.
- The search alternates between selecting the most promising action, instantiating and evaluating code until improvement stagnates, and evolving the world model through insert, update, and prune operations.
Hardware is changing rapidly - not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago. With so much of computing now centered on AI, GPU kernels are a crucial component of its success.
These are the low-level programs that run inside the GPU, and writing efficient ones is far from obvious - it takes years of expertise to get right. Transferring a kernel from one vendor's hardware to another is harder still, and often means rediscovering the same optimizations from scratch. The CUDA ecosystem, for example, has accumulated decades of hard-won kernel expertise: hand-tuned implementations of attention, state space models, and other critical operations representing thousands of engineering hours.
Newer hardware ecosystems (Apple Silicon, custom AI accelerators, and others) are growing fast but lack this depth. In this work we ask whether that expertise can be transferred automatically. We built on K-Search , an evolutionary kernel search framework introduced by Cao et al.
For more details please read the original article at Berkeley BAIR.
Continue Learning
Comments
Sign in to join the conversation