Macs Can Now Be Used in Clusters More Efficiently
New updates to macOS and Exo allow Macs to leverage RDMA and MLX over Thunderbolt 5, enabling higher performance in AI model training on Mac clusters.
Why it matters
This news showcases the improving suitability of Macs for AI/ML workloads, which could expand their adoption in the AI research and development community.
Key Points
- 1macOS 26.2 now supports RDMA and MLX over Thunderbolt 5
- 2Devstral2 4-bit model sees a 2.5x increase in tokens/s on a 4-Mac cluster vs. single Mac Studio
- 3Kimi K2 Instruct 4-bit model achieves 33.8 tokens/s on the Mac cluster
- 4DeepSeek v3.1 8-bit model reaches 25.5 tokens/s on the Mac cluster
Details
The article discusses how recent updates to macOS and the Exo software have enabled Macs to be used more efficiently in AI model training clusters. The new macOS 26.2 release adds support for RDMA (Remote Direct Memory Access) and MLX (Mellanox) networking technologies over Thunderbolt 5, allowing for higher-performance inter-node communication in Mac-based clusters. This is demonstrated through benchmarks of various AI models, including Devstral2, Kimi K2 Instruct, and DeepSeek, which show significant performance improvements when running on a 4-Mac cluster compared to a single Mac Studio. The article highlights the growing capabilities of Macs for AI workloads, enabled by software and hardware advancements.
No comments yet
Be the first to comment