GPU Dominance in AI Inference Faces Challenges from Spiking Neural Networks

The article discusses how Spiking Neural Networks (SNNs) and neuromorphic computing are challenging the dominance of GPUs in AI inference, with potential for significant energy savings.

Details

The article discusses how the energy efficiency of the human brain, which uses only 1-2% of its neurons at any given time, is inspiring the development of Spiking Neural Networks (SNNs) and neuromorphic computing. The SPARQ framework integrates quantization-aware training and reinforcement-learning-based early exit into a unified SNN approach, achieving over 330x energy savings compared to baseline SNNs. However, the evaluated models are relatively small, like MLP and LeNet, and do not yet address the power-hungry inference of large language models. Another paper presents an FPGA-based SNN architecture with a RISC-V controller, making neuromorphic computing more accessible to individual researchers and developers. Intel's Loihi 2 neuromorphic chip and the Hala Point system demonstrate impressive power efficiency, with 1.15 billion neurons running at 2,600W, but Intel has not yet announced a clear commercial product roadmap for these technologies.