A Raspberry Pi + eGPU isn't as dumb as I thought
The author tested a variety of AMD and Nvidia GPUs on a Raspberry Pi CM5 using an eGPU dock, and found that for larger AI models, the performance delta between the Pi and a high-end Intel PC was less than 5%. For the LLaMA 13B model, the Pi was even faster with some Nvidia cards.
Why it matters
This finding suggests that low-cost Raspberry Pi systems can deliver competitive AI performance, potentially opening up new use cases and applications for edge computing.
Key Points
- 1Raspberry Pi CM5 with eGPU dock can deliver competitive performance for large AI models
- 2Performance delta between Pi and high-end Intel PC was less than 5% for many models
- 3Pi was faster than Intel PC for LLaMA 13B model with some Nvidia cards
- 4AMD cards performed much slower on the Pi, likely due to driver issues
Details
The author conducted benchmarks on a Raspberry Pi CM5 system using an eGPU dock, testing a variety of AMD and Nvidia GPUs. Surprisingly, for larger AI models, the performance difference between the Pi setup and a high-end Intel PC build with 64GB DDR5 RAM and PCIe Gen 5 was less than 5%. For the LLaMA 13B model specifically, the Pi was even faster than the Intel PC when using certain Nvidia cards, though the author is unsure why. However, AMD cards performed much worse on the Pi, likely due to driver issues or compatibility problems. The author has published all the benchmark data on GitHub for further analysis.
No comments yet
Be the first to comment