Testing 1-bit LLM Bonsai on a Google Pixel 7a
The author tested the 1-bit LLM models Bonsai-8B and Bonsai-1.7B on a Google Pixel 7a using the Linux terminal app. They found the models ran slowly and the terminal app crashed frequently, making practical use difficult.
Why it matters
This article provides real-world insights into the challenges of running large AI models on mobile devices, which is an important area of research and development.
Key Points
- 1Tested Bonsai-8B and Bonsai-1.7B 1-bit LLMs on a Google Pixel 7a
- 2Used the Linux terminal app on Android 16
- 3Encountered performance issues and app crashes, making practical use difficult
Details
The author was interested in testing the Bonsai-8B and Bonsai-1.7B 1-bit large language models, which are claimed to run on a Raspberry Pi 4B. They set up the Linux terminal app on their Google Pixel 7a running Android 16, built the llama.cpp project, and ran the two Bonsai models. However, they found that the models produced output too slowly to be practical, and the Linux terminal app crashed frequently, requiring a full recovery each time. The author was only able to test a few basic prompts before the issues made further testing difficult.
No comments yet
Be the first to comment