Evaluating Local AI Options for Document Analysis and Writing
The user's company has banned the use of cloud services for security reasons, so they are considering two local AI options - a TiinyAI device with 80GB RAM and 190TOPS, or a Mac Mini M4 with 64GB RAM. The goal is to run 30B language models smoothly for document analysis and report writing.
Why it matters
This decision will impact the user's ability to effectively analyze documents and generate reports without relying on cloud-based AI services, which are now banned by their company.
Key Points
- 1Company banned use of cloud services for security reasons
- 2Considering two local AI options: TiinyAI device or Mac Mini M4
- 3Need portability and ability to run 30B language models for document analysis and writing
Details
The user is looking for a portable AI solution that can run large language models locally, as their company has banned the use of cloud services for security reasons. They are considering two options within a budget of around $1500 - a TiinyAI device with 80GB RAM and 190TOPS, or a Mac Mini M4 with 64GB RAM. The key requirements are portability, the ability to run 30B language models smoothly, and potentially smaller models concurrently. While the TiinyAI device seems powerful, the user is concerned about its startup status and potential shipping delays. The Mac Mini M4 is a more established option, and the user can get a $300 discount by trading in their old Mac. Ultimately, the user is looking for the best choice within their budget to meet their local AI needs for document analysis and report writing.
No comments yet
Be the first to comment