Self-Hosted AI: Buying GPUs Instead of Renting from Cloud
The article discusses the author's decision to buy 7 RTX 5090 GPUs instead of renting from AWS for their AI-powered image and video generation platform, ZSky. It provides a detailed cost analysis and explains why self-hosting made more financial sense.
Why it matters
This article provides a detailed case study on the financial and technical tradeoffs of self-hosting AI infrastructure versus using cloud-based services, which is a key consideration for AI startups and developers.
Key Points
- 1The author has aphantasia and a traumatic brain injury, which led them to build ZSky as a free tool to generate images and videos for people with similar conditions
- 2Renting GPU instances from AWS would have cost $17,250-$45,120 per month, which was not viable for a free service
- 3Buying 7 RTX 5090 GPUs cost $23,350 upfront, with $834/month in ongoing electricity and cooling costs, which is 20-55x cheaper than the AWS option
- 4The self-hosted hardware can handle the expected daily load of 35,000 image generations and 4,000 video generations
Details
The author explains that they have aphantasia, a condition where they cannot visualize mental imagery, and a traumatic brain injury. This led them to build ZSky, a free AI-powered platform that generates images and videos from text prompts. The author ran the numbers on renting GPU instances from AWS, which would have cost $17,250-$45,120 per month, making it unviable for a free service. Instead, they decided to buy 7 RTX 5090 GPUs for a total of $23,350 upfront, with $834/month in ongoing electricity and cooling costs. This self-hosted approach is 20-55x cheaper than the AWS option. The author's calculations show that the 7 GPUs can handle the expected daily load of 35,000 image generations and 4,000 video generations, making it a feasible and cost-effective solution.
No comments yet
Be the first to comment