Building JarvisOS: An Agentic System for Mobile AI

The article discusses the development of JarvisOS, an agentic system built on top of Cactus, an inference engine for running large language models, computer vision, and speech models on mobile devices. JarvisOS aims to treat the phone as the compute platform, with a system-level orchestration layer and persistent agentic runtime.

Details

The article discusses the development of JarvisOS, an agentic system built on top of Cactus, an inference engine that allows running large language models, computer vision, and speech models on mobile devices. The key goal of JarvisOS is to treat the phone itself as the compute platform, with a system-level orchestration layer and persistent agentic runtime that other apps can plug into. This differs from existing mobile AI frameworks that operate at the app level. To achieve this, the team is building JarvisOS as privileged Android system services, keeping all processing on the device without any cloud routing or API calls. The project requires a custom Android distribution like LineageOS to gain the necessary system-level access and control. Developers interested in contributing will need a high-specification machine (64GB RAM, 400GB storage) to build LineageOS from source. The initial target devices are the Nothing Phone 2 and Google Pixel 6, which have on-device neural processing units (NPUs) for efficient AI inference.