llama.cpp Speculative Checkpointing, Ollama Multimodal Tool, MLX vs GGUF for Gemma 4

This article covers significant updates in local AI, including a new speculative decoding enhancement for llama.cpp, an open-source tool for local audio/video analysis with Ollama, and a comparison between MLX and GGUF for running the Gemma 4 model on consumer hardware.

Details

The llama.cpp project has officially merged speculative checkpointing, a technique that speeds up token generation by anticipating subsequent tokens. This integration directly enhances the performance capabilities of models run through llama.cpp on consumer hardware. AmicoScript is a new open-source Python CLI tool that integrates Whisper for local transcription and then leverages Ollama-hosted LLMs for analysis, transforming a local Ollama environment into a powerful personal knowledge management system. A user comparison of running the Gemma 4 model using Apple's native MLX framework against the GGUF format suggests that MLX does not consistently outperform the more established GGUF ecosystem in terms of raw speed or VRAM utilization, highlighting an ongoing debate and practical challenge for users deciding which framework to adopt for local model deployment.