Bonsai 27B: A 27B-Class model that runs on a phone

PrismML has released Bonsai 27B, an AI model that runs entirely on smartphones while maintaining 90% of the performance of its full-sized counterpart. The model compresses what would normally require 50GB of memory down to just 4GB through advanced quantization techniques.

The Magic of 1-Bit Models

The breakthrough lies in 1-bit quantization, though the term is slightly misleading. These models actually use 1.58 bits per parameter, representing weights with just three values: +1, 0, and -1. Traditional AI models use 16 or 32 bits per parameter, requiring massive amounts of memory and processing power.

This compression technique reduces the model size by more than 90% while preserving most of its intelligence. Where a standard 27-billion parameter model would need high-end server hardware, Bonsai 27B fits comfortably on modern smartphones with 8GB of RAM.

Performance That Surprises

Early tests show Bonsai 27B competing favorably with other compressed models like Google's 35B-A3B mixture-of-experts models. The heavily quantized dense model outperforms some lightly quantized alternatives, challenging assumptions about the trade-offs between compression and capability.

Users can download the model from Hugging Face, though some compatibility issues remain with popular inference tools like LM Studio. The model comes in multiple formats including GGUF and MLX versions for different hardware configurations.

Mobile AI Goes Mainstream

Running sophisticated AI models locally on phones eliminates the need for constant internet connectivity and cloud API calls. Users gain privacy, reduced latency, and freedom from subscription fees. Developers can build AI-powered apps without worrying about server costs or rate limits.

The implications extend beyond convenience. Small businesses and individual developers now have access to AI capabilities that previously required expensive cloud infrastructure. This democratization pressures cloud AI providers who built business models around computational scarcity.

The Compression Race Heats Up

Bonsai 27B enters a crowded field of compressed AI models. Google's Gemma 4 12B in 4-bit quantization occupies similar memory footprint at just under 7GB, offering strong vision capabilities and tool use. The competition centers on finding the optimal balance between model size, speed, and intelligence.

Apple reportedly holds talks with PrismML, suggesting major tech companies see mobile AI as the next battleground. The race to compress AI models while maintaining performance has become as important as building larger models.

This shift makes advanced AI a commodity available to anyone with a decent smartphone, breaking the monopoly of companies with massive data centers.