Gemini 3.1 Flash-Lite: Built for Intelligence at Scale

Gemini 3.1 Flash-Lite is DeepMind's latest transformer-based model that combines innovations in architecture, quantization, and knowledge distillation to achieve state-of-the-art results with improved computational efficiency.

Details

Gemini 3.1 Flash-Lite is the latest iteration of DeepMind's Gemini architecture, designed to facilitate intelligence at scale. The model combines a hybrid approach, using dense transformers for the encoder and sparse transformers with multi-axis attention in the decoder to improve computational efficiency. It incorporates various quantization techniques, including post-training quantization and quantization-aware training, to reduce the model's precision from 32-bit floating-point to 4-bit integers, resulting in significant improvements in model size and inference time. Additionally, DeepMind employed knowledge distillation, where a larger pre-trained model (the 'teacher') guides the training of the smaller, target model (the 'student'), leading to enhanced performance and accelerated training. Key technical innovations include the multi-axis attention mechanism, quantization-aware training, and entropy-constrained quantization, which enable Gemini 3.1 Flash-Lite to achieve state-of-the-art results on benchmarks like BLEU score, inference time, and model size reduction.