Boosting Low-Traffic AI Systems with Zero-Shot Cross-Domain Knowledge Distillation

Google researchers present a case study on using zero-shot cross-domain knowledge distillation to improve the performance of a music recommendation model by leveraging a large-scale YouTube video recommender system.

Details

The paper presents a case study from Google on applying Zero-Shot Cross-Domain Knowledge Distillation (KD) to improve the quality of latency-sensitive ranking models in a low-traffic recommender system. The researchers leveraged a pre-existing, massive-scale teacher model from YouTube's video recommendation platform and distilled its knowledge into a target domain model for a music recommendation application with significantly lower traffic. The 'zero-shot' aspect means the YouTube teacher model was used as-is, without any fine-tuning or adaptation on music-specific data. The paper shares offline evaluation results and live experiment outcomes, demonstrating that this cross-domain transfer is a practical and effective method for enhancing model performance on 'low traffic surfaces'. The core innovation is applying KD across domains in a zero-shot manner, overcoming challenges like feature mismatch, task differences, and architectural alignment between the source and target models. The successful application suggests the teacher model learns high-level, transferable patterns about user intent, content relevance, and engagement dynamics that can be effectively communicated to the student model, even when the surface-level features and tasks differ.