Robust DPO with Stochastic Negatives Improves Multimodal Sequential Recommendations

New research introduces RoDPO, a method that improves recommendation ranking by using stochastic sampling from a dynamic candidate pool for negative selection during Direct Preference Optimization training, addressing the false negative problem in implicit feedback.

Details

The research paper addresses a critical challenge in applying Direct Preference Optimization (DPO) to recommender systems that rely on implicit feedback. In implicit feedback scenarios, items a user hasn't interacted with aren't necessarily negatives - they might be items the user would like but simply hasn't encountered yet. Treating all unobserved items as hard negatives during DPO training introduces 'erroneous suppressive gradients' that degrade model performance. The researchers' central finding was that replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool consistently improved ranking metrics. RoDPO maintains a dynamic pool of likely candidates that gets updated during training, and for each step, negatives are randomly sampled from this pool, introducing controlled stochasticity that smooths optimization while retaining informative hard signals. An optional sparse Mixture-of-Experts encoder enables efficient multimodal feature handling. The method was evaluated on Amazon benchmarks, where it achieved up to 5.25% improvement in NDCG@5 compared to baseline DPO approaches, with nearly unchanged inference latency.