NitroGen: A Foundation Model for Generalist Gaming Agents

NitroGen is a vision-action foundation model for generalist gaming agents, trained on 40,000 hours of gameplay videos across over 1,000 games. It exhibits strong competence across diverse gaming domains.

Details

NitroGen is a novel approach to developing generalist gaming agents that can perform well across a wide range of games. The key components are: 1) an internet-scale video-action dataset constructed by automatically extracting player actions from publicly available gameplay videos, 2) a multi-game benchmark environment to measure cross-game generalization, and 3) a unified vision-action policy trained with large-scale behavior cloning. This allows NitroGen to exhibit strong competence in diverse gaming domains, including combat encounters in 3D action games, high-precision control in 2D platformers, and exploration in procedurally generated worlds. The model has been shown to transfer effectively to unseen games, achieving up to 52% relative improvement in task success rates over models trained from scratch.