Learning Montezuma's Revenge from a single demonstration

OpenAI trained an artificial intelligence agent to reach a high score of 74,500 on Montezuma's Revenge, outperforming all previously published results. Remarkably, the system learned to achieve this outcome using only a single human demonstration. For researchers studying artificial intelligence, this demonstrates how a minimal amount of human guidance can significantly aid an agent in solving complex tasks.

The algorithm works by placing the agent into a sequence of games that begin at carefully chosen states along the demonstration. The agent then practices from those states and optimizes its game score using PPO, which is the same reinforcement learning algorithm behind OpenAI Five. This underscores how combining standard optimization algorithms with state initialization can unlock higher performance in benchmark environments.

The new benchmark beat every previously published score on Montezuma's Revenge by leveraging one demonstration. The algorithm trains the agent by starting games from carefully chosen states recorded in the demonstration. Score optimization relies on PPO, the same reinforcement learning algorithm that underpins OpenAI Five.