What is the place of planning?
Malcolm A. MacIver, Northwestern University
Nathaniel D. Daw, Princeton University
German Espinosa, Northwestern University
Jessica B. Hamrick, DeepMind
Mark K. Ho, Princeton University, NYU
A. David Redish, University of Minnesota
Bradly C. Stadie, Northwestern University
Jane X. Wang, DeepMind
Are there problems that fundamentally require planning and cannot be solved via model-free learning? If so, what defines those problems? If not, does this mean that intelligence has, in principle, already been solved by current deep RL methods? The past decade has shown marked growth in interest in reinforcement learning (RL) and deep learning (DL). Model-free methods have allowed computers to achieve super-perhuman performance in complex domains like board-games, and even in real-time execution like Atari games and Starcraft, suggesting that with sufficient experience and computational resources, cognitively simple architectures (e.g., feedforward NN) can learn rich behavioral repertoires. Where does this leave more complex cognitive processes, in particular, model-based planning? Despite being a central concern at the founding of artificial intelligence (AI) and a key topic in psychology/neuroscience, planning plays only a secondary role in modern deep learning approaches and is primarily conceptualized as one of several techniques for reducing sample complexity in tasks that would otherwise be solved with purely model-free methods. This GAC will discuss the evolutionary process that constrained the specific place planning has in nature and if it is possible to use this information to determine planning’s applicable domains in AI.