The One-Step Trap (In AI Research)
Summary
Rich Sutton argues that relying on one-step predictive models for long-term planning is flawed due to error accumulation and combinatorial explosion; long-horizon predictions require temporally abstract models like options and GVFs. The article highlights computational challenges and advocates for model-based RL approaches with temporal abstraction.