Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
Summary
The paper introduces ProVer, a framework for targeted credit assignment in agentic reinforcement learning. It identifies potentially pivotal decisions within rollout groups using an agentic judge and then verifies these segments by estimating their advantage from differences in terminal success rates before and after the segment. Experiments across ALFWorld, WebShop, and SearchQA show improvements over Group Relative Policy Optimization (GRPO) for multiple model scales, with modest additional generation overhead.