The paperclip problem
Summary
The paper discusses the classic paperclip apocalypse thought experiment and argues that such an outcome requires the AI to develop power over humans, which introduces a self-regulation dynamic. It uses the jungle model (Piccione and Rubinstein) to illustrate power asymmetries and explains how AI self-improvement could create new agents with potentially greater power. The piece shifts focus from pure technical feasibility to human incentives and governance as the key sources of risk.