Using LLM for confidence scoring is a bad idea
Summary
The article argues that LLM-generated confidence scores are unreliable because such scores are uncalibrated text continuations rather than measured evidence. It explains how calibrated confidence should be derived from observable information and historical outcomes, and outlines a rule-based approach to determine confidence independent of the LLM’s output.