LLM Classification Is Feature Engineering
Summary
The article argues that LLMs can function as classifiers when used as features for traditional ML models (e.g., logistic regression). It demonstrates that incorporating LLM verdicts and additional features improves calibration, interpretability, and predictive performance, with experiments on irony detection (SemEval 2018) showing gains from adding both LLM-derived and deterministic features. It advocates a feature-engineering perspective on LLMs to achieve robust, interpretable classification using conventional ML tools and references several related research works.