Extracting Steering Vectors from the J space
Summary
The post explores extracting steering vectors from Jacobian space (J space) to influence an LLM’s behavior using concept tokens. It demonstrates that J-space can yield activation vectors corresponding to simple behaviors (e.g., all-caps output) but notes brittleness and hallucinations for more complex prompts, and it outlines an iterative method to derive a refusal steering vector. The work leverages Qwen-3.1B and publicly available code, highlighting both potential and limitations of activation steering as a prompt-engineering/interpretability technique.