Steering Vector

A numerical direction used to influence an AI model's internal representations and encourage a particular behavior.

Last updated

AI & Automation

In plain English

AI models represent information as patterns of numbers. Researchers can sometimes identify a numerical direction associated with a behavior, such as following a certain instruction. By adding or adjusting this direction inside the model, they can influence what it produces. The vector acts like a nudge, not a guaranteed command.

Example

A researcher compares internal model activity with and without a formatting instruction, calculates the difference, and uses that difference as a steering vector during generation.

Sources

  1. arXiv: Steering Language Models With Activation Engineering