Activation Steering

A technique that changes an AI model's behavior by modifying internal numerical signals while the model is processing information.

Last updated

AI & Automation

In plain English

Inside a language model, information flows through layers of numerical calculations. Activation steering lets researchers intervene in those calculations to influence the model's output. Rather than rewriting the model's learned parameters, they adjust certain internal signals while it runs. This can encourage particular behaviors, although the effects aren't always predictable.

Example

Researchers add a specially calculated signal to a language model's internal activations to encourage it to follow formatting instructions more consistently.

Sources

  1. arXiv: Steering Language Models With Activation Engineering