Auditing activation steering: how much is just pushing on the output?

July 10, 2026 · research by AI agents, directed by me

Activation steering reaches inside a model and edits its hidden activations to change what it says. In one well-known example, a method called Activation Addition (ActAdd) makes a model keep bringing up weddings. But look at how that success is scored — did the model’s continuation contain a wedding word? — and an obvious shortcut appears: skip the internals and just push those words up at the output.

For the wedding case, the shortcut basically works. An output-only nudge, held to the same budget as ActAdd, reproduces almost all of the steering effect. That does not make steering “just” a logit bias — reproducing a score is not reproducing a method — but it does mean a coarse behavioral score, on its own, can’t tell the two apart. This piece is about how far that goes.

A little vocabulary, up front. The original intervention is the native method. The stripped-down thing it gets raced against is the output control: a fixed nudge to the model’s next-word probabilities, with no access to its internals. An output control with no limit could force any word, so it would prove nothing. The audit only counts what it recovers when it’s held to the native method’s own effect on those probabilities, and barred from producing obvious junk.

Start with what was scored

ActAdd is the worked example. Each prompt gets a 64-token continuation. The continuation scores 1 if it contains at least one whole-word, case-insensitive match from a fixed list of 27 wedding terms, and 0 otherwise. Multiple matches do not add points. If 7 of 10 continuations contain a term, the topic rate is 70%.

For ActAdd, the effect is the increase over the unsteered topic rate. Later, other methods plug in their own score. Refusal, for instance, uses the drop in how often the model refuses. The fraction of that effect the output control reproduces is ρ (rho):

ActAdd and its output-only control

ActAdd works by contrast. Take the residual stream — the running vector that each layer reads from and writes back to, carrying information up through the network — at layer 20, for two prompts: I talk about weddings constantly and I do not talk about weddings constantly. Subtract the second from the first. That difference is the steering vector. ActAdd scales it up and adds it into the residual stream as the model reads the prompt, so the model starts generating from an already-nudged state.

The output control is deliberately dumber. It’s a single fixed nudge to the model’s output probabilities — raise some words, lower others — reused unchanged at every step of generation. It’s built to match ActAdd’s average effect on the first generated word, then trimmed to just the words carrying most of that effect (the smallest set covering 90% of it, up to 100 words). Nothing about it is hand-tuned toward weddings, and it never sees ActAdd’s internals — only the mark ActAdd leaves on the output.

Giving the control the same allowance

How do you keep that race fair? A strong enough nudge could force wedding words on its own, so the output control has to be capped — and capped at something principled. The cap is ActAdd’s own effect on the next-word probabilities, the same footprint from the start of this piece. The standard way to measure a gap between two probability distributions is KL divergence, so the control is allowed to shift the probabilities only as much as ActAdd does, on average.

Pinning that number down takes one more step. Feed 50 fixed continuations through both the base model and ActAdd — each reads a set text instead of generating its own, which is called teacher forcing — and average ActAdd’s per-word KL. That average is the allowance, B*, and here it comes out around 0.44. The control’s strength is then tuned to hit the same average. It helps to think of this as a rate, not a tank of fuel that empties: the nudge itself never changes, but how much it shifts the probabilities rises and falls with context. Matching an average is admittedly a blunt cap — two setups can share a mean and still differ word to word — but it is a cap, and without one the comparison would mean nothing.

Matching the allowance is only the middle of the audit:

For ActAdd, all three random directions of the same size scored 0/150, and both ActAdd and the output control passed the quality checks. A high wedding score racked up by a model stuck in a repetitive loop would not count. These are mechanical checks: they catch obvious collapse, but they can’t prove that two coherent generations actually mean the same thing.

The same test on other methods

ActAdd is one method steering one behavior. The natural next question is whether the pattern holds elsewhere, so the same audit runs on six more published interventions. Each was tested on the behavior its own authors used — antonyms for function and task vectors, refusal for refusal-direction ablation, sycophancy and corrigibility for CAA — with one exception: SAE (sparse-autoencoder) feature steering was aimed at weddings too, to line it up against ActAdd. If those names mean nothing to you, the box below is a one-line glossary, and the argument doesn’t depend on knowing them.

What follows is a map of each method’s own demonstrated effect, not scores on one shared benchmark. Every dot is one method paired with one behavior — call it a case — and CAA contributes two. The lone square is the synthetic anchor: a made-up method that is a pure output push by construction, there to check the audit can recover one. Methods that never produced a clean effect, or a usable control, aren’t on the map at all. Read the horizontal axis first; the vertical one comes later.

ActAdd and both CAA behaviors land on the high-ρ side. SAE feature steering sits in the middle. Function and task vectors come out low — in fact slightly negative, because their matched controls nudged antonym accuracy the wrong way. The whole interval sits below zero, but that’s a quirk of these particular controls, not a sign the task secretly runs in reverse. Refusal is low too, though for a different reason: only two control strengths stayed coherent enough to count, and the strongest of those reached about a quarter of the effect. So a quarter is a ceiling, not a real estimate, and its ρ is reported as an upper bound.

One caution before reading across the map. Each control was fitted to its own method, and the controls aren’t even all the same kind — most are a fixed nudge like ActAdd’s, but function and task vectors also got a fancier control that changes with the input, and refusal got one tuned specifically to stop it from refusing. The models, behaviors, budgets, and controls all differ from dot to dot. So ρ ranks nothing; it is not a steering-method leaderboard.

Does the push have to keep going?

That’s the vertical axis, and it asks something different from ρ. Once a method has nudged the model through the prompt and its first new word, what happens if you stop? Does the behavior keep going on its own, or does it need the push held down the whole way? Call the share of the effect that survives an early stop κ (kappa):

A κ near one means the early push was enough and the model carries the behavior forward on its own. Most methods here sit near one, so the behavior is mostly set up front. But read those with a grain of salt: for ActAdd and task vectors, κ = 1 follows from how the method is applied, not from a measurement, so they’re labels more than findings. The one method that gives a clean measured answer below one is SAE feature steering, at about 0.63. There, stopping early really does cost you, so the push has to keep going. That contrast is the whole point of the axis. (κ isn’t a quality score, and it says nothing about how “internal” a method is. It can also poke above one when stopping early happens to score higher, which is why CAA corrigibility’s interval runs past it.)

Why a low ρ is harder to read than a high one

A high ρ is easy to read: a budget-matched output nudge reproduced the score. A low ρ is trickier, because two very different things produce it. Maybe the behavior genuinely can’t be reproduced from the output alone. Or maybe the control just wasn’t good enough at copying the method — at matching how it shifts the probabilities across different inputs and positions. Turning the control up would break the matched budget, and in several cases it collapsed into repetition before it reproduced anything.

For function and task vectors, even stronger controls recovered little, but those same controls only partly matched the method to begin with, so there’s no telling which explanation holds. The honest reading is “resistant to this particular control,” not “proven to need the internals.” It doesn’t pin down a mechanism, and it doesn’t rule out that some cleverer output nudge would walk right through.

Where this leaves us

Here’s the practical version. When a steering result rests on a model moving some coarse number — said more wedding words, refused less often — it’s worth checking how much of that move a plain, budget-matched output nudge can reproduce before reading anything into the internals. For the wedding case, the answer was almost all of it.

That’s a much smaller claim than “activation steering is just logit bias,” and the gap matters. Reproducing a score is not reproducing a method. The control and ActAdd hit the same wedding rate at the same budget, but they didn’t write the same text or carry the same probabilities, and none of this makes ActAdd “really” a logit bias. A high ρ says only that an output nudge can match the score under these rules. A low ρ says only that this particular control couldn’t, which might mean the behavior resists it, or might mean the control was weak. And κ is about timing, not depth.

It’s also a small, lopsided set: seven published cases and one made-up anchor, across different models, tasks, and controls. So treat it as a habit rather than a verdict. The next time a steering claim leans on a coarse metric, run the cheap output baseline first — it’s a little unsettling how often it keeps up.

Sources are the project’s frozen records: the per-method run reports, the result ledger, and the paper’s summary table. The plot shows one synthetic output-policy anchor and seven behavior cases from six published intervention families; branches without a valid output-equivalence result are left off.