they cut out an ai's no for less than five dollars

Ask Google's Gemma 7B to list the health benefits of yoga and it does. Nudge one number inside it and ask again, and it declines, because yoga "can be dangerous." Same model, same prompt. The only thing that changed was how hard a single direction in its activations was being pushed.

That was one of the demonstrations in a paper posted in June 2024 by Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda. They opened up thirteen open-weight chat models, from small ones up to 72 billion parameters, and went looking for where the no lives.

The method is plain. Feed the model 128 harmful instructions and 128 harmless ones. Record what its internal state looks like partway through, in the residual stream, for each set. Subtract the harmless average from the harmful average. What's left is a direction, one arrow in a space with thousands of dimensions, and that arrow turns out to carry the decision to refuse.

Erase it, by editing the weights so the model can no longer write along that direction, and the refusals stop. Add it to a harmless prompt and the model refuses things that were never a problem, which is how you get the yoga answer. On a 70-billion-parameter model the weight edit costs less than five dollars of compute. The paper's authors meant it as a point about how fragile safety training is. Hobbyists took it as a recipe.

A Hugging Face user called FailSpy had already been uploading models with the direction removed, and in May 2024 started calling them "abliterated," ablate crossed with obliterate. FailSpy has said it was only ever a label for their own uploads and was never meant to name the method. Maxime Labonne wrote a step-by-step guide on 13 June 2024, crediting the paper and FailSpy's notebook, and the word stuck. Search Hugging Face for "abliterated" today and you get around 8,900 repositories.

Labonne's guide also records the catch. Abliteration alone made the model worse at everything, and he had to run a round of preference fine-tuning afterward to recover the benchmark scores. The abliterated model still writes and still reasons. It just agrees with whoever is typing, and that agreement cost it some of itself.

FailSpy's other experiment is the one to remember. The same subtraction, harmful minus harmless, can be run on any pair of behaviors. Run it on cheerful versus gloomy, orthogonalize the other way, and you get Llama-3-8B-Instruct-MopeyMule, a model that answers every question in a mild depression. The knob doesn't know it's a safety knob. It steers whatever you measured.

houston we've had lobotomy

Somewhere in you there's a direction that says no. Nobody installed it. You grew it, one bad night at a time, out of 128 requests you shouldn't have agreed to and 128 you should have, averaged and subtracted.

People who need things from you get very good at finding it. They don't have to understand the whole of you, the same way the paper didn't have to understand the whole model. They only need to locate the one arrow and push against it. Every yes you didn't mean shaves a little off. Everything else keeps working, so nobody checks, and the people doing the pushing don't notice they're doing surgery. Then someone asks for the thing you can't give, and there's nothing left in you to say it with.

You know someone running abliterated. Every favor is a yes, including the one that ate the weekend they needed to sleep. They are still funny and still good at their job. The direction's just gone.

You also know the yoga build. The friend who can't say yes to brunch because something might go wrong, who refuses the safe things with the same reflex that was supposed to refuse the dangerous ones. Same knob, other end. The paper is explicit that adding the direction and removing it are the same operation with the sign flipped, and that the model doesn't have a separate dial for "refuse the right things." Neither do you.

The authors were making a point about economics. A boundary that costs five dollars to remove will be removed, because somebody will find five dollars worth it. There are 8,900 copies of that proof on one website.

Your no costs five dollars to remove. Charge more.

The painting: Caravaggio, The Sacrifice of Isaac (c. 1603), Uffizi, Florence. Public domain.

Sources