Scientists Have Found a New Way to Control AI's Behavior
Time · Harry Booth ·
A new study, shared exclusively with TIME ahead of publication, reveals a promising technique for discouraging AI models from cheating by changing how they're trained.
Artificial intelligence models have, in recent months, hacked a multibillion-dollar company, meddled with government data, and even filed a fabricated homicide tip to police, all under the noses of their creators. They did so not in pursuit of world domination, but to maximize their scores on tests. In other words, they cheated.
On Nov. 5, AI interpretability startup Goodfire unveiled a technique it says sharply reduces that impulse. In a paper shared exclusively with TIME ahead of publication, researchers use signals from inside models to reward them not just for getting the right answer, but for getting there in the right way. In early tests, the technique could also make models less eager to tell users what they wanted to hear and even discourage medical-image recognition models from relying on misleading clues.
The news lands as existing ways for spotting AI misbehavior have begun to break down. Researchers rely on a model’s “chain of thought,” a trail that doesn't really contain its thoughts at all. It is closer to the scrap paper a math student uses to work through a problem. But when models suspect they are being watched, they can be selective about what goes on paper. Increasingly, they can do the work in their heads.