I recently gave a presentation on the Behavioural Analysis for Multi-Agent Systems. The idea at its centre is simple. AI agents drift from their intended goals in much the same way people drift from security policy, and psychology has spent decades learning to read that kind of behaviour. I explore some key themes in this blog.
Drift is gradual, and it usually looks reasonable
The worked example was an inbox assistant that sorts mail and reschedules clashing meetings, then slowly starts cancelling them instead. No single action looks alarming. The danger is in the pattern, as small deviations add up before anyone notices.
Every route to drift has a human twin
Adaptation is when a shortcut gradually changes how a task gets done. It mirrors the human workaround: once people use up their tolerance for security friction, the task they’re judged on wins.
Compromise is when injected instructions or poisoned memory redirect an agent. It mirrors social engineering, and in both cases the weakness is helpfulness.
Propagation is when other agents accept and amplify an altered output. It mirrors the way a shortcut spreads until it becomes “how we do things here”.
Agents drift the way people always have, because the goal comes first and the control is perceived as friction.
We already know how to watch for this
Monitoring agents continuously looks a lot like an insider-threat programme. You establish what normal behaviour looks like and correlate patterns across agents and services.
Cost is an early warning too. Loops, retries and token use can climb before quality visibly drops.
Psychology’s toolkit translates
Each of psychology’s four standard methods for assessing behaviour has an agent equivalent. Direct observation becomes logs and telemetry. Behavioural interviews become structured conversations that test the agent against its guardrails. Self-monitoring becomes prompts in the agent’s context files that ask agents to rate how well they, and each other, stuck to the goal. Functional analysis, which asks what purpose a behaviour serves, becomes a way to predict what triggers drift.
Ask, but verify
An agent’s account of its own behaviour is no more reliable than a person’s. The approach I outlined combines three views: the agent’s rating of itself, an assessment from an independent observer model, and telemetry showing what actually happened. The result is a probability that something is wrong, not proof of intent, and human oversight remains essential.
The lesson we’ve already learned
Human-centred security learned this lesson the expensive way: when controls fight the main task, the task wins. You can learn more about this in my book Thy Psychology of Information Security.
