Move by Move: Measuring and Steering How LLMs Conduct Psychotherapy
2026-08-21 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how large language models behave in therapy-like conversations by creating a simple set of ten types of therapist actions, called an ontology. They tested this system with real therapists and found that models ask many more questions than humans do, teach less, and mostly repeat strategies started by humans rather than starting their own. By using their ontology as a guide, the authors improved the models’ behavior to be more like real therapists without retraining the models. This helps us understand and improve how AI can support emotional conversations.
large language modelspsychotherapytherapeutic movesontologyannotationpsychoeducationhuman-computer interactionbehavioral analysiscounseling transcriptsmodel alignment
Authors
Afonso Baldo, Hugo Pitorro, Areti Vassilopoulos, Anabela C. Areias, Maya D'Eon, Fabíola Costa, Ricardo Rei, Nuno M. Guerreiro
Abstract
Users increasingly turn to large language models for emotional support, yet little is known about how these models actually conduct a psychotherapy interaction. We introduce an ontology of ten therapeutic moves: compact, function-based categories grounded in the MULTI-60 inventory, validated through an annotation campaign with five licensed psychologists, and scaled with a judge-based approach that matches expert agreement. Applying it to real counseling transcripts and model-led sessions, we compare the move distributions between human clinicians and a panel of frontier models. Models over-use inquiry at up to three times the human rate, neglect psychoeducation, and are strongly context-anchored: they carry forward strategies initiated by a human clinician but rarely initiate them themselves. Exposing the ontology as a set of tools roughly halves the mean deviation from the human move distribution and improves turn-level alignment with human therapist by 7-9 percentage points, without any fine-tuning.