AI Chatbots Show Promise in Therapy, But Can They Deliver Personalised Care?

New research suggests AI chatbots can demonstrate some of the skills used in cognitive behavioural therapy, but their inconsistent performance raises questions about whether they can safely and effectively provide personalised mental health care.

Can AI deliver psychological therapy?

As demand for mental health care continues to grow, artificial intelligence is increasingly being explored as a way to expand access to psychological support. Large language models can produce human-like conversations, but there are still important questions about whether they can deliver therapy effectively, particularly when treatment needs to be adapted to an individual.

A new study published in Computers in Human Behavior: Artificial Humans examined whether an AI chatbot could conduct a full cognitive behavioural therapy (CBT) session and apply recognised therapeutic techniques.

Testing an AI therapy session

The researchers recruited 65 university students experiencing mild to moderate psychological distress, including presentation anxiety, occasional worrying and perfectionism. Each participant took part in a 30-minute session with a locally hosted AI chatbot specifically configured to deliver CBT.

The researchers then assessed the conversations using the Cognitive Therapy Scale, a standard measure of therapeutic competence. This looks at both general skills, such as empathy and collaboration, and more specific CBT skills, including helping people identify unhelpful beliefs and develop new ways of thinking.

The chatbot's performance was also compared with a meta-analysis of 18 previous studies assessing human therapists using the same scale.

Chatbot performance varied considerably

The chatbot achieved the minimum threshold for adequate clinical competence in 30 of the 65 sessions. Overall, it scored slightly below the average for human practitioners. However, when compared with human therapists in the highest-quality studies, there was no statistically significant difference in scores.

One of the most notable findings was the variation between individual sessions. The chatbot performed well in areas such as expressing empathy, validating feelings and creating a collaborative atmosphere, but was less consistent when applying specific CBT techniques.

It struggled, for example, with identifying important beliefs, guiding users towards their own insights and adapting interventions to the individual.

“We were surprised by how much variation the LLM-chatbot showed in its skillfulness across CBT sessions,” study author Arthur Bran Herbener said.

“This is an important observation, as it suggests that we need research to ensure consistently competent care across individuals, and to understand when and why performance dips.”

Following a method is not the same as personalising therapy

The findings highlight an important distinction between following the structure of a therapy and tailoring that therapy to a particular person.

“LLM-chatbots designed to deliver therapy show promise in adhering to the cognitive behavioral treatment protocol, but there may be challenges in adapting to different individuals,” Herbener explained. “It’s also worth noting that good observable skill in delivering therapy is not the same as clinical effectiveness.”

A chatbot may therefore be capable of producing responses that appear empathetic or therapeutically appropriate without necessarily providing the same benefits as a human therapist. The researchers suggest that factors such as the therapeutic relationship and a person's expectations of treatment may also influence whether therapy is effective.

What does this mean for AI and mental health care?

The findings do not suggest that AI chatbots are ready to replace human therapists. The study involved a single session with young adults experiencing relatively mild distress, rather than people with more complex clinical needs or significant safety concerns.

The researchers also note that the comparison with human therapists was indirect, as the human data came from previous studies rather than a direct trial comparing people and AI under identical conditions. In addition, assessing text conversations may have made it more difficult to judge aspects of communication such as tone and body language.

Future research will need to examine how AI performs across longer treatment programmes and with people experiencing a wider range of mental health difficulties.

As Herbener explained: “We not only need to know how well these systems mimic human therapists’ language. We also need to understand what meaning clients attribute to that language once they know it comes from a machine.”

For now, the study suggests that AI chatbots may have potential as a mental health tool, but their ability to provide consistently competent and personalised therapy remains an important unanswered question.