University of Newcastle and HMRI researchers Dr Louise Thornton, Dr Dara Sampson, and Dr Jamin Day from the HMRI Healthy Minds Research Program want to find out whether people prefer chatting with a trained AI chatbot or a human mental health practitioner. 

The researchers are hoping to recruit 100 people into a blinded study wherein participants rate AI responses and human responses to questions regarding mental health and drug and alcohol issues.  

Study co-lead Dr Louise Thornton says, “Chat GPT is getting really sophisticated at natural language communication. It can give well-written, grammatically correct answers to complex questions.

“What we don’t know is how good it is at understanding complex situations that might involve nuance, sarcasm and reading between the lines.  

“Drug and alcohol, as well as mental health support, is sensitive stuff. What we want to find out is if can AI provide empathetic and appropriate responses, like human mental health practitioners can?”

Dr Louise Thornton

In this study, AI is going head-to-head with experienced clinical psychologists and social workers. 

“We’re super excited to see the results because we have a social networking site called Breathing Space that is a place for people to come together. It’s manually moderated which requires a lot of resources. This makes it hard to scale,” says Dr Thornton.  

“We have tried to set up AI to succeed as we trained up an AI model by giving it access to our online treatment programs for mental health and drug and alcohol concerns, then asked it to generate responses to the types of questions and comments we often received on our Breathing Space site.”

“We never want to replace humans but we do want to scale up. What we’re hoping to understand is how can AI be used to augment what we’re doing, in order to extend its impact.” Dr Thornton said.

“We wouldn’t dream of rolling this out until it’s sufficient. We want to be sure it won’t hallucinate or go off track. If we ever used it, there would be an acknowledgement that it’s an AI that’s responding.   

“We know that it’s not just about providing facts. It’s about nuanced, personalised, empathetic, intelligent responses. Can a chatbot build trust and rapport like a human? And does it still work if the person knows it’s a chatbot? 

“This is just a piece of the puzzle. It’s not a panacea. We want to know how we can leverage this technology to improve reach and impact,” says Dr Thornton. 

The AI study is recruiting now

It’s a survey open to people aged over 18. All you have to do is rate the messages generated by both the chatbot and human practitioners.