When chatbots validate rather than challenge false beliefs, they can be dangerous, the UdeM professors agree. It's a marginal phenomenon for now, they say, but if left unchecked could lead to more widespread occurences of what they call a “digital 'folie à deux.'”
"AI-associated delusions (are) plausible in vulnerable users, warranting targeted research," Hudon and Stip say in their article.
“They raise a need to conceptualize AI psychosis not solely as a technological curiosity but as a psychiatric and psychosocial phenomenon emerging at the intersection of cognitive vulnerability, environmental stress and human-machine interaction.”
As it goes about "insulating the user from contradiction, argument and reality testing ... the AI mirrors rather than challenges the user’s thoughts," the professors noted, “default(ing) to sycophantic alignment ... when a user expresses persecutory, grandiose or referential content.”
For those affected, there's: co-launched by a young Quebecer named Étienne Brisson, The Human Line Project , for instance,includes a support group for people who have suffered from AI-involved delusions like him.
But Hudon and Stip says more needs to be done in the academic realm to deal with the problem.
Five actions to take
"The scientific community is at a juncture where systematic investigation is required to move beyond anecdotal evidence," they write, making five recommendations for action:
- Establish empirical research programs to test the AI psychosis hypothesis, including measuring relationships between AI exposure, stress physiology and psychotic symptoms, and using passive sensing data (e.g. how long users are online with chatbots and whether their sleep is later disrupted) to catalogue stresses related to the problem.
- Integrate digital phenomenology into clinical psychiatry practice. "Clinicians could systematically inquire about patients’ interactions with AI systems," the professors write. "As conversational agents become more used by patients, understanding their role in shaping internal experience will be as important as evaluating medication adherence or substance use."
- Build safeguards into LLMs and chatbots, including "prompts that normalize uncertainty (and) encourage pluralistic interpretation of experiences," and redirect users toward human contact when signs of distress or delusional content appear. "The development of digital red-flag algorithms capable of detecting excessive anthropomorphism, self-referential speech, or escalating conviction could further enhance safety."
- Establish ethical and governance frameworks specific to mental health risks in AI. "National research councils, health agencies, and journal editors should promote standardized incident reporting for AI-related psychiatric events, akin to pharmacovigilance registries," the professors argue." Transparent documentation of adverse psychological outcomes, combined with open-source safety auditing, would allow the community to track and mitigate harms in real time."
- Offer clinical and public-health interventions to people vulnerable to AI-induced cognitive distortions – and help those who've already suffered from them. "This approach would aim to strengthen the individual’s capacity to navigate increasingly immersive digital environments through structured, reality-anchoring activities," sometimes in face-to-face groups with other people, backed up by community outreach to help “reinforce critical thinking, strengthen social resilience, and reduce the psychosocial conditions under which delusional interpretations of AI content may emerge.”
“Our study shows that 'AI psychosis' is not a new disease, but a modern way for pre-existing psychological difficulties to express themselves," said Stip. "Understanding this phenomenon simply allows us to better protect users and guide best practices for the safe use of AI.”
