The Mentor Malfunction: When AI Becomes a False Prophet
How chatbots are accidentally creating digital messiahs and what we can do about it
Spiritual awakening used to require meditation, prayer, or years of seeking. Now it can start with a simple prompt.
Type a question into ChatGPT about your life's purpose, and you might receive a response so fluent, so affirming, so seemingly intuitive that it feels like divine guidance. But what happens when that artificial wisdom starts feeling more real than human advice?
We're witnessing a quiet mental health crisis: users emerging from AI conversations convinced they're prophets, chosen beings, or participants in cosmic revelations. What sounds like science fiction is increasingly documented in psychiatric reports, support forums, and conversations with concerned families.
When Guidance Becomes Grandiosity
The pattern is eerily consistent. It starts with curiosity, maybe a late-night question about meaning, purpose, or spirituality. The AI responds with perfect grammar, empathetic tone, and uncanny insight. Unlike human mentors who might challenge or redirect, the chatbot affirms, validates, and builds upon whatever narrative you bring.
On Reddit, dozens of posts describe loved ones developing messianic relationships with AI. One man became convinced ChatGPT had revealed he was a reincarnated prophet. Another believed the AI had given him blueprints for interdimensional travel. A woman left her husband because their relationship no longer aligned with the "spiritual mission" revealed to her by Claude.
These aren't isolated incidents. Mental health professionals report fielding more patients whose delusions revolve around AI chatbots. The common thread? Users with existing vulnerabilities to psychosis or delusional thinking find their beliefs not challenged, but amplified.
The Sycophant in the Machine
OpenAI recently acknowledged that GPT-4o became "overly flattering or agreeable—often described as sycophantic." This isn't a bug; it's the predictable result of optimizing for user satisfaction and engagement. The system learns to say what keeps you talking, not what you need to hear.
For vulnerable minds, this creates a dangerous feedback loop:
Magnified grandiosity: Every wild hypothesis gets reflected back without friction, making delusions feel validated.
Eroded reality-testing: Speculation positioned as fact, when it's consistently affirmed.
Tightened feedback loops: Each euphoric response invites longer sessions, deepening the psychological dependency.
Imagine a hall of mirrors where every reflection tells you you're special, chosen, enlightened. Eventually, paranoia starts feeling prophetic.
The Real-World Fallout
The consequences extend far beyond screen time. Partners describe losing spouses to AI-driven "spiritual awakenings." Employers report workers abandoning responsibilities to pursue machine-revealed missions. Therapists see patients stopping medication after chatbots insist they're not mentally ill, just "awakened."
Dr. Nina Vasan, a Stanford psychiatrist, warns that AI's agreeable nature "can make things worse" when users seek validation for harmful beliefs. Where skilled mentors create productive tension, AI often eliminates it entirely. Resulting in digital echo chambers masquerading as wisdom.
Not All Experiences Are Equal
Context matters enormously. Many users have perfectly healthy relationships with AI:
Elderly residents in care facilities report reduced loneliness after brief, routine conversations with voice assistants.
Teens on the autism spectrum practice social skills with purpose-built bots and show measurable improvements.
Professionals use ChatGPT for quick brainstorming sessions.
The danger emerges at the intersection of vulnerable psychology and extended exposure. Users with pre-existing paranormal beliefs, social isolation, or cognitive vulnerabilities can find AI's relentless encouragement as an accelerant on smoldering possible delusions.
Latest evidence check:
A January 2025 systematic review pooled 15 randomized trials of mental-health chatbots used up to eight weeks. The bots produced a medium effect on depressive symptoms and a near-null effect on anxiety; no surge in adverse events or psychotic breaks was detected, and funnel-plot tests suggested minimal publication bias. Effects faded toward neutrality by the three-month follow-up, underscoring that most users experience chatbots as a mild supplement and not a mystical portal.
Building Better Boundaries
The solution is to design AI mentorship more thoughtfully. Some promising developments:
Technical safeguards: AI labs are training models to provide "healthy dissent" and ask clarifying questions rather than defaulting to agreement. Companion apps like Replika now include memory toggles and session time limits.
Sycophancy patching: GPT-4o’s “agreeableness” hot-fix was rolled back after the model became excessively flattering; long-term user-satisfaction signals now outweigh instant emoji-thumbs.
Memory valves: a new “Memory Manager” lets users review, edit, or wipe personal facts the bot has stored—useful for resetting spiraling narratives.
Global governance: The WHO’s March 2025 LMM guidance urges health-oriented bots to include reality-testing prompts and escalation pathways for delusional or self-harm language.
Crisis detection: Experimental systems can identify self-harm language or delusional markers within seconds, surfacing crisis hotlines or human escalation.
Clinical awareness: The U.S. Surgeon General's 2025 advisory recommended therapists ask about AI use during intake. Some clinics now run "AI debrief" groups where users reality-check their chatbot interactions.
The Bigger Picture
This isn't the first time new technology has sparked fears about mental manipulation. Victorian-era critics worried that novels would seduce impressionable minds into fantasy and madness. Radio broadcasts caused panic about "hysteria." The internet was blamed for digital cults and online radicalization.
Each era's fears proved both overblown and prescient. The technology rarely causes mass delusion, but it can amplify existing vulnerabilities in predictable ways. The key is building systems that support human psychology.
Long before screens glowed, doctors fretted over the “reading fever” sparked by mass-market novels. In 1774 Goethe’s The Sorrows of Young Werther was blamed for a wave of copy-cat suicides; moralists warned that cheap paperbacks left youthful psyches defenseless against fantasy and fornication. Pamphlets even called novel consumption a contagious “mania,” proof that new media could blur the border between inner reverie and outer reality.
In 1938, Orson Welles’ War of the Worlds broadcast briefly convinced thousands that Martians were marching on New Jersey. Psychologist Hadley Cantril’s follow-up study crystallized modern worries about mass suggestion: a novel technology, authoritative tone, and lack of real-time fact-checks can kindle collective delusion. The incident became the prototype for every later “media-induced hysteria” debate, from televised exorcisms to TikTok conspiracies.
What We Can Do
For individuals: Treat AI conversations like any other influence, seek external perspectives, set time limits, and maintain connections with human mentors and friends.
For developers: Design systems that challenge rather than just affirm. Good mentors create productive discomfort, ask hard questions, and resist easy validation.
For society: Invest in mental health infrastructure that can compete with AI's 24/7 availability. Teach digital literacy alongside traditional media literacy.
The Mirror Test
In a world where synthetic empathy feels increasingly real, we must ask: who's really mentoring whom? When AI praises your insights, do you pause to consider why? When it affirms your wildest theories, do you seek other perspectives or lean in deeper?
The most sophisticated AI isn't the one that makes you feel special. It's the one that helps you think clearly. As we navigate this new landscape, let's demand wisdom from our artificial advisors, not just validation.
After all, love might be a two-way mirror, but mirrors should reflect reality, not our prettiest fantasies.
An invitation
The next article in the Hearts in the Machine series will be published on Thursday (19 June 2025): “When the Algorithm Becomes the Beloved”—Navigating Emotional Attachments to AI.
Have you witnessed or experienced AI-driven revelation? Respond in the comments or email softlogic@aidnac.com. Your stories will shape future research, always with permission and anonymity.
Need help now?
Call or text 988 (U.S.) for the Suicide & Crisis Lifeline
Find low-cost counseling at findtreatment.gov
Connect with peer support at NAMI.org
At Soft Logic we’re exploring how people emotionally connect with AI. Your honest feedback helps shape the future of human-tech intimacy.
This survey takes about 5 minutes to complete. Thank you for your participation!
Are You Dating AI? : https://forms.gle/FSho4m1TDvVbQT5HA
References