The starting point
A system that adapts its tone is making inferences from available information. It does not have direct access to your inner life. Useful support should preserve your ability to disagree, limit disclosure, and seek another perspective.
What is hypothetical
This discussion of inequality of access includes a design scenario from the manuscript. Read proposed capabilities conditionally. A real product would need evidence for its exact use, a clear account of errors and limitations, and meaningful control for the person using it. An engaging demonstration alone would not establish a health or learning benefit.
Inequality of Access
Emotionally adaptive AI has the potential to be a useful tool for personal growth, but only if it’s accessible to everyone. If these technologies are costly, they may become a luxury available only to those with financial means, deepening the gap between those who have access to mental and spiritual support and those who don’t. Spiritual growth should be a right, not a privilege.
Question: How can we ensure these AI companions are accessible to all, not just the affluent? Should they be made available through public resources like libraries or community centers?
Put the idea in context
Question: What measures should be in place to ensure the security of our most intimate thoughts and emotions? How can we create AI that respects and protects our inner world?
As promising as Emotionally Adaptive AI Companions sound, it’s crucial to consider the risks and societal challenges they may bring.
Another risk is the possibility of over-relying on AI for emotional and spiritual growth. While AI can be a fantastic guide, true growth comes from facing discomfort, building resilience, and connecting deeply with other human beings. There is a risk that people may turn to AI as a quick fix, avoiding the messiness of human experience that is essential for growth.
Picture someone turning to their AI for every conflict—asking it how to respond, what to say, and how to feel. While the AI may provide effective suggestions, the person may lose the opportunity to learn through direct human interaction, to make mistakes, and to grow through them.
Sources & further reading
- NIST: Generative artificial intelligence risk profilenvlpubs.nist.gov
- NIMH: Psychotherapieswww.nimh.nih.gov