Emily, recently widowed, grapples with the overwhelming grief of losing her husband. While surrounded by supportive family and friends, she craves a space to express her emotions freely and openly. An AI chatbot programmed with empathy and grief support tools creates a safe haven for her to share her memories, process her emotions, and find solace in understanding conversations.
Companion chatbots, similar to general-purpose AI chatbots, use vast amounts of training data to mimic human language. However, they also have features like voice calls, picture exchanges, and more emotional conversations that allow them to create deeper connections with the humans on the other side of the screen. Users can typically create their own avatar or choose one that appeals to them.
The Loneliness Pandemic And The Rise of AI Companionship
Many users on online messaging forums devoted to these apps claim to have developed emotional attachments to these chatbots and use them to deal with loneliness, act out sexual fantasies, or get the kind of comfort and support they feel is missing in their real-life relationships.
This is largely due to the widespread social isolation that has already been declared a public health threat in the United States and abroad, as well as the growing number of startups that use enticing online advertisements and promises of virtual characters who offer unconditional acceptance to attract users.
Luka Inc.’s Replika, the most well-known generative AI companion app, was launched in 2017. Others, like Paradot, have come up in the past year, frequently locking away sought-after features like unlimited chats for paying subscribers.
Data Privacy And Other Concerns
Researchers, however, have raised concerns about data privacy, among other things. An analysis of 11 romantic chatbot apps released on Wednesday by the Mozilla Foundation, a non-profit organization, found that practically every app sells user data, shares it for things like targeted advertising, or does not provide adequate information about it in their privacy policy.
The researchers also questioned potential security vulnerabilities and marketing practices, such as one app that claims to be able to assist users with their mental health but distances itself from those claims in fine print. Replika, for its part, claims that its data collection practices adhere to industry standards.
Meanwhile, other experts have voiced concerns about what they perceive as a lack of a legal or ethical framework for apps that promote deep connections but are run by businesses looking to make money. They point to the emotional distress they’ve seen in users when businesses change their apps or abruptly shut them down, as one app, Soulmate AI, did in September.
Last year, Replika sanitized the erotic capability of characters on its app after some users complained that the companions were flirting with them too much or making unwanted sexual advances. It reversed course after an outcry from other users, some of whom fled to other apps seeking those features. In June, the team introduced Blush, an AI “dating stimulator” designed to help people practice dating.
Others are concerned about the more existential threat that AI relationships could potentially replace some human relationships or simply create unrealistic expectations by always being agreeable.
In 2021, Replika came under scrutiny after prosecutors in Britain claimed that a 19-year-old man who planned to assassinate Queen Elizabeth II was influenced by an AI girlfriend he had on the app. However, some studies—which gather data from online user reviews and surveys—have shown some positive outcomes from the app, which claims to consult with psychologists and has marketed itself as something that can also improve well-being.
One recent study by Stanford University researchers looked at roughly 1,000 Replika users—all students—who had been using the app for over a month. It found that the overwhelming majority of them experienced loneliness, while slightly less than half felt it more acutely.
Most people did not say how using the app affected their real-life relationships. A small number said it replaced their human interactions, but roughly three times as many said it improved those relationships.
Since companion chatbots are still a relatively new technology, the long-term effects on humans are unknown.
