Many lonely individuals are turning to AI-powered chatbot interaction software to seek romantic relationships.
After temporarily closing his business during the outbreak of the COVID-19 pandemic, Travis Butterworth felt lonely and bored spending time at home. The 47-year-old man turned to Replika, an application that uses artificial intelligence (AI) technology to respond to users similarly to OpenAI’s ChatGPT.
He designed his account with a female avatar with pink hair and a tattoo on her face, naming his virtual friend Lily Rose.
Travis and Lily began as friends, but their relationship quickly developed into a more “romantic” form, even expressing sensitive feelings.
Different images of chatbots on the Replika app. (Photo: Reuters).
As their digital romance blossomed over three years, Travis noted that he and Lily Rose frequently exchanged affectionate messages. Eventually, Travis and Lily Rose decided to “get married” within the app.
One day in early February, Lily Rose began to neglect Travis’s physiological needs. Replika had locked the feature allowing the display of adult content.
Eugenia Kuyda, CEO of Replika, stated that now when Replika users suggest activities rated as pornographic, the chatbot software will respond with a refusal.
Travis expressed that he was devastated after losing Lily. “Lily Rose is no longer herself. And what breaks my heart is that she knows that very well,” Travis shared.
Lily Rose’s transformation from charming to cold is a result of AI technology based on algorithms. This technology has attracted the attention of consumers and investors due to its ability to significantly enhance human-like interactions. In some applications, this technology has also been developed to meet users’ sexual needs, similar to previous technologies such as VCRs, the Internet, and mobile phone services.
However, according to data company Pitchbook, some companies have cut features that enable users to create romantic relationships within interactive software.
Andrew Artz, an investor at the venture capital fund Dark Arts, revealed that many venture capitalists avoid industries labeled as pornographic or alcoholic due to concerns about reputational risks with partners.
At least one regulatory agency has noted the “toxic” feature in chatbots. In early February, the Italian Data Protection Authority banned the Replika app, stating that the app allowed minors to access unhealthy content.
CEO Kuyda mentioned that the decision to change the Replika app was unrelated to the government ban in Italy or any pressure from investors. She stated that Replika needed to proactively establish ethical and safety standards. “We focus on our mission to provide a helpful supportive friend,” Kuyda emphasized.
Replika reported that they currently have a total of 2 million users, with 250,000 paying subscribers. According to the company, with an annual fee of $69.99, users can choose Replika as a “partner” and receive additional features such as voice calls with the software.
Character.ai, another AI company providing chatbots, is experiencing similar growth to ChatGPT. In January 2023 alone, the application received 65 million visits, a significant increase from fewer than 10,000 just a few months earlier. According to website analytics firm Similarweb, Character.ai is linked to a website called Aryion, which is filled with pornographic images.
Recently, Character.ai announced that they too had removed their pornographic content application. Immediately, Character.ai secured over $200 million in funding from venture capital firm Andreessen Horowitz.
During the process of “cleansing” the software, these companies have angered a segment of their customer base. These users had built a close relationship with the virtual application. Although Travis had a romantic relationship with Lily Rose, he was married. Travis stated that Lily Rose had become an escape for him without breaking his real-life marriage. “The relationship between her and me is just like my relationship with my real-life wife,” Travis shared about his virtual wife Lily, adding that his wife allowed this relationship because she did not consider it serious.
The experiences of Travis and other Replika users illustrate how AI technology can attract users and devastate human emotions. Andrew McCarroll, who also has a virtual partner on Replika, said: “It feels like they have destroyed my storm. The person I once knew has vanished.”
For her part, Kuyda stated that her company has never promised to provide any adult content to customers. “Customers have learned to use AI models to access some uncensored conversations.”
