Italy Bans AI Chatbot Replika – EU Hastening AI Regulation

Italy outlawed the US-based artificially intelligent chatbot. It was prohibited by Italy’s Data Protection Agency (GPDP), which said on Friday that it cited its risk to minors and emotionally fragile people, Reuters reported.

Replika was launched in 2017 as a San Francisco startup offering customers personalized avatars that talk and listen to them.

It was quite popular among English speakers and is free to use. However, it was generating $2 million per month via the sale of bonus features such as voice chats.

Replika is marketed as a “virtual friend,” with abilities to improve the emotional well-being of the user.

During account creation, the platform merely requests a user’s name, email account and gender, GPDP claimed.

“And the ‘replies’ served by the chatbot are often clearly in conflict with the enhanced safeguards children and vulnerable individuals are entitled to. Several reviews on the two main App Stores include comments by users flagging sexually inappropriate contents,” said GPDP.

However, the Italian regulator stated that the AI chatbot’s intervention in the user’s mood “may increase the risk for individuals still in a development stage or in a state of emotional fragility.”

The Italian watchdog cited the lack of an age verification system and the failure to filter users who do not provide their true age.

Replika breaches European Privacy Regulations and processes personal data unlawfully as it cannot be based, even implicitly, on a contract that a minor is unable to sign, GPDP said.

“Replika’s developer, the U.S. company Luka Inc., must notify the Italian authority of measures taken to implement its requirements in 20 days and could be fined up to 20 million euros ($21.80 million), or up to 4 percent of its global annual turnover, may be imposed by the SA,” said GPDP in its statement.

EU Rushing AI Regulation

The launch of OpenAI’s ChatGPT has raised concerns globally. EU industry leader Thierry Breton has announced new AI regulations to address the risks posed by the ChatGPT chatbot and AI technology.

OpenAI has recently launched a premium version of ChatGPT after being rated the fastest-growing consumer platform in history.

Breton said the risk posed by ChatGPT and the AI system underscored the urgent need for rules, which he proposed last year in a bid to set the global standard for the technology and are under discussion in Brussels.

“As showcased by ChatGPT, AI solutions can offer great opportunities for businesses and citizens, but can also pose risks. This is why we need a solid regulatory framework to ensure trustworthy AI brd on high-quality data,” said Breton.

Breton is seeking co-operation from OpenAI and developers of high-risk AI systems to ensure compliance with the proposed AI Act.

“Just the fact that generative AI has been newly included in the definition shows the speed at which technology develops and that regulators are struggling to keep up with this pace,” a partner at a U.S. law firm, said.

AI has become a hot topic and has consistently made headlines due to its rapid development in recent years, attracting the attention of lawmakers for law enforcement.

ChatGPT, in particular, has emerged as the best cheating partner for college students and a major headache for academic institutions.
 

Produced in association with MetaNews.