Italy’s Data Protection Agency said on Friday it was banning artificial intelligence (AI) chatbot company Replika from using the personal data of Italian users, citing risks to minors and emotionally fragile people.
Replika, a San Francisco startup launched in 2017, offers users personalized avatars that speak and listen to them.
It has led the way among English speakers and is free to use, though it generates about $2 million in monthly revenue from selling extra features like voice chats.
CHATGPT LEADS LEGISLATORS TO CALL FOR ARTIFICIAL INTELLIGENCE REGULATION
The ‘virtual friend’ is marketed as capable of improving the emotional well-being of the user.
But the Italian watchdog said that by intervening in the user’s mood, it “may increase the risks for people who are still in a stage of development or in a state of emotional fragility.”
Jen Persson, director of children’s privacy advocacy group Defend Digital Me, told Reuters that tools designed to influence a child’s mood or mental well-being should be classified as health products and therefore should be be subject to strict security standards.
Italy has banned US-based Replika AI from collecting data about users in the Mediterranean nation.
(Luka, Inc./Handout via REUTERS/File Photo)
“These tools are being used with children without much supervision or protection against potential misuse,” he said.
Italian regulators highlighted the absence of an age verification mechanism, such as filters for minors or a blocking device if users do not explicitly state their age.
Replika breaks European privacy standards and processes personal data illegally, as it cannot be based, even implicitly, on a contract that a minor cannot sign, the watchdog said.
THE EUROPEAN UNION WILL AGGREGATE CANCER IMAGING DATA WITH ARTIFICIAL INTELLIGENCE TO ACCELERATE EARLY DIAGNOSIS
Replika did not immediately respond to an email from Reuters seeking comment.
Robert Grosvenor, managing director of consultancy Alvarez & Marsal, said the Italian regulator is unlikely to be the only European regulator considering taking action against companies like Replika.
“While age verification could provide means to protect some of the most vulnerable groups, it does not address the risks and harms that AI-based services and solutions can bring if they are unregulated, in terms of the potential for bias and unintentional discrimination,” he said. .
CLICK HERE TO GET THE FOX NEWS APP
Replika’s developer, US company Luka Inc, must notify Italian authorities of measures taken to implement its requirements within 20 days and could be fined up to $21.8 million, or up to 4% of its annual turnover. globally, according to the statement.
