OpenAI said on Friday it is tightening limits on the weapons-related information ChatGPT provides to teens, even when a question appears harmless, as part of new safeguards aimed at preventing real-world violence.
The move comes as AI companies face growing scrutiny over the risks chatbots pose to young users and the companies’ role in detecting threats of harm.
Here are more details:
OpenAI will restrict practical information on weapons and security systems that could be misused, with the limits applying to teens even when there is no sign of violent intent
READ ALSO:First Algerian Troops Arrive In Niger Where jihadists Threaten Stability
For all users, ChatGPT will be able to automatically end chats where someone appears to be fixating on violence, and OpenAI may temporarily pause accounts after multiple such chats
The company also said it strengthened its age prediction system to move users it estimates are under 18 into the teen experience, and it plans to notify parents if their teen’s account is paused for repeatedly discussing violence
For all users, OpenAI is limiting detail on past mass attacks, including the weapons used and how they were obtained
The company said the new safeguards will begin rolling out in the next few weeks, and it will continue to refine them
OpenAI said earlier this month that teens spend under 15 minutes a day on average on ChatGPT, in its first report on teen use
A report by non-profit Common Sense Media earlier this month found that ChatGPT’s teen version often alerted parents late or not at all when teen accounts held conversations about suicide, self-harm or eating disorders
