OpenAI improves its AI-powered content screening tool

Metaverse avatar makers use the tool to make characters ‘stay on-script.’

Ben Wodecki, Jr. Editor

August 16, 2022

2 Min Read

Metaverse avatar makers use the tool to make characters ‘stay on-script.’

OpenAI, the renowned AI outlet behind DALL-E 2, GPT-3 and Codex, has published an updated version of its AI-powered content screening tool that is “faster and more accurate.”

The Microsoft-backed company said the tool can detect “undesired content” such as references to violence, self-harm or content of a sexual nature. The “new and improved” version better protects applications from misuse.

When given a text input, such as social media posts, the tool assesses the content and whether it falls under categories the user wants to filter out.

Called the Moderation Endpoint, the tool was trained to be “quick, accurate, and perform robustly across a range of applications,” according to OpenAI. Its speed and accuracy “reduces the chances of products ‘saying’ the wrong thing, even when deployed to users at scale,” its blog post said. “As a consequence, AI can unlock benefits in sensitive settings, like education, where it could not otherwise be used with confidence.”

The company’s new content moderation tool is available for free to OpenAI API developers. Instead of building and maintaining their own classifiers, developers can make use of current GPT-based classifiers through a single API call.

One outlet already using the tool is Inworld AI, which is developing a platform to create AI-powered virtual characters for metaverse applications.

OpenAI said Inworld is using the tool to make sure its characters “stay on-script.” “By leveraging OpenAI’s technology, Inworld can focus on its core product, which is creating memorable characters.

The tool also can monitor non-API traffic for a fee; this implementation is currently in private beta. OpenAI did, however, release the evaluation dataset under an MIT license for free use in a move to “spur further research in this area.”

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like