New and improved content moderation tooling | OpenAI
Skip to main content
Research
Products
Business
Developers
Company
Foundation(opens in a new window)
Log in
Try ChatGPT
(opens in a new window)
Research
Products
Business
Developers
Company
Foundation
(opens in a new window)
Try ChatGPT
(opens in a new window)Login
OpenAI
August 10, 2022ProductNew and improved content moderation tooling
Read documentation
(opens in a new window)
Loading…
Share
To help developers protect their applications against possible misuse, we are introducing the faster and more accurate Moderation endpoint⁠(opens in a new window). This endpoint provides OpenAI API developers with free access to GPT‑based⁠ classifiers that detect undesired content—an instance of using AI systems⁠ to assist with human supervision of these systems. We have also released both a technical paper⁠(opens in a new window) describing our methodology and the dataset⁠(opens in a new window) used for evaluation.
When given a text input, the Moderation endpoint assesses whether the content is sexual, hateful, violent, or promotes self-harm—content prohibited by our content policy⁠(opens in a new window). The endpoint has been trained to be quick, accurate, and to perform robustly across a range of applications. Importantly, this reduces the chances of products “saying” the wrong thing, even when deployed to users at-scale. As a consequence, AI can unlock benefits in sensitive settings, like education, where it could not otherwise be used with confidence.
Loading...
The Moderation endpoint helps developers to benefit from our infrastructure investments. Rather than build and maintain their own classifiers—an extensive process, as we document in our paper⁠(opens in a new window)—they can instead access accurate classifiers through a single API call.
As part of OpenAI’s commitment⁠ to making the AI ecosystem safer⁠, we are providing this endpoint to allow free moderation of all OpenAI API-generated content. For instance, Inworld⁠(opens in a new window), an OpenAI API customer, uses the Moderation endpoint to help their AI-based virtual characters remain appropriate for their audiences. By leveraging OpenAI’s technology, Inworld can focus on their core product: creating memorable characters. We currently do not support monitoring of third-party traffic.
Get started with the Moderation endpoint by checking out the documentation⁠(opens in a new window). More details of the training process and model performance are available in our paper⁠(opens in a new window). We have also released an evaluation dataset⁠(opens in a new window), featuring Common Crawl data labeled within these categories, which we hope will spur further research in this area.
View documentation
(opens in a new window)
API Platform
2022
Authors
Todor Markov, Chong Zhang, Sandhini Agarwal, Tyna Eloundou, Teddy Lee, Steven Adler, Angela Jiang, Lilian Weng
Related articlesView all
Global news partnerships: Le Monde and Prisa Media
CompanyMar 13, 2024
Review completed & Altman, Brockman to continue to lead OpenAI
CompanyMar 8, 2024
OpenAI announces new members to board of directors
CompanyMar 8, 2024
Research
Research Index
Research Overview
Economic Research
Latest Advancements
GPT-5.6
GPT-5.5
GPT-5.4
Safety
Safety Approach
Deployment Safety
(opens in a new window)
Security & Privacy
Trust & Transparency
Products
ChatGPT
(opens in a new window)
ChatGPT Business
(opens in a new window)
ChatGPT Enterprise
(opens in a new window)
ChatGPT for Education
(opens in a new window)
Codex
Release Notes
API Platform
Overview
API Log In
(opens in a new window)
Docs
(opens in a new window)
Business
Overview
Solutions
Resources
Customer Stories
Partner Network
Contact Sales
Developers
Apps SDK
(opens in a new window)
Open Models
Docs
(opens in a new window)
Resources
(opens in a new window)
Developer Forum
(opens in a new window)
Company
About Us
Our Charter
Careers
News
Support
Help Center
(opens in a new window)
More
Stories
Academy
Supply Co.
Livestreams
Podcast
RSS
Terms & Policies
Terms of Use
Privacy Policy
Other Policies
(opens in a new window)
(opens in a new window)
(opens in a new window)
(opens in a new window)
(opens in a new window)
(opens in a new window)
(opens in a new window)
OpenAI © 2015–2026Your privacy choices
EnglishUnited States