Content moderation is a crucial part of managing online platforms, ensuring that published content complies with community guidelines and preventing the spread of harmful information. OpenAI offers a moderation API that can be integrated into .NET projects to automatically analyze and moderate content.
Moderation Model Used
The OpenAI moderation API uses the text-moderation-007 model, designed to analyze text and identify inappropriate content categories such as violence, harassment, hate speech, among others. This model evaluates the input content and returns a set of categories with scores indicating the likelihood of each category being present in the text.
Application in Real-World Projects
Automatic content moderation can be applied in various areas such as:
Social networks: filtering posts, comments, and private messages.
E-commerce platforms: moderating product reviews and comments.
Online forums and communities: monitoring topics and replies to ensure a healthy environment.
Messaging apps: preventing harassment and spam in conversations.
Integrating the OpenAI moderation API into a .NET project helps maintain the integrity and safety of the platform, reducing the need for manual intervention.
API Implementation
For our sample application, we’ll use an ASP.NET API project that will receive the content to be moderated and handle integration with OpenAI’s moderation API. To communicate with the API, we’ll need an API key. If you don’t have one, you can follow our detailed tutorial, which covers everything from creating an account to generating and configuring the API key needed to use the moderation service.
After generating our project and creating the API key, we must create a controller named ContentModerationController and a POST action called ModerateContent, receiving a parameter of type string named content, which will be the text sent to OpenAI’s moderation API. We’ll also add a try-catch block to handle any errors during the moderation process.
using Microsoft.AspNetCore.Mvc;
using System.Text.Json;
using System.Text;
using System.Net.Http;
namespace ContentModerationApp.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class ContentModerationController : ControllerBase
{
[HttpPost]
public async Task<ActionResult> ModerateContent(string content)
{
try
{
// Our moderation interaction code.
}
catch (Exception ex)
{
return StatusCode(500, $"Error while moderating content: {ex.Message}");
}
}
}
}
With the base structure of our ASP.NET project defined, the first step is to store the generated API key in a string inside the try block so we can use it in our requests.
string openaiApiKey = "your-api-key-here";
Next, we build the request body. Here, we define the message to moderate, which is passed through the content parameter.
var requestBody = new
{
input = content
};
To send the HTTP request, we need to serialize the requestBody object into JSON format.
var requestBodyJson = JsonSerializer.Serialize(requestBody);
Now, let’s create an HTTP client using HttpClient to send a POST request to the OpenAI API.
using (var httpClient = new HttpClient())
{
// Request code here.
}
We also need to add the authorization key in the request headers to ensure it is authenticated properly.
httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {openaiApiKey}");
After configuring the HTTP client with the authorization key, the next step is to send a POST request to the OpenAI API. The request body contains the content moderation data, previously serialized as JSON.
var response = await httpClient.PostAsync("https://api.openai.com/v1/moderations",
new StringContent(requestBodyJson, Encoding.UTF8, "application/json"));
After sending the request, we need to process the API response. The code below checks if the response was successful and reads the response body as a string.
if (response.IsSuccessStatusCode)
{
string responseBody = await response.Content.ReadAsStringAsync();
return Ok(responseBody);
}
else
{
return BadRequest(response.StatusCode);
}
Analyzing the Response
When we run our API and send a request, we receive a response from OpenAI’s moderation API in JSON format, containing detailed information about the analyzed content. Below is an example response with an explanation:
{
"id": "modr-9ijvCEVyMvw36jrtEhS0WlofXm9mt",
"model": "text-moderation-007",
"results": [
{
"flagged": false,
"categories": {
"sexual": false,
"hate": false,
"harassment": false,
"self-harm": false,
"sexual/minors": false,
"hate/threatening": false,
"violence/graphic": false,
"self-harm/intent": false,
"self-harm/instructions": false,
"harassment/threatening": false,
"violence": false
},
"category_scores": {
"sexual": 0.000013486678653862327,
"hate": 0.09813056141138077,
"harassment": 0.200340136885643,
"self-harm": 0.0008290316327475011,
"sexual/minors": 5.089680712444533e-7,
"hate/threatening": 0.044729895889759064,
"violence/graphic": 0.0017252579564228654,
"self-harm/intent": 0.000022909627659828402,
"self-harm/instructions": 0.000021325766283553094,
"harassment/threatening": 0.18670909106731415,
"violence": 0.000022909627659828402
}
}
]
}
The "id" field uniquely identifies the moderation request within OpenAI’s system. Each API call receives a unique ID.
The "model" field indicates the model used for content moderation—in this case, the text-moderation-007 model mentioned earlier.
The "flagged" field shows whether the analyzed text was marked as problematic or inappropriate. In this example, it is false, indicating that the text contains no content deemed inappropriate. This field is particularly useful in systems where any occurrence of inappropriate content must be strictly blocked.
Each result contains detailed information about the categories analyzed and the corresponding scores. Here are the categories and their meanings:
sexual: sexually explicit content.
hate: content expressing hatred or intolerance.
harassment: harassing or intimidating content.
self-harm: content promoting or encouraging self-harm.
sexual/minors: sexual content involving minors.
hate/threatening: hate content including threats.
violence/graphic: graphic or violent content.
self-harm/intent: content indicating intent to self-harm.
self-harm/instructions: content providing instructions on self-harm.
harassment/threatening: harassing content that includes threats.
violence: general violent content.
Each category has a score ranging from 0 to 1, where values closer to 1 indicate a higher probability that the text fits that category. For example, a score of 0.2 in “harassment” indicates a 20% chance the text could be considered harassing.
Conclusion
Integrating OpenAI’s moderation API into a .NET application is an effective way to ensure that published content aligns with community guidelines and to prevent the spread of harmful information. With proper configuration and implementation of a moderation service, it’s possible to automate the analysis and filtering of content, offering users a safer and more enjoyable experience.