Updates

Image moderation now available

Christopher Dengsø

Jan 26, 2024 — 2 min read

I'm excited to announce that you can now moderate images with moderation API. Setting up image moderation works similarly to text moderation. You can adjust thresholds, and disable labels that you do not care about when flagging content.

Image Moderation Configuration with Moderation API

We offer 9 different labels out of the box - all available through a single model.

Nudity: Exposed male or female genitalia, female nipples, sexual acts.
Suggestive: Partial nudity, kissing.
Gore: Blood, wounds, death.
Violence: Graphic violence, causing harm, weapons, self-harm.
Weapon: Weapons either in use or displayed.
Drugs: Drugs such as pills.
Hate: Symbols related to nazi, terrorist groups, white supremacy and more.
Smoking: Smoking or smoking related content.
Alcohol: Alcohol or alcohol related content.

In case you need any other labels, please let us know.

Support in the review queue

Image moderation is also working seamlessly with the content review queue. Here you'll be able to see a small preview of the images, and otherwise perform actions just as you do with text.

New endpoint

The new capabilities are used via a new endpoint for image moderation.

You can find the API reference here:

Or use our Typescript package from NPM here:

Llama-Guard 3 on Moderation API

Llama Guard 3, now on Moderation API, offers precise content moderation with Llama-3.1. It’s faster and more accurate than GPT-4, perfect for real-time use and customizable for nuanced moderation needs.

How to self-host Llama-Guard 3 for content moderation

Llama-guard is a one of the best available models for content moderation at the moment. Here's a guide on everything from setting up a server to configuring Llama for your use case.

Context awareness for message moderation

Context is crucial when handling content moderation. One thing might seem innocent in one context, but hateful in a different context. You can already supply contextId and authorId with content, and this can help you understanding the context when reviewing items in the review queue. Now you can also enable

GPT-4o-mini for AI Agents

Update: since the creation of this post we've also added Llama Guard 3. Llama Guard 3 is now the recommended model for AI agents. Read about Llama Guard here. OpenAI have just released their latest model GPT-4o-mini. We're excited about the updated model and are already