New model

New image toxicity model

Christopher Dengsø

Jan 29, 2025 — 1 min read

The new image toxicity model adds a single but robust label for detecting and preventing harmful images.

Where the image NSFW model can distinguish between multiple types of unwanted content, it can fail to generalise to toxic content outside of the provided labels.

The toxicity model on the other hand is better at understanding a wider range of toxicity, but provides just a single label.

Feel free to use the two models alongside each other. For example use the NSFW model if you want to give users a reason for the flag, but include the toxicity model to make sure you catch all unwanted content.

How to use the model

You can find the toxic image model in your project configuration now. Add it under "Pre-built models".

Support and feedback

We are committed to providing you with the best tools for content moderation. If you have any questions, encounter issues, or have suggestions for further improvements, please don't hesitate to contact our support team.

We look forward to seeing how this new model enhance your automated moderation and contribute to maintaining a safe and respectful online environment.

Happy Moderating!

How to handle users reporting inappropriate content

Users often come across inappropriate content, and it's crucial for social platforms to handle this scenario effectively. Allowing your users to report content builds trust, maintains a safe online environment, and ultimately improves the bottom line. But as your user base grows, managing these reports can become challenging.

New API endpoints for wordlists and review queues

Today we just rolled out a suite of new API endpoints designed to improve the experience with our Wordlists and Review Queues for enterprise plans. These enhancements offer greater flexibility if you're aiming to customise your moderation interface and leverage our robust moderation and review queue engine. There&

Object moderation endpoint

Until now, Moderation API allowed for the moderation of individual pieces of text or images. In practice, there’s often a need to moderate entire entities composed of multiple content fields. While one solution has been to call the API separately for each field, this approach can be inefficient and

Smart wordlists for moderation now available

You can now add smart wordlists that understand semantic meaning, similar words, and obfuscations in your Moderation API projects. When to use a wordlist In many cases an AI agent is a better solution to enforce certain guidelines as they understand context and intent, but wordlists are useful if you