​​To protect AI applications from generating or responding to inappropriate content, a new capability adds toxic content detection to LLM requests and responses. This advanced detection is designed to counteract sophisticated prompt injection techniques used by malicious actors to bypass standard LLM guardrails. The feature identifies and mitigates content that contains hateful, sexual, violent, or profane themes.

This capability is vital for maintaining the ethical integrity and safety of AI applications. It helps protect brand reputation, ensures user safety, mitigates misuse, and promotes a responsible AI. By analyzing both user inputs and model outputs, the system acts as a filter to intercept requests and responses that violate predefined safety policies.

The system can either block the request entirely or rewrite the output to remove the toxic language. In addition to detecting toxic content, it also helps prevent bias and misinformation, which are common risks associated with LLMs. By implementing this security layer, you can ensure that your AI agents and applications operate securely and responsibly, safeguarding against both intentional and unintentional generation of harmful content.