Skip to main content

Documentation Index

Fetch the complete documentation index at: https://nvd-54.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

专为 OpenAI 模型设计的中间件。了解更多关于中间件的信息。
中间件描述
Content moderation使用 OpenAI 的审核端点审核智能体流量

内容审核

使用 OpenAI 的审核端点审核智能体流量(用户输入、模型输出和工具结果),以检测和处理不安全内容。 内容审核适用于以下场景:
  • 需要内容安全和合规的应用
  • 过滤有害、仇恨或不当内容
  • 需要安全护栏的面向客户的智能体
  • 满足平台审核要求
了解更多关于 OpenAI’s moderation models and categories.
API reference: OpenAIModerationMiddleware
from langchain_openai import ChatOpenAI
from langchain_openai.middleware import OpenAIModerationMiddleware
from langchain.agents import create_agent

agent = create_agent(
    model=ChatOpenAI(model="gpt-5.4"),
    tools=[search_tool, database_tool],
    middleware=[
        OpenAIModerationMiddleware(
            model="omni-moderation-latest",
            check_input=True,
            check_output=True,
            exit_behavior="end",
        ),
    ],
)
model
ModerationModel
default:"omni-moderation-latest"
OpenAI moderation model to use. Options: 'omni-moderation-latest', 'omni-moderation-2024-09-26', 'text-moderation-latest', 'text-moderation-stable'
check_input
bool
default:"True"
Whether to check user input messages before the model is called
check_output
bool
default:"True"
Whether to check model output messages after the model is called
check_tool_results
bool
default:"False"
Whether to check tool result messages before the model is called
exit_behavior
string
default:"end"
How to handle violations when content is flagged. Options:
  • 'end' - End agent execution immediately with a violation message
  • 'error' - Raise OpenAIModerationError exception
  • 'replace' - Replace the flagged content with the violation message and continue
violation_message
str | None
Custom template for violation messages. Supports template variables:
  • {categories} - Comma-separated list of flagged categories
  • {category_scores} - JSON string of category scores
  • {original_content} - The original flagged content
Default: "I'm sorry, but I can't comply with that request. It was flagged for {categories}."
client
OpenAI | None
Optional pre-configured OpenAI client to reuse. If not provided, a new client will be created.
async_client
AsyncOpenAI | None
Optional pre-configured AsyncOpenAI client to reuse. If not provided, a new async client will be created.
The middleware integrates OpenAI’s moderation endpoint to check content at different stages:Moderation stages:
  • check_input - User messages before model call
  • check_output - AI messages after model call
  • check_tool_results - Tool outputs before model call
Exit behaviors:
  • 'end' (default) - Stop execution with violation message
  • 'error' - Raise exception for application handling
  • 'replace' - Replace flagged content and continue
from langchain_openai import ChatOpenAI
from langchain_openai.middleware import OpenAIModerationMiddleware
from langchain.agents import create_agent


# Basic moderation
agent = create_agent(
    model=ChatOpenAI(model="gpt-5.4"),
    tools=[search_tool, customer_data_tool],
    middleware=[
        OpenAIModerationMiddleware(
            model="omni-moderation-latest",
            check_input=True,
            check_output=True,
        ),
    ],
)

# Strict moderation with custom message
agent_strict = create_agent(
    model=ChatOpenAI(model="gpt-5.4"),
    tools=[search_tool, customer_data_tool],
    middleware=[
        OpenAIModerationMiddleware(
            model="omni-moderation-latest",
            check_input=True,
            check_output=True,
            check_tool_results=True,
            exit_behavior="error",
            violation_message=(
                "Content policy violation detected: {categories}. "
                "Please rephrase your request."
            ),
        ),
    ],
)

# Moderation with replacement behavior
agent_replace = create_agent(
    model=ChatOpenAI(model="gpt-5.4"),
    tools=[search_tool],
    middleware=[
        OpenAIModerationMiddleware(
            check_input=True,
            exit_behavior="replace",
            violation_message="[Content removed due to safety policies]",
        ),
    ],
)