Visionular

Content Moderation Guide

This guide shows you how to set up a live stream with content moderation to automatically detect and flag inappropriate content in real-time.

Overview

Content Moderation is a powerful feature that analyzes video frames from your live stream in real-time to detect potentially inappropriate content. When the system detects content that exceeds your configured confidence thresholds, it sends an event notification to your webhook endpoint, allowing you to take immediate action.

Supported Detection Models

ModelDescription
nudeDetects nudity and adult content
weaponDetects weapons and dangerous objects

How It Works

  1. Frame Extraction: The system captures frames from your live stream at the configured interval (e.g., every 1 second).
  2. AI Analysis: Each frame is analyzed by the configured detection models.
  3. Threshold Check: If the detection confidence exceeds your configured threshold, an alert is triggered.
  4. Event Notification: An audit_check event is sent to your webhook endpoint with details about the detected content.

Prerequisites

Before you begin, ensure you have:

Create a Live Stream with Content Moderation

Using the API

To create a live stream with content moderation enabled, include the audit_settings parameter in your request:

Request:

curl -X POST https://api.visionular.com/live/v1/live-streams \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "user_metadata": "Content Moderation Demo",
    "latency_mode": "low",
    "audit_settings": {
      "frame_interval": 1,
      "models": {
        "nude": 0.6,
        "weapon": 0.8
      }
    }
  }'

Parameters:

ParameterTypeDescription
audit_settingsobjectContent moderation configuration
audit_settings.frame_intervalintegerInterval in seconds between frame captures for analysis. Minimum value: 1
audit_settings.modelsobjectDetection models and their confidence thresholds
audit_settings.models.nudenumberConfidence threshold (0.0-1.0) for nudity detection. Lower values are more sensitive.
audit_settings.models.weaponnumberConfidence threshold (0.0-1.0) for weapon detection. Lower values are more sensitive.

Response:

{
  "code": 0,
  "data": {
    "stream_key": "202601-e95e9a27-e904-4340-9186-c27310dd729c",
    "status": "idle",
    "id": "202601-MjNkZWY5NmVhY2M3OTZmODAwZjIxNzI3ZGIzMGI2ZDE",
    "user_metadata": "Content Moderation Demo",
    "audit_settings": {
      "frame_interval": 1,
      "models": {
        "nude": 0.6,
        "weapon": 0.8
      }
    }
  },
  "message": "success"
}

Update Audit Settings

You can update the content moderation settings for an existing live stream:

Request:

curl -X PUT https://api.visionular.com/live/v1/live-streams/{live_stream_id}/audit-settings \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "audit_settings": {
      "frame_interval": 5,
      "models": {
        "nude": 0.8,
        "weapon": 0.9
      }
    }
  }'

Response:

{
  "code": 0,
  "data": {
    "live_stream_id": "202601-MjNkZWY5NmVhY2M3OTZmODAwZjIxNzI3ZGIzMGI2ZDE",
    "audit_settings": {
      "frame_interval": 5,
      "models": {
        "nude": 0.8,
        "weapon": 0.9
      }
    }
  },
  "message": "success"
}

Delete Audit Settings

To disable content moderation for a live stream, delete its audit settings:

curl -X DELETE https://api.visionular.com/live/v1/live-streams/{live_stream_id}/audit-settings \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Response: 204 No Content

Stop Audit Task

To stop the content moderation task for an active streaming session:

Request:

curl -X POST https://api.visionular.com/audit/v1/tasks/stop-audit-task \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "stream_session_id": "MjNxMmM0NnctMjAyNTEwLWQ0OWM4NDA5LWJjNzYtNGU4Mi1iOTJmLTg5YzVhOTQzMGRjNC0xNzYyMTU1MjU5"
  }'

Response:

{
  "code": 0,
  "message": "stop audit task success"
}

Query Moderation Records

Retrieve the list of content moderation records:

Request:

curl -X GET "https://api.visionular.com/audit/v1/records/moderation-records?page=1&limit=10&stream_session_id={SESSION_ID}&alert_type=nude" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Query Parameters:

ParameterTypeRequiredDescription
pageintegerNoPage number
limitintegerNoNumber of items per page (default: 25)
stream_session_idstringNoFilter by streaming session ID
alert_typestringNoFilter by alert type: nude or weapon

Response:

{
  "request_id": "bba7828e-3ac9-4f6a-adae-71dd18991a4f",
  "code": 0,
  "message": "get audit records success",
  "data": [
    {
      "id": 581,
      "task_id": 35,
      "event_id": "0000019a9c0d4afd-14d26dc5-00000014-e4f4ee94",
      "account_id": "1200",
      "channel_id": "bjJoMjQzcTctMjAyNTExLWVlNTA0OTE1LTRlZmYtNDdhMi1iNWE4LWMzODZlMmQ1MzdjMS0xNzYzNTU0NDk0",
      "url": "https://audit-test.visionular.com/...",
      "detect_info": "[{\"model\": \"nude\", \"confidence\": 0.32, \"risk_level\": \"pass\", \"bounding_box\": [0, 0, 0, 0]}]",
      "detect_status": 1,
      "audit_from": 1,
      "review_at": 1763554708917,
      "created_at": 1763554725
    }
  ]
}

Record Fields:

FieldTypeDescription
idintegerRecord ID
task_idintegerTask ID
event_idstringEvent ID
channel_idstringStream session ID
urlstringSigned URL to the captured frame image
detect_infostringJSON string with detection details (model, confidence, risk_level, bounding_box)
detect_statusintegerDetection status
created_atintegerRecord creation timestamp (Unix)

Configuring Confidence Thresholds

The confidence threshold determines how sensitive the detection should be:

Threshold RangeSensitivityUse Case
0.3 - 0.5HighStrict moderation, may have more false positives
0.5 - 0.7MediumBalanced moderation, recommended for most use cases
0.7 - 0.9LowLenient moderation, only flags high-confidence detections

Handling Moderation Events

When content is detected that exceeds your threshold, AuroraLive sends an audit_check event to your configured webhook endpoint.

Event Structure

{
  "type": "audit_check",
  "id": "0000019a48a474e8-0aaf4f34-0000000f-e14056d5",
  "data": {
    "stream_info": {
      "stream_id": "202510-M2Y1NzBjMjVmZDI5NmJmNGYxNDliMjEyNmJiZTA0YWE",
      "stream_session_id": "MjNxMmM0NnctMjAyNTEwLWQ0OWM4NDA5LWJjNzYtNGU4Mi1iOTJmLTg5YzVhOTQzMGRjNC0xNzYyMTU1MjU5",
      "title": "demo",
      "start_time": 1762155261
    },
    "frame_url": "https://audit-test.visionular.com/...",
    "alerts": [
      {
        "type": "nude",
        "confidence": 0.68,
        "description": "Nudity detected in the frame",
        "detection_info": {
          "model": "nude",
          "confidence": 0.68,
          "bounding_box": [0, 0, 0, 0],
          "risk_level": "reject"
        }
      }
    ]
  },
  "created_at": "2025-11-03T07:35:29Z"
}

Risk Levels

Risk LevelDescriptionRecommended Action
passContent is acceptableNo action needed
reviewContent may be inappropriateFlag for manual review
rejectContent is inappropriateImmediately disable stream or take action

Best Practices

  1. Set Appropriate Thresholds: Start with medium thresholds (0.5-0.7) and adjust based on your content and false positive rates.

  2. Configure Event Notifications: Ensure your webhook endpoint is properly configured before enabling content moderation.

  3. Implement Automated Actions: For high-risk content (reject level), consider automatically disabling the stream using the Disable Live Stream API.

  4. Review Flagged Content: For review level alerts, implement a manual review workflow to verify detections before taking action.

  5. Frame Interval Optimization:

    • Use frame_interval: 1 for high-risk content (adult, gambling)
    • Use frame_interval: 2-5 for general moderation to reduce costs
1/19/2026 08:46:13