AI Auto-Tracking Guide

This guide walks you through enabling AI auto-tracking on a live stream. Given a fixed wide-angle source feed that captures the entire field of play, Visionular analyzes each frame in real time, locates the tracked object (e.g., the ball), and produces an auto-framed output stream that mimics a human camera operator following the action — no operator required.

Typical use cases: amateur volleyball/soccer matches, training sessions, and school events where a dedicated camera operator is unavailable.

Prerequisites

Before proceeding, ensure you have the following ready:

• Valid API Key (for Basic Authentication).
• AI Tracking entitlement enabled on your account. If your account is not provisioned for this feature, requests will fail with error code 100504 (Your account has no AI tracking permission). Contact your account manager to have it enabled.
• A fixed, wide-angle source camera capturing the full field (see Source Stream Recommendations below).
• Python 3.x and the requests library (for the code examples below).

Source Stream Recommendations

AI auto-tracking works by cropping a moving "virtual camera" window out of a much wider source frame. The wider and sharper your source, the better the output looks. Use one of the following recommended input resolutions:

Frame rate: The source stream must be 30 fps. Other frame rates are not supported by the AI tracking pipeline.

Why these resolutions? Both share the same ultra-wide aspect ratio, which matches the typical side-court fixed camera framing used in volleyball and soccer. Feeding a standard 16:9 source will still work, but the virtual camera has less room to pan, and fast action near the edges may be clipped.

Supported Tracking Objects

API Reference

Critical State Dependency Warning

⚠️ IMPORTANT: The PUT and DELETE endpoints above can only be called when the live stream is in the idle state. To change or disable AI tracking on a running broadcast, stop the stream first, update the configuration, then restart the broadcast.

Feature Compatibility

AI tracking is compatible with most other live-stream features. The only hard conflict is with audio-only mode.

Request Parameters (ai_tracking_setting)

Complete Code Examples (Python)

Example A: Create a Volleyball Auto-Tracking Stream

import requests
import json

url = "https://api.visionular.com/live/v1/live-streams"

payload = {
    "user_metadata": "volleyball-demo",
    "policy": ["public"],
    "latency_mode": "standard",
    "reconnect_window": 20,
    "ai_tracking_setting": {
        "tracking_object": "volleyball",
        "target_resolution": "1280x720"
    }
}

headers = {
    "Content-Type": "application/json",
    "Authorization": "Basic YOUR_BASE64_AUTH",
    "Auth-Type": "use-basic"
}

response = requests.post(url, headers=headers, json=payload)
print(json.dumps(response.json(), indent=4))

Feed the returned stream_key into an ultra-wide source (e.g., 8192x1728) and you will see the auto-framed 1280x720 output on the returned playback_ids.

Example B: Switch an Existing Stream to Soccer at 1080p

Make sure the stream is idle before calling this endpoint.

import requests
import json

live_stream_id = "YOUR_LIVE_STREAM_ID"
url = f"https://api.visionular.com/live/v1/live-streams/{live_stream_id}/ai-tracking-settings"

payload = {
    "ai_tracking_setting": {
        "tracking_object": "soccer",
        "target_resolution": "1920x1080"
    }
}

headers = {
    "Content-Type": "application/json",
    "Authorization": "Basic YOUR_BASE64_AUTH",
    "Auth-Type": "use-basic"
}

response = requests.put(url, headers=headers, json=payload)
print(json.dumps(response.json(), indent=4))

Example C: Disable AI Tracking

import requests

live_stream_id = "YOUR_LIVE_STREAM_ID"
url = f"https://api.visionular.com/live/v1/live-streams/{live_stream_id}/ai-tracking-settings"

headers = {
    "Authorization": "Basic YOUR_BASE64_AUTH",
    "Auth-Type": "use-basic"
}

response = requests.delete(url, headers=headers)
print(response.status_code)  # 204 on success

Best Practices

• Lock your source framing before kickoff. The auto-tracking window is derived from the source frame; any mid-match camera repositioning will confuse the tracker.
• Push the source at exactly 30 fps. AI tracking only processes 30 fps sources — streams pushed at other frame rates will fail or produce degraded output.
• Match target_resolution to your players' devices. Use 1920x1080 if most viewers watch on desktop/TV; stick with 1280x720 for mobile-first audiences or bandwidth-constrained networks.
• Enable AI tracking at creation time whenever possible. Reserving PUT/DELETE changes for between-match switches avoids the idle-state constraint.
• Combine with record: true if you want a highlight-ready VOD asset of the auto-framed output.

Troubleshooting