Video Inputs

Infron.AI supports sending video files to compatible models via the API. This guide will show you how to work with video using our API.

Infron.AI supports both direct URLs and base64-encoded data URLs for videos:

• URLs: Efficient for publicly accessible videos as they don't require local encoding

• Base64 Data URLs: Required for local files or private videos that aren't publicly accessible

Important: Video URL support varies by provider. Infron.AI only sends video URLs to providers that explicitly support them.

For example, Google Gemini on AI Studio only supports YouTube links (not Vertex AI).

API Only: Video inputs are currently only supported via the API.

Video Inputs

Requests with video files to compatible models are available via the /v1/chat/completions API with the video_url content type. The url can either be a URL or a base64-encoded data URL. Note that only models with video processing capabilities will handle these requests.

Using Video URLs

Here's how to send a video using a URL. Note that for Google Gemini on AI Studio, only YouTube links are supported:

Python

import requests
import json

url = "https://llm.onerouter.pro/v1/chat/completions"
headers = {
    "Authorization": f"Bearer {API_KEY_REF}",
    "Content-Type": "application/json"
}

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Please describe what's happening in this video."
            },
            {
                "type": "video_url",
                "video_url": {
                    "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
                }
            }
        ]
    }
]

payload = {
    "model": "{{MODEL}}",
    "messages": messages
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Using Base64 Encoded Videos

For locally stored videos, you can send them using base64 encoding as data URLs:

Python

import requests
import json
import base64
from pathlib import Path

def encode_video_to_base64(video_path):
    with open(video_path, "rb") as video_file:
        return base64.b64encode(video_file.read()).decode('utf-8')

url = "https://llm.onerouter.pro/v1/chat/completions"
headers = {
    "Authorization": f"Bearer {API_KEY_REF}",
    "Content-Type": "application/json"
}

# Read and encode the video
video_path = "path/to/your/video.mp4"
base64_video = encode_video_to_base64(video_path)
data_url = f"data:video/mp4;base64,{base64_video}"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "What's in this video?"
            },
            {
                "type": "video_url",
                "video_url": {
                    "url": data_url
                }
            }
        ]
    }
]

payload = {
    "model": "{{MODEL}}",
    "messages": messages
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Supported Video Formats

Infron.AI supports the following video formats:

• video/mp4

• video/mpeg

• video/mov

• video/webm

Common Use Cases

Video inputs enable a wide range of applications:

• Video Summarization: Generate text summaries of video content

• Object and Activity Recognition: Identify objects, people, and actions in videos

• Scene Understanding: Describe settings, environments, and contexts

• Sports Analysis: Analyze gameplay, movements, and tactics

• Surveillance: Monitor and analyze security footage

• Educational Content: Analyze instructional videos and provide insights

Best Practices

File Size Considerations

Video files can be large, which affects both upload time and processing costs:

• Compress videos when possible to reduce file size without significant quality loss

• Trim videos to include only relevant segments

• Consider resolution: Lower resolutions (e.g., 720p vs 4K) reduce file size while maintaining usability for most analysis tasks

• Frame rate: Lower frame rates can reduce file size for videos where high temporal resolution isn't critical

Optimal Video Length

Different models may have different limits on video duration:

• Check model-specific documentation for maximum video length

• For long videos, consider splitting into shorter segments

• Focus on key moments rather than sending entire long-form content

Quality vs. Size Trade-offs

Balance video quality with practical considerations:

• High quality (1080p+, high bitrate): Best for detailed visual analysis, object detection, text recognition

• Medium quality (720p, moderate bitrate): Suitable for most general analysis tasks

• Lower quality (480p, lower bitrate): Acceptable for basic scene understanding and action recognition

Provider-Specific Video URL Support

Video URL support varies significantly by provider:

• Google Gemini (AI Studio): Only supports YouTube links (e.g., https://www.youtube.com/watch?v=...)

• Google Gemini (Vertex AI): Does not support video URLs - use base64-encoded data URLs instead

• Other providers: Check model-specific documentation for video URL support