LiteLLM

Integration with LiteLLM's OpenAI-Compatible Endpoints with Infron AI

Account & API Keys Setup

The first step to start using Infron AI is to create an account and get your API key.

The second step to start using Google AI Studio is create a project and get your API Key.

Usage - completion

Python

import litellm
import os

response = litellm.completion(
    model="openai/<<Model Name>>",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    messages=[
                {
                    "role": "user",
                    "content": "Hey, how's it going?",
                }
    ],
)
print(response.json())

Please copy the <<Model name>> at model marketplace.

For example:

Qwen(Vertex Provider)

import litellm
import os

response = litellm.completion(
    model="openai/vertex/qwen3-next-80b-a3b-instruct",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    messages=[
        {
            "role": "user",
            "content": "Hey, how's it going?",
        }
    ],
)
print(response.json())

Qwen(Chutes Provider)

Usage - embedding

Python

import litellm
import os

response = litellm.embedding(
    model="openai/qwen/qwen3-embedding-0.6b",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    input=["good morning from litellm"]
)
print(response.json())

Usage with LiteLLM Proxy Server

1. Modify the config.yaml

config.yaml

model_list:
  - model_name: my-model
    litellm_params:
      model: openai/<your-model-name>  # add openai/ prefix to route as OpenAI provider
      api_base: <model-api-base>       # add api base for OpenAI compatible provider
      api_key: api-key                 # api key to send your model

qwen/qwen3-next-80b-a3b-instruct

1. Start the proxy

litellm --config ./config.yaml

1. Send Request to LiteLLM Proxy Server

Python

import openai

client = openai.OpenAI(
    api_key="sk-1234",             # pass litellm proxy key, if you're using virtual keys
    base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)

response = client.chat.completions.create(
    model="qwen/qwen3-next-80b-a3b-instruct",
    messages = [
        {
            "role": "user",
            "content": "what llm are you"
        }
    ],
)

print(response.json())

An response example is like below:

{
  "id": "gen-1768374179-WLdXmwS75xBQTPGgj6Fg",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "I am Qwen, a large-scale language model independently developed by the Tongyi Lab under Alibaba Group. I am designed to answer questions, generate text, perform logical reasoning, programming, and more. If you have any questions or need assistance, feel free to let me know anytime!",
        "refusal": null,
        "role": "assistant",
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": null
      }
    }
  ],
  "created": 1768374179,
  "model": "qwen/qwen3-next-80b-a3b-instruct",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 57,
    "prompt_tokens": 13,
    "total_tokens": 70,
    "completion_tokens_details": {
      "accepted_prediction_tokens": null,
      "audio_tokens": null,
      "reasoning_tokens": null,
      "rejected_prediction_tokens": null
    },
    "prompt_tokens_details": {
      "audio_tokens": null,
      "cached_tokens": null
    },
    "input_tokens": 0,
    "output_tokens": 0,
    "ttft": 0,
    "server_tool_use": {
      "web_search_requests": ""
    }
  },
  "request_id": "49f4d38ad4fd43699fad4fb312d371a0"
}