# LiteLLM

### **Account & API Keys Setup**

The first step to start using Infron is to [create an account](https://infron.ai/login) and [get your API key](https://infron.ai/dashboard/apiKeys).

The second step to start using Google AI Studio is [create a project](https://aistudio.google.com/app/projects) and [get your API Key](https://aistudio.google.com/app/api-keys).

### Usage - completion <a href="#usage---completion" id="usage---completion"></a>

{% tabs %}
{% tab title="Python" %}

```python
import litellm
import os

response = litellm.completion(
    model="openai/<<Model Name>>",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    messages=[
                {
                    "role": "user",
                    "content": "Hey, how's it going?",
                }
    ],
)
print(response.json())
```

{% endtab %}
{% endtabs %}

Please copy the `<<Model name>>` at [model marketplace](https://infron.ai/models).

<figure><img src="https://3822312837-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZ9C9AjT7j46HAcQrOVWw%2Fuploads%2FD4XmYrotKmC9cBfRWgGp%2Fe033194e0f487597e027c3e9b0e4cf38.png?alt=media&#x26;token=dfff1344-511e-474f-a837-7653b61853ff" alt=""><figcaption></figcaption></figure>

For example:

{% tabs %}
{% tab title="Qwen(Vertex Provider)" %}

```python
import litellm
import os

response = litellm.completion(
    model="openai/vertex/qwen3-next-80b-a3b-instruct",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    messages=[
        {
            "role": "user",
            "content": "Hey, how's it going?",
        }
    ],
)
print(response.json())
```

{% endtab %}

{% tab title="Qwen(Chutes Provider)" %}

```python
import litellm
import os

response = litellm.completion(
    model="openai/chutes/qwen3-next-80b-a3b-instruct",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    messages=[
        {
            "role": "user",
            "content": "Hey, how's it going?",
        }
    ],
)
print(response.json())
```

{% endtab %}
{% endtabs %}

### Usage - embedding <a href="#usage---embedding" id="usage---embedding"></a>

{% tabs %}
{% tab title="Python" %}

```python
import litellm
import os

response = litellm.embedding(
    model="openai/qwen/qwen3-embedding-0.6b",               # add `openai/` prefix to model so litellm knows to route to OpenAI
    api_key="<<API key>>",                  # api key to your openai compatible endpoint
    api_base="https://llm.onerouter.pro/v1",     # set API Base of your Custom OpenAI Endpoint
    input=["good morning from litellm"]
)
print(response.json())
```

{% endtab %}
{% endtabs %}

### Usage with LiteLLM Proxy Server <a href="#usage-with-litellm-proxy-server" id="usage-with-litellm-proxy-server"></a>

1. Modify the `config.yaml`

{% tabs %}
{% tab title="config.yaml" %}

```yml
model_list:
  - model_name: my-model
    litellm_params:
      model: openai/<your-model-name>  # add openai/ prefix to route as OpenAI provider
      api_base: <model-api-base>       # add api base for OpenAI compatible provider
      api_key: api-key                 # api key to send your model
```

{% endtab %}

{% tab title="qwen/qwen3-next-80b-a3b-instruct" %}

```yaml
model_list:
  - model_name: qwen/qwen3-next-80b-a3b-instruct
    litellm_params:
      model: openai/qwen/qwen3-next-80b-a3b-instruct  # add openai/ prefix to route as OpenAI provider
      api_base: https://llm.onerouter.pro/v1       # add api base for OpenAI compatible provider
      api_key: your-api-key                 # api key to send your model
```

{% endtab %}
{% endtabs %}

2. Start the proxy

```bash
litellm --config ./config.yaml
```

<figure><img src="https://3822312837-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZ9C9AjT7j46HAcQrOVWw%2Fuploads%2FqxDrlZTJ4Uq7oUw9Pw0t%2Fimage.png?alt=media&#x26;token=0316fe7a-9798-4851-be60-f4edcbdd7a07" alt=""><figcaption></figcaption></figure>

3. Send Request to LiteLLM Proxy Server

{% tabs %}
{% tab title="Python" %}

```python
import openai

client = openai.OpenAI(
    api_key="sk-1234",             # pass litellm proxy key, if you're using virtual keys
    base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)

response = client.chat.completions.create(
    model="qwen/qwen3-next-80b-a3b-instruct",
    messages = [
        {
            "role": "user",
            "content": "what llm are you"
        }
    ],
)

print(response.json())
```

{% endtab %}
{% endtabs %}

An response example is like below:

```json
{
  "id": "gen-1768374179-WLdXmwS75xBQTPGgj6Fg",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "I am Qwen, a large-scale language model independently developed by the Tongyi Lab under Alibaba Group. I am designed to answer questions, generate text, perform logical reasoning, programming, and more. If you have any questions or need assistance, feel free to let me know anytime!",
        "refusal": null,
        "role": "assistant",
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": null
      }
    }
  ],
  "created": 1768374179,
  "model": "qwen/qwen3-next-80b-a3b-instruct",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 57,
    "prompt_tokens": 13,
    "total_tokens": 70,
    "completion_tokens_details": {
      "accepted_prediction_tokens": null,
      "audio_tokens": null,
      "reasoning_tokens": null,
      "rejected_prediction_tokens": null
    },
    "prompt_tokens_details": {
      "audio_tokens": null,
      "cached_tokens": null
    },
    "input_tokens": 0,
    "output_tokens": 0,
    "ttft": 0,
    "server_tool_use": {
      "web_search_requests": ""
    }
  },
  "request_id": "49f4d38ad4fd43699fad4fb312d371a0"
}
```