# LiteLLM
### **Account & API Keys Setup**
The first step to start using Infron is to [create an account](https://infron.ai/login) and [get your API key](https://infron.ai/dashboard/apiKeys).
The second step to start using Google AI Studio is [create a project](https://aistudio.google.com/app/projects) and [get your API Key](https://aistudio.google.com/app/api-keys).
### Usage - completion
{% tabs %}
{% tab title="Python" %}
```python
import litellm
import os
response = litellm.completion(
model="openai/<>", # add `openai/` prefix to model so litellm knows to route to OpenAI
api_key="<>", # api key to your openai compatible endpoint
api_base="https://llm.onerouter.pro/v1", # set API Base of your Custom OpenAI Endpoint
messages=[
{
"role": "user",
"content": "Hey, how's it going?",
}
],
)
print(response.json())
```
{% endtab %}
{% endtabs %}
Please copy the `<>` at [model marketplace](https://infron.ai/models).
For example:
{% tabs %}
{% tab title="Qwen(Vertex Provider)" %}
```python
import litellm
import os
response = litellm.completion(
model="openai/vertex/qwen3-next-80b-a3b-instruct", # add `openai/` prefix to model so litellm knows to route to OpenAI
api_key="<>", # api key to your openai compatible endpoint
api_base="https://llm.onerouter.pro/v1", # set API Base of your Custom OpenAI Endpoint
messages=[
{
"role": "user",
"content": "Hey, how's it going?",
}
],
)
print(response.json())
```
{% endtab %}
{% tab title="Qwen(Chutes Provider)" %}
```python
import litellm
import os
response = litellm.completion(
model="openai/chutes/qwen3-next-80b-a3b-instruct", # add `openai/` prefix to model so litellm knows to route to OpenAI
api_key="<>", # api key to your openai compatible endpoint
api_base="https://llm.onerouter.pro/v1", # set API Base of your Custom OpenAI Endpoint
messages=[
{
"role": "user",
"content": "Hey, how's it going?",
}
],
)
print(response.json())
```
{% endtab %}
{% endtabs %}
### Usage - embedding
{% tabs %}
{% tab title="Python" %}
```python
import litellm
import os
response = litellm.embedding(
model="openai/qwen/qwen3-embedding-0.6b", # add `openai/` prefix to model so litellm knows to route to OpenAI
api_key="<>", # api key to your openai compatible endpoint
api_base="https://llm.onerouter.pro/v1", # set API Base of your Custom OpenAI Endpoint
input=["good morning from litellm"]
)
print(response.json())
```
{% endtab %}
{% endtabs %}
### Usage with LiteLLM Proxy Server
1. Modify the `config.yaml`
{% tabs %}
{% tab title="config.yaml" %}
```yml
model_list:
- model_name: my-model
litellm_params:
model: openai/ # add openai/ prefix to route as OpenAI provider
api_base: # add api base for OpenAI compatible provider
api_key: api-key # api key to send your model
```
{% endtab %}
{% tab title="qwen/qwen3-next-80b-a3b-instruct" %}
```yaml
model_list:
- model_name: qwen/qwen3-next-80b-a3b-instruct
litellm_params:
model: openai/qwen/qwen3-next-80b-a3b-instruct # add openai/ prefix to route as OpenAI provider
api_base: https://llm.onerouter.pro/v1 # add api base for OpenAI compatible provider
api_key: your-api-key # api key to send your model
```
{% endtab %}
{% endtabs %}
2. Start the proxy
```bash
litellm --config ./config.yaml
```
3. Send Request to LiteLLM Proxy Server
{% tabs %}
{% tab title="Python" %}
```python
import openai
client = openai.OpenAI(
api_key="sk-1234", # pass litellm proxy key, if you're using virtual keys
base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)
response = client.chat.completions.create(
model="qwen/qwen3-next-80b-a3b-instruct",
messages = [
{
"role": "user",
"content": "what llm are you"
}
],
)
print(response.json())
```
{% endtab %}
{% endtabs %}
An response example is like below:
```json
{
"id": "gen-1768374179-WLdXmwS75xBQTPGgj6Fg",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "I am Qwen, a large-scale language model independently developed by the Tongyi Lab under Alibaba Group. I am designed to answer questions, generate text, perform logical reasoning, programming, and more. If you have any questions or need assistance, feel free to let me know anytime!",
"refusal": null,
"role": "assistant",
"annotations": null,
"audio": null,
"function_call": null,
"tool_calls": null
}
}
],
"created": 1768374179,
"model": "qwen/qwen3-next-80b-a3b-instruct",
"object": "chat.completion",
"service_tier": null,
"system_fingerprint": null,
"usage": {
"completion_tokens": 57,
"prompt_tokens": 13,
"total_tokens": 70,
"completion_tokens_details": {
"accepted_prediction_tokens": null,
"audio_tokens": null,
"reasoning_tokens": null,
"rejected_prediction_tokens": null
},
"prompt_tokens_details": {
"audio_tokens": null,
"cached_tokens": null
},
"input_tokens": 0,
"output_tokens": 0,
"ttft": 0,
"server_tool_use": {
"web_search_requests": ""
}
},
"request_id": "49f4d38ad4fd43699fad4fb312d371a0"
}
```
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://infronai.gitbook.io/docs/frameworks-and-integrations/litellm.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.