> For the complete documentation index, see [llms.txt](https://infronai.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://infronai.gitbook.io/docs/features/service-tiers.md). # Service Tiers Control cost and latency tradeoffs with service tier selection ### Overview A model can have endpoints in multiple platform service tiers. Infron first determines the permitted tiers, then applies endpoint constraints, filters, and routing policies within the selected tier. | Tier | Value | Usage | | -------- | ---------- | ---------------------------------------------- | | Flex | `flex` | Explicitly select flex supply | | Standard | `standard` | The platform default when no tier is specified | | Priority | `priority` | Explicitly select priority supply | {% hint style="info" %} A tier name is not a universal discount, latency, or availability guarantee. The implementation does not apply a fixed discount solely because a request selects `flex`, or guarantee faster processing solely because it selects `priority`. Prices and performance depend on the configured endpoints. {% endhint %} ### Quick start Set `service_tier` inside the request's `provider` object: ```bash curl https://llm.onerouter.pro/v1/chat/completions \ -H "Authorization: Bearer $INFRON_API_KEY" \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -d '{ "model": "", "messages": [{"role": "user", "content": "Summarize the main points."}], "provider": {"service_tier": "flex"} }' ``` Replace `` with a marketplace model that supports the requested tier. This request allows only the platform's flex pool. It does not automatically switch to standard or priority. If there are no eligible flex endpoints, the request fails unless you explicitly authorize another tier. #### Parameters | Parameter | Type / default | Behavior | | --------------------------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------- | | `provider.service_tier` | String; `standard` | Primary platform tier: `flex`, `standard`, or `priority` | | `provider.service_tier_fallbacks` | String or string array; none | Additional permitted tiers, in order; at most 3 entries | | `provider.allow_fallbacks` | Boolean; `true` | `false` truncates the tier sequence to the primary tier; retries within it may still occur | | `provider.only` | Endpoint selector array | Restricts endpoints within every permitted tier | | `provider.order` | Endpoint selector array | Prioritizes endpoints within a tier; with `allow_fallbacks: false`, only matching endpoints remain | | `provider.sort` | For example `price`, `throughput`, `latency` | Changes selection within the current tier without admitting additional tiers | {% hint style="info" %} Tier values are validated as exact lowercase strings. `default`, `auto`, `fast`, and `Priority` are not valid platform tier values. Fallbacks accept a string such as `"standard"`, or an array such as `["standard", "priority"]`. An empty array means no fallback tiers. Explicit `null`, mixed arrays, and invalid tier values are rejected. Duplicates are removed while preserving first occurrence; the maximum of 3 entries is checked before deduplication. {% endhint %} ### How Routing Works Infron separates **tier eligibility, tier order, and endpoint selection within a tier**. For platform channels, it selects the first tier that still has eligible candidates after filtering, then scores, sorts, and selects within that tier. Fallback tiers do not compete with the current tier in a combined price or throughput pool. #### 1. Default to standard only Without explicit tier configuration, the effective sequence is `["standard"]`. Setting `provider.sort` to `price` or `throughput` changes selection within standard only. It does not admit flex or priority, even if no eligible standard endpoint exists. When using a preset, inspect the merged effective `provider` configuration. These defaults apply when the effective configuration contains no tier settings. #### 2. Select the primary tier separately from sorting `provider.service_tier` sets the primary tier without implying a sorting strategy or a fallback path. Priority does not implicitly add standard; flex does not revert to standard when flex supply is absent. Configure `provider.sort` separately if you want a price or throughput strategy. #### 3. Authorize each fallback tier To allow standard endpoints when priority supply cannot serve the request: ```json { "provider": { "service_tier": "priority", "service_tier_fallbacks": ["standard"], "allow_fallbacks": true } } ``` Infron evaluates priority before standard. Each tier goes through the endpoint, region, data policy, and other filters. Routing can advance when the current tier has no eligible candidates, or its candidates have been exhausted during an eligible retry flow. Error retryability, global retry limits, and request lifecycle still apply; the platform does not promise to attempt every endpoint. | Configuration | Effective platform sequence | | ----------------------------------------------------------- | ---------------------------- | | No primary or fallback tiers | `standard` | | Only `service_tier: "priority"` | `priority` | | `flex` with `["standard", "priority"]` | `flex → standard → priority` | | No primary tier, fallback `"flex"` | `standard → flex` | | `priority` with `["priority", "standard"]` | `priority → standard` | | `priority` with `["standard"]` and `allow_fallbacks: false` | `priority` | {% hint style="info" %} Cross-tier fallback can change the price. If only one tier is acceptable, do not add other tiers. Setting `allow_fallbacks: true` alone does not authorize cross-tier routing. {% endhint %} #### 4. Apply endpoint constraints within each permitted tier Tier order takes precedence over endpoint order. Listing a standard endpoint in `provider.order` does not bypass eligible priority endpoints when priority is the primary tier. To restrict routing to a specific endpoint: ```json { "provider": { "service_tier": "priority", "only": [""], "allow_fallbacks": false } } ``` Copy `` from the selected model and tier. Ordinary `only`, `order`, and `ignore` entries match `endpoint_slug`; do not assume a provider brand name or a constructed `openai/priority` identifier is valid. Naming an endpoint does not admit its tier into the permitted sequence. One endpoint may map to multiple channels, so restricting an endpoint does not guarantee a single upstream attempt. ### Comparing Tier Selection Options This table describes Infron platform channels. A next tier means an explicitly authorized fallback tier; retry eligibility still applies. | Configuration | Eligible scope | Selection order | Fallback boundary | | ---------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------- | --------------------------------------------------------------------------------- | | No tier settings | Standard | Existing within-tier policy | Standard only | | Only `sort: "price"` | Standard | Within-tier price strategy | Does not admit flex | | Only `sort: "throughput"` | Standard | Within-tier throughput strategy | Does not admit priority | | `service_tier: "priority"` | Priority | Existing strategy; no implicit throughput strategy | No implicit standard fallback | | `service_tier: "flex"` | Flex | Existing strategy; the tier alone does not force sorting | Fails without eligible candidates; no implicit standard fallback | | Flex + `["standard"]` + price sorting | Flex, then standard | Tier first, then price within that tier | May advance to standard after flex exhaustion | | Priority + `["standard"]` + throughput sorting | Priority, then standard | Priority first, even if a standard endpoint has higher throughput | May advance after priority exhaustion | | `only: [""]` | Intersection of permitted tiers and endpoint allowlist | Within-tier policy in that intersection | Naming a tier in a slug does not admit it | | `order: [...]`, fallback enabled | Listed endpoints first within each permitted tier; others may remain | Tier order before endpoint order | Unlisted endpoints within the tier; cross-tier requires explicit authorization | | `order: [...]` + `allow_fallbacks: false` | Intersection of primary tier and listed endpoints | Listed endpoint priorities | No cross-tier or unlisted endpoints; retries may remain within the matching scope | | Primary + fallbacks + `allow_fallbacks: false` | Primary only | Existing policy and endpoint constraints | Fallback list is truncated; not necessarily one endpoint | #### Three examples that clarify the boundary **A tier sequence is not a global price search.** Suppose flex costs 2 and standard costs 1 on the same hypothetical basis. Flex → standard with price sorting still starts with eligible flex supply; the lower standard price does not cause an early tier switch. **An endpoint name does not authorize its tier.** The live snapshot lists `openai/priority` for `openai/gpt-5.4`. With only `only: ["openai/priority"]`, no tier setting, and no preset supplying one, the platform still permits standard only. Also set `service_tier: "priority"` to admit that endpoint's tier. **Fallback does not bypass an allowlist.** Priority → standard combined with an `only` list containing exclusively priority endpoints cannot fall back to arbitrary standard endpoints. Include the required endpoints from both tiers, or remove that restriction when appropriate. ### Request examples Replace the model placeholder and verify protocol and tier support. In all three APIs, `provider` is a top-level request object. #### Chat Completions `POST https://llm.onerouter.pro/v1/chat/completions` ```json { "model": "", "messages": [{"role": "user", "content": "Hello"}], "provider": {"service_tier": "standard"} } ``` #### Responses `POST https://llm.onerouter.pro/v1/responses` ```json { "model": "", "input": "Hello", "provider": { "service_tier": "priority", "service_tier_fallbacks": ["standard"] } } ``` #### Messages `POST https://llm.onerouter.pro/v1/messages` ```json { "model": "", "max_tokens": 256, "messages": [{"role": "user", "content": "Hello"}], "provider": {"service_tier": "flex"} } ``` #### Python SDK Pass platform routing options through `extra_body`: ```python import os from openai import OpenAI client = OpenAI( api_key=os.environ["INFRON_API_KEY"], base_url="https://llm.onerouter.pro/v1", default_headers={ "Accept": "application/json", "Connection": "keep-alive", }, ) response = client.chat.completions.create( model="", messages=[{"role": "user", "content": "Hello"}], extra_body={"provider": { "service_tier": "priority", "service_tier_fallbacks": ["standard"], }}, ) print(response.choices[0].message.content) print(response.model_dump().get("service_tier")) ``` **Do not use the SDK's top-level `service_tier=` as a platform routing selector.** The compatible request structures contain a separate top-level field, which some protocol paths preserve or transform. It is not equivalent to `provider.service_tier` and does not construct the platform tier sequence. Endpoint configuration and adapters can also affect upstream parameters. S1, S4, S8 ### Reading the actual platform tier The platform response reports the final routing selection as `flex`, `standard`, or `priority`. Rerouting updates this value. The shared helper defaults to `standard` if a recognizable selected tier is absent. | API | Non-streaming JSON | Streaming SSE | | ---------------- | --------------------------------------------- | ------------------------------------------------------------------------ | | Chat Completions | Top-level `service_tier` | Top-level field in terminal / usage chunks, not necessarily every delta | | Responses | Top-level `service_tier` | `response.service_tier` in lifecycle events, not necessarily text deltas | | Messages | Top-level `service_tier`, an Infron extension | Top-level field in the `message_stop` event | This describes the inspected shared response paths. Top-level routing metadata injection normally also requires a provider identifier. If Messages preserves upstream `usage.service_tier`, that field does not have the same meaning as Infron's top-level platform tier. Do not reuse another gateway's field-location assumptions. A request that falls back from priority to standard can return this non-streaming fragment: ```json { "provider": "", "service_tier": "standard" } ``` A Responses completion event can contain: ```json { "type": "response.completed", "response": { "id": "resp_example", "object": "response", "service_tier": "standard" } } ``` These are field-location fragments, not complete responses. The platform overwrites upstream fields at the relevant location. Consequently, this tier identifies the selected Infron endpoint class; it does not prove which native capacity tier the upstream ultimately used. S2, S5 ### Pricing and tier availability Charges use the selected model-provider configuration, measured usage, and applicable pricing rules. Billing prioritizes model-provider-specific pricing using `ModelProviderID`, and considers team pricing, group ratios, and other rules. Generic pricing may apply when a specific price is absent. Do not calculate charges solely from the requested tier string, or assume the platform automatically reprices from an upstream native response tier. S6 Check tiers, endpoints, prices, and performance on the model detail page. Public read endpoints include: | Endpoint | Purpose | | ------------------------------------------------------- | ------------------------- | | `GET /api/models/{id}/service-tiers` | Tier summaries | | `GET /api/models/{id}/service-tiers/{tier}/providers` | Endpoint listing | | `GET /api/models/{id}/service-tiers/{tier}/pricing` | Pricing information | | `GET /api/models/{id}/service-tiers/{tier}/performance` | Performance observations | | `GET /api/models/{id}/service-tiers/{tier}/uptime` | Availability observations | These are `/api` marketplace routes, separate from `/v1` inference routes. Use the numeric ID from the model listing to avoid path-encoding ambiguity. Provider and pricing listings exclude unavailable, ignored, deprecated, and inactive-provider configurations. Historical metrics are not guarantees for an individual request. S9, S10 ### Troubleshooting | Situation | What to check | | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | Tier validation fails | Correct nesting, exact lowercase values, fallback type and length | | `404 / NO_ELIGIBLE_SERVICE_TIER` | No endpoint initially matches the tier sequence and other constraints; check tier, `only`, region, and related filters | | `503 / SERVICE_TIER_SEQUENCE_EXHAUSTED` | Routing exhausted the sequence after retries; the client may instead receive the first upstream error | | Fallback enabled but no tier change | Add explicit `service_tier_fallbacks`; retry eligibility still applies | | Named endpoint unavailable | Its tier must be permitted and all other constraints must match | | Response tier differs from primary tier | Inspect explicit fallback tiers and BYOK selection | | First stream chunk lacks the tier | Read the terminal / usage chunk or corresponding lifecycle event | This is not an exhaustive error catalog. Authentication, model validation, upstream failures, and other constraints can produce different statuses. When a retry cannot obtain another channel, the current controller may retain the first real upstream error. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://infronai.gitbook.io/docs/features/service-tiers.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.