Batch Prices, Plain API Calls: Three Routes for Offline LLM Jobs

Last month a customer asked us: "Do you support the Batch API for this closed-source small model? We have a large offline job to run."

We checked upstream and found something slightly awkward: the model's cloud provider has no batch service for it, and none planned.

This is not a one-off. Batch looks like the default answer for offline work, but it keeps getting less reliable as one. Here are the three routes that exist today, what each costs, and how to choose.

What counts as an offline task

One test: nobody is sitting at a screen waiting for the result. Bulk analysis of sales leads, user reviews, or call transcripts. Background enrichment and tagging of customer records. Document summarization, classification, extraction. Evals and synthetic training data.

These jobs are high volume, cost-sensitive, and "done by tomorrow morning" is fine. So you can trade the thing you do not care about (response speed) for the thing you care about most (unit price).

Route one: the Batch API

Pack requests into a job file, upload, queue, get results within 24 hours, pay about half the online rate. It fits very large jobs (hundreds of thousands of requests and up) on a specific closed model that actually offers batch.

Three growing potholes:

1. Not every model has one. Our customer's model has no batch service and no plan for one. Check before you build.

2. It is a separate codebase. Packing, uploading, polling, retrieving, retrying the failures yourself. One evaluation of batch workflows adds three hidden costs: you cannot iterate while a job runs, output quality can differ slightly from online runs, and failed rows are your problem, not the vendor's.

3. The 24-hour uncertainty. Results usually come back sooner, but the promise is 24 hours, so every downstream step has to plan for the worst case.

Route two: flex tiers

A normal API call plus one parameter (service_tier) drops the request into a low-priority lane. Half price again; the trade is queueing or retries at peak hours.

This is the clearest industry trend of 2026. Google shipped five tiers for the Gemini API in April (Standard, Flex, Priority, Batch, Caching): Flex is half price, and the official blog's pitch is that you stop managing batch job files and polling; Priority runs 1.75 to 2 times standard for faster, steadier service. OpenAI's flex charges batch prices through the plain API, aimed at exactly the background jobs above, and charges nothing when capacity is short. xAI added a priority tier too. As one commentator put it, vendors are taking the queueing logic that used to hide inside your architecture and selling it back to you as a product.

The trend in one line: batch prices, plain API calls.

On the Infron gateway, the flex lane for gemini-3-flash-preview is live: input 0.25, output 1.50 (USD per million tokens, same below), exactly half the standard lane (0.50 / 3.00). First customers are onboarding, and enterprise customers get a further discount on top (up to 30% off at Infron).

Three practical notes from Google's docs: flex runs on spare capacity, so write your own retry for peak hours; requests can queue, so set client timeouts to 10 minutes or more; and the response labels which tier actually served it, so monitor that field.

Route three: cheap open-weight models on the plain API

The first two routes keep the model and change how you run it. The third asks a different question: if the task does not require a specific model, why not use one whose list price already sits at batch levels?

Current public list prices on the Infron gateway:

(USD per million tokens. Public list prices measured on the gateway, 2026-07-17. "Up to X% off" is the ceiling of Infron's enterprise discount, set by volume and contract.)

Look at deepseek-v4-flash: its list price is already below gpt-5.4-nano's online price, and nano has no batch option at all. Add Infron's discount (up to 50% off) and you are under half of other people's batch prices. No 24-hour wait, no code changes, no job files.

Capability-wise, third-party evals put it in the same tier as the closed small models: on Artificial Analysis, deepseek-v4-flash scores 29 on the aggregate intelligence index against 27 for gemini-3-flash, at about 7 times cheaper on AA's aggregate price measure. Sutro, who specialize in batch inference, have argued that open-weight models are overtaking closed ones for bulk work. We push it one step further: at these prices, you do not need the batch machinery at all.

The honest catch: this route requires that your task tolerates a model swap. If your prompts and output formats are deeply tuned to one closed model, migration has a cost. Test a few hundred real samples first, then scale.

Check our numbers yourself

Infron maintains a public test library (infronai.github.io/prompt-cache-bench, mirrored at infron.ai/benchmark): the same requests run on Infron and OpenRouter side by side, comparing cost and speed. It covers 23 models; reports, raw data, and test code are all public.

Take qwen3.6-flash (2026-07-13 report, 800 paired samples). Infron's actual spend was lower under all four routing strategies, about 0.67 versus 1.53 to 1.63 (USD per 200 pairs), about 57 to 59% cheaper. OpenRouter was faster (up to 36% on throughput, about 11% on first token). That split is exactly the point: offline work cares about the total bill, not the first token, so the trade is a net win. With a user at the screen, take the standard or priority lane instead.

The deepseek-v4-flash (07-04), kimi-k2.7-code (07-10, 13 to 51% cheaper), and glm-5.2 (07-04, 42 to 72% cheaper) reports point the same way: cost is Infron's strength, response speed is OpenRouter's.

How to choose

First the route:

Then the model, by task type. Prices and specs below are measured; the task-fit evidence varies in strength, noted per row. Whatever the row says, run a few-hundred-sample test on your real data before switching.

Most teams end up hybrid: tagging on the cheapest small model, long documents on deepseek-v4-flash, media on gemini flex, hard-wired legacy left in place. Splitting by task beats one company-wide model by a wide margin.

Batch is not dead, it is just not the default

Sutro's defense of batch is real: the flow is simpler, tolerant of one-off failures, a full rerun fixes everything, and rate limits stop mattering. At millions of rows, overnight, on a model with a batch service, batch still wins.

But Google, OpenAI, and xAI all moved the cheap path onto plain API tiers in the same year. Cost control is heading toward picking a tier and picking a model, not maintaining a batch pipeline. If batch is still your default for offline work, odds are you are overpaying, or maintaining complexity you no longer need.

FAQ

My model has no batch service. Now what? If the vendor has a flex tier, use it: same code, half price. If not, that is the push to test an open-weight replacement on a few hundred real samples.

Does flex change output quality? The vendors' docs say no: same model, same online API, lower scheduling priority. Budget for retries at peak and a client timeout of 10 minutes or more. That is still far less work than a batch pipeline. (Batch is the one where an evaluation found output can differ slightly from online runs.)

When is the Batch API still the right call? Hundreds of thousands of rows or more, a deadline no tighter than "tomorrow", and a model that actually has the service.

We are building this tier logic into Infron's routing. If you have a pile of offline tasks, talk to us and we will price all three routes against your real workload: infron.ai.

Less orchestration. More innovation.