OpenRouter Integration

OpenRouter (openrouter.ai) exposes a unified OpenAI-compatible API that fronts many upstream providers — Anthropic, OpenAI, Google, DeepSeek, Mistral, Together, Fireworks, and more. First-class support arrived in v0.9.0 via OpenRouterModel, OpenRouterStrategy, and OpenRouterErrorClassifier.

Installation

pip install 'llm-minder[openrouter]'

(Internally this installs the openai SDK; OpenRouter speaks the OpenAI chat-completions wire format with a different base_url.)

OpenRouterModel always uses Chat Completions. It has no api_surface parameter, and the v0.27 switch of OpenAIModel to the Responses API doesn't affect it.

Authentication

Set OPENROUTER_API_KEY, or pass api_key= to OpenRouterModel.from_api_key(). Keys live at openrouter.ai/keys.

Quick start

import asyncio
from llm_minder import (
    LLMWorkItem,
    OpenRouterErrorClassifier,
    OpenRouterModel,
    OpenRouterStrategy,
    ParallelBatchProcessor,
    ProcessorConfig,
)


async def main() -> None:
    model = OpenRouterModel.from_api_key(
        "anthropic/claude-haiku-4-5",
        api_key="sk-or-...",
        # Optional — used by OpenRouter for app attribution / leaderboard.
        referer="https://my-app.example.com",
        title="My App",
    )
    strategy = OpenRouterStrategy(model)
    config = ProcessorConfig(max_workers=5, attempt_timeout=60.0)

    async with ParallelBatchProcessor[None, str, None](
        config=config,
        error_classifier=OpenRouterErrorClassifier(),
    ) as processor:
        await processor.add_work(
            LLMWorkItem(item_id="hi", strategy=strategy, prompt="Hello!")
        )
        result = await processor.process_all()

    print(result.results[0].output)


asyncio.run(main())

Model selection

OpenRouter model ids are prefixed with the upstream provider:

• anthropic/claude-sonnet-5.5, anthropic/claude-haiku-4-5
• openai/gpt-6-luna, openai/gpt-6-sol, openai/gpt-6-luna-pro
• google/gemini-3.5-flash-lite, google/gemini-3.5-flash
• deepseek/deepseek-v4-flash, deepseek/deepseek-v4-pro
• meta-llama/llama-4-maverick
• mistralai/mistral-large

See openrouter.ai/models for the full catalog.

Provider routing

OpenRouter lets you constrain which upstream host serves a request via the extra_body["provider"] configuration. Forward it through OpenRouterModel:

model = OpenRouterModel.from_api_key(
    "deepseek/deepseek-v4-flash",
    api_key="sk-or-...",
    extra_body={
        "provider": {
            # Try Fireworks first, then Together; fall back to others.
            "order": ["Fireworks", "Together"],
            # Or whitelist:
            # "allow_fallbacks": False,
            # "data_collection": "deny",
        }
    },
)

Full reference: openrouter.ai/docs/features/provider-routing.

Prompt caching

Caching behavior depends on which upstream provider serves your request. This is the most common gotcha when migrating between providers — read this section.

Upstream provider	Caching behavior	Action required
OpenAI	Automatic for long prefixes (minimum varies by model)	None
Gemini (implicit caching)	Automatic for repeated long prefixes	None
DeepSeek	Automatic, on-disk context cache	None
Anthropic	Opt-in via cache_control markers on message blocks	Build structured prompt
Gemini explicit CachedContent	Not exposed via OpenRouter	Use GeminiCachedModel

cached_input_tokens is populated when the upstream cache hits; BatchResult.total_cached_tokens aggregates across the batch, and the per-group statistics and hit-rate floor in Prompt caching break it down by job. prompt_cache_key= and cache_group= work on OpenRouterModel as on OpenAIModel. To estimate billable tokens, pass the matching rate from CachedTokenRates:

from llm_minder import CachedTokenRates

# Pick the rate for whichever upstream actually served your request
# (visible in LLMResponse.metadata['provider']).
billable = result.effective_input_tokens(CachedTokenRates.OPENAI)
billable = result.effective_input_tokens(CachedTokenRates.ANTHROPIC_READ)
billable = result.effective_input_tokens(CachedTokenRates.DEEPSEEK)

Cache writes land in cache_write_input_tokens when OpenRouter reports them (usage.prompt_tokens_details.cache_write_tokens), and BatchResult.total_cache_write_tokens adds them up. Anthropic bills writes at 1.25× normal input (2× with the 1-hour TTL), so pass the write rate too:

billable = result.effective_input_tokens(
    CachedTokenRates.ANTHROPIC_READ,
    cache_write_rate=CachedTokenRates.ANTHROPIC_WRITE,
)

Per-item provider metadata for mixed-provider batches

As of v0.10.0, OpenRouterStrategy (and the other built-in strategies) forwards LLMResponse.metadata straight through to WorkItemResult.metadata. For OpenRouter that includes the upstream provider name ("Anthropic", "OpenAI", "DeepSeek", etc.), the actually-routed model, and finish_reason. No custom parser needed.

from llm_minder import (
    CachedTokenRates,
    LLMWorkItem,
    OpenRouterModel,
    OpenRouterStrategy,
    ParallelBatchProcessor,
    ProcessorConfig,
)

# Per-provider rate lookup.
PROVIDER_RATES = {
    "OpenAI": CachedTokenRates.OPENAI,
    "Anthropic": CachedTokenRates.ANTHROPIC_READ,
    "DeepSeek": CachedTokenRates.DEEPSEEK,
}

model = OpenRouterModel.from_api_key("openrouter/auto")
strategy = OpenRouterStrategy(model)

# After process_all():
total_billable = 0
for r in result.results:
    if not r.success:
        continue
    provider = (r.metadata or {}).get("provider")
    rate = PROVIDER_RATES.get(provider, CachedTokenRates.OPENAI)
    cached = r.token_usage.get("cached_input_tokens", 0)
    inp = r.token_usage.get("input_tokens", 0)
    discount = int(cached * (1.0 - rate))
    total_billable += inp - discount

BatchResult.effective_input_tokens() takes a single rate, so it's appropriate when every item in the batch uses the same upstream. For mixed batches, use the per-item arithmetic above.

If you need the provider info inside your output type (e.g. to feed into a strict Pydantic model rather than read from WorkItemResult.metadata), a custom response_parser still works:

from dataclasses import dataclass


@dataclass
class TaggedOutput:
    text: str
    provider: str | None


strategy = OpenRouterStrategy(
    model,
    response_parser=lambda r: TaggedOutput(
        text=r.text,
        provider=(r.metadata or {}).get("provider"),
    ),
)

Both paths (reading WorkItemResult.metadata or capturing into a typed output) are fully supported.

Reasoning traces, tool calls, and logprobs

OpenRouterModel shares the OpenAI-compatible extractor, so reasoning traces (message.reasoning on OpenRouter, message.reasoning_content on DeepSeek-style upstreams), tool calls, and logprobs land under reserved metadata keys with typed views on each per-item WorkItemResult (not the batch-level BatchResult): item_result.reasoning, item_result.tool_calls, item_result.logprobs. See Typed auxiliary output for shapes and boundaries (stable from 1.0, except the provisional logprobs).

Anthropic prompt caching via OpenRouter

Anthropic requires you to mark the message blocks you want cached with cache_control: {"type": "ephemeral"}. OpenRouter passes the markers through. The cleanest way to do this through the framework is a small custom strategy that builds the message list inside execute():

from llm_minder.base import LLMResponse, RetryState, TokenUsage
from llm_minder.llm_strategies import LLMCallStrategy


class CachedAnthropicStrategy(LLMCallStrategy[str]):
    def __init__(self, model: OpenRouterModel, system_text: str) -> None:
        self.model = model
        self.system_text = system_text

    async def execute(
        self,
        prompt: str,
        attempt: int,
        timeout: float,
        state: RetryState | None = None,
    ) -> tuple[str, TokenUsage, None]:
        messages = [
            {
                "role": "system",
                "content": [
                    {
                        "type": "text",
                        "text": self.system_text,
                        "cache_control": {"type": "ephemeral"},
                    }
                ],
            },
            {"role": "user", "content": prompt},
        ]
        response: LLMResponse = await self.model.generate(messages)
        return response.text, response.token_usage, None

The first call writes the cache (paid at 1.25× normal input rate). Calls within ~5 minutes (default ephemeral TTL) hit the cache (paid at 0.1× normal rate). Use a 1-hour TTL with {"type": "ephemeral", "ttl": "1h"} for longer-lived caches if your account supports it.

DeepSeek caching note

DeepSeek's automatic cache uses native field names (prompt_cache_hit_tokens, prompt_cache_miss_tokens) rather than OpenAI's prompt_tokens_details.cached_tokens. Whether OpenRouter normalizes those into cached_tokens depends on the upstream host and OpenRouter's current mapping. If you specifically want reliable DeepSeek cache telemetry, call DeepSeek's API directly with the built-in DeepSeekModel (v0.10.0), which reads those native fields into cached_input_tokens:

from llm_minder import DeepSeekModel, DeepSeekStrategy

model = DeepSeekModel.from_api_key("deepseek-v4-flash", thinking=False)  # reads DEEPSEEK_API_KEY
strategy = DeepSeekStrategy(model)

Cross-provider batches

Because every model is just a different OpenRouterModel(model="..."), mixing providers in one batch is trivial:

anthropic = OpenRouterModel.from_api_key(
    "anthropic/claude-haiku-4-5", api_key=KEY,
)
openai = OpenRouterModel.from_api_key(
    "openai/gpt-6-luna", api_key=KEY,
)

async with ParallelBatchProcessor(...) as processor:
    await processor.add_work(LLMWorkItem(
        item_id="anthropic_q",
        strategy=OpenRouterStrategy(anthropic),
        prompt="...",
    ))
    await processor.add_work(LLMWorkItem(
        item_id="openai_q",
        strategy=OpenRouterStrategy(openai),
        prompt="...",
    ))
    result = await processor.process_all()

Error handling

OpenRouterErrorClassifier extends OpenAIErrorClassifier and adds:

• 502 with body containing no_provider_available (no upstream host could serve the request) → retryable, network_error. Without the override these would otherwise look like generic server errors.
• "No allowed providers" (your provider routing settings exclude every host) → retryable network_error only on 502/503; any other status is the non-retryable client_error category, since retrying the same routing won't help.
• An HTTP-200 body that carries an error object instead of choices → OpenRouterModel raises ProviderResponseError. The classifier treats it as retryable upstream_error, or as rate_limit when the embedded status is 429 (or, with no status, when the message reads like a rate limit). See OpenRouter reports an error inside HTTP 200.
• An explicit insufficient_quota code → non-retryable insufficient_balance, whether it arrives as a status error or inside an HTTP-200 body.

Everything else (rate limits, timeouts, 4xx vs 5xx) inherits from the OpenAI classifier.

See also

• docs/OPENAI_INTEGRATION.md — the OpenAI sibling.
• examples/example_openrouter.py — runnable example with the cross-provider and cached-Anthropic patterns.