AI
LLM access is a seam, not a vendor choice. Call sites ask the factory for an adapter; which provider answers is configuration. Moving an app between hosted and local inference is an environment change, not a code change.
The block
Live, in its entirety:
import { createOllamaChat } from "@tanstack/ai-ollama";
import { openRouterText } from "@tanstack/ai-openrouter";
/**
* Environment-pluggable LLM adapter — the local-LLM seam from GRUNDSTACK.md.
* Every AI call site asks this factory instead of hardcoding a provider, so
* moving an app from hosted to air-gapped inference is a config change, not a
* code change. Same graceful-degradation contract as @repo/jobs and
* @repo/mailer:
*
* OLLAMA_BASE_URL → local/on-prem inference (e.g. http://localhost:11434
* or the GPU box inside the closed network). Wins when
* both are set — on-prem intent is explicit.
* OPENROUTER_API_KEY → hosted inference via OpenRouter (today's default).
* neither → `getTextAdapter()` returns null; callers fall back
* to their stubs/canned copy and the app keeps working.
*
* `AI_MODEL` overrides the per-provider default model id.
*/
const DEFAULT_OPENROUTER_MODEL = "anthropic/claude-sonnet-4.5";
const DEFAULT_OLLAMA_MODEL = "llama3.1";
export function isAiEnabled(): boolean {
return Boolean(process.env.OLLAMA_BASE_URL || process.env.OPENROUTER_API_KEY);
}
/** The model id the active provider will run — "stub" when AI is off. */
export function aiModelId(): string {
if (process.env.OLLAMA_BASE_URL) {
return process.env.AI_MODEL ?? DEFAULT_OLLAMA_MODEL;
}
if (process.env.OPENROUTER_API_KEY) {
return process.env.AI_MODEL ?? DEFAULT_OPENROUTER_MODEL;
}
return "stub";
}
/**
* The text adapter for the configured provider, or null when AI is off.
* Pass straight to `chat({ adapter, ... })`.
*/
export function getTextAdapter() {
const ollamaUrl = process.env.OLLAMA_BASE_URL;
if (ollamaUrl) {
return createOllamaChat(process.env.AI_MODEL ?? DEFAULT_OLLAMA_MODEL, ollamaUrl);
}
if (process.env.OPENROUTER_API_KEY) {
// The typed model list is OpenRouter's catalogue of known slugs; the API
// accepts any routable slug, so a free-form AI_MODEL is cast through.
return openRouterText(
(process.env.AI_MODEL ?? DEFAULT_OPENROUTER_MODEL) as Parameters<typeof openRouterText>[0],
);
}
return null;
}Provider precedence
Loading diagram...
Local wins when both are set — pointing at a local model is an explicit act.
AI_MODEL overrides the per-provider default model id.
The null contract
getTextAdapter() returning null is a feature: every call site has a defined
no-AI behavior. In the example app:
- the chat endpoints return a clear, probe-able error,
- the dashboard briefing falls back to canned copy,
- document analysis writes a deterministic stub result (
model: "stub").
Your call sites should do the same — degrade the feature, never the app.
Usage shape
import { chat } from "@tanstack/ai";
import { getTextAdapter } from "@repo/ai";
const adapter = getTextAdapter();
if (!adapter) return fallback();
const stream = chat({ adapter, messages, systemPrompts: [SYSTEM_PROMPT] });One adapter, one chat() API — tools, streaming and structured output ride on top
regardless of provider.