Llm

llm

Provider-Agnostic LLM Layer

Exposes a thin LLMClient protocol with an OpenAI-Compatible adapter and a dedicated Anthropic adapter

Model Selection

• Model selection uses a provider:model string (e.g. "openai:gpt-4.1-mini")

• Which providers are usable in a given deployment is detected from installed SDKs and configured environment variables, so requesting an unconfigured provider raises a domain exception

AnthropicClient

LLMClient backed by the native Anthropic Messages API

Parameters:

Name	Type	Description	Default
api_key	str	Anthropic API key	required

LLMClient

Bases: Protocol

Protocol defining the minimal adapter surface required by the server for LLM calls

complete(*, system, prompt, model)

Returns the full completion text for a single prompt

list_models()

Returns the provider's available model ids

stream(*, system, prompt, model)

Yields completion text chunks as they arrive

LLMProvider

Bases: str, Enum

Enum listing all supported LLM providers

gemini and local are served through the OpenAI-compatible adapter while anthropic uses the native SDK

LLMProviderSpec dataclass

Describes how to detect and build an LLM provider's client

Parameters:

Name	Type	Description	Default
kind	str	openai_compat or anthropic	required
module	str	Importable SDK module the provider needs	required
key_env	str | None	Env variable holding the API key, if any	required
key_required	bool	Whether key_env must be set for availability	required
base_url	str | None	Static base URL for the OpenAI-compatible adapter	None
base_url_env	str | None	Env var holding the base URL (for local)	None

OpenAICompatClient

Adapter for the OpenAI SDK pointed at any OpenAI-compatible endpoint via base_url. Covers the following providers.

• OpenAI
• Gemini (OpenAI-Compatible Endpoint)
• Local (Any Local OpenAI-Compatible LLM Server)
• `Modal-Hosted (Any OpenAI-Compatible LLM Server Running In a Modal Container)

Parameters:

Name	Type	Description	Default
base_url	str | None	Endpoint base URL; None uses OpenAI's default	required
api_key	str	API key (use a non-empty placeholder for local servers that don't authenticate)	required

available_models(provider)

Lists the available model ids for an LLM provider

Parameters:

Name	Type	Description	Default
provider	LLMProvider	The provider to query	required

Returns:

Type	Description
list[str]	The provider's available model ids

Raises:

Type	Description
LLMProviderUnavailableError	If the provider isn't configured
LLMRequestError	If the provider's models endpoint fails

build_client(provider) cached

Builds a client for an LLM provider, resolving its config from the environment

Cached per provider so clients aren't rebuilt on every request. The environment is fixed for the server's lifetime

Parameters:

Name	Type	Description	Default
provider	LLMProvider	LLM provider to build a client for	required

Returns:

Type	Description
LLMClient	A ready LLMClient

Raises:

Type	Description
LLMProviderUnavailableError	If the provider's SDK or configuration is missing

client_for_model(selector)

Resolve a "provider:model" selector to a client and bare model name

Parameters:

Name	Type	Description	Default
selector	str	e.g. "anthropic:claude-sonnet-4-6"	required

Returns:

Type	Description
tuple[LLMClient, str]	Tuple of the built LLMClient and the bare model name

Raises:

Type	Description
InvalidModelStringError	If the selector is malformed
LLMProviderUnavailableError	If the provider is unavailable

custom_breakdown_prompt(sentence, focus, system_message, prompt_template)

Build the (system, prompt) pair for a profile's custom breakdown template

Parameters:

Name	Type	Description	Default
sentence	str	Full Japanese sentence	required
focus	str	Target word to explain in context	required
system_message	str	Custom system message	required
prompt_template	str	Template using {0} = sentence, {1} = focus	required

Returns:

Type	Description
tuple[str, str]	Tuple of the system message and the formatted user prompt

Raises:

Type	Description
LLMRequestError	If the template cannot be formatted

default_breakdown_prompt(sentence, focus)

Build the (system, prompt) pair for the default word-nuance breakdown

Parameters:

Name	Type	Description	Default
sentence	str	Full Japanese sentence	required
focus	str	Target word to explain in context	required

Returns:

Type	Description
tuple[str, str]	Tuple of the system message and the user prompt

parse_model(selector)

Parse a provider:model selector

Parameters:

Name	Type	Description	Default
selector	str	e.g. "openai:gpt-4.1-mini"	required

Returns:

Type	Description
tuple[LLMProvider, str]	Tuple of the resolved LLMProvider and the bare model name

Raises:

Type	Description
InvalidModelStringError	If the selector is malformed or names an unknown LLMProvider

provider_available(provider)

Whether an LLM provider is usable in this deployment

A provider is available when its required environment variables (API key, base URL) are configured

Parameters:

Name	Type	Description	Default
provider	LLMProvider	LLM provider to check	required

Returns:

Type	Description
bool	True if the LLM provider can be used

provider_status()

Report availability of every known LLM provider

Returns:

Type	Description
list[dict[str, Any]]	List of {"provider": str, "available": bool} for each provider, so the frontend can offer configured ones and grey out the rest

sentence_breakdown_prompt(sentence)

Build the (system, prompt) pair for explaining a whole sentence

Parameters:

Name	Type	Description	Default
sentence	str	Full Japanese sentence	required

Returns:

Type	Description
tuple[str, str]	Tuple of the system message and the user prompt

sse_breakdown(focus_json, chunks)

Stream a word breakdown. The structured focus first, then the explanation

The focus word (its stitched token + dictionary data) is emitted once as a focus event, after which the LLM explanation streams as normal data: frames via sse_format

Parameters:

Name	Type	Description	Default
focus_json	str | None	The focus word as compact JSON, or None	required
chunks	Iterable[str]	The explanation text chunks to stream	required

Yields:

Type	Description
str	A leading event: focus frame (when a focus is given), then the sse_format explanation frames

sse_format(chunks)

Wrap text chunks as Server-Sent Events, ending with a done event

JSON-Encoding

• Each chunk is JSON-encoded so that it can survive transport as a single SSE data: line, even when it contains newlines (the explanations are multi-line markdown)

• The client must use JSON.parse on every data: payload

Parameters:

Name	Type	Description	Default
chunks	Iterable[str]	Text chunks to emit	required

Yields:

Type	Description
str	SSE data: <json> frames, an event: error frame if a domain error interrupts the stream, then a terminal event: done frame