Text

text

Defines functions that use fugashi and kotobase to tokenize Japanese sentences and build pydantic models containing relevant dictionary data

enrich(sentence, mode=BundleMode.grammar)

Segments a sentence and enriches each stitched word with dictionary data

Runs one kotobase lookup per stitched word, keyed on the word's lemma. Stitching first means a compound like 図書館 (library) is looked up as one word and gets a real dictionary entry, instead of looking up the fragments 図書 and 館 separately (which is both slower and less useful)

Parameters:

Name	Type	Description	Default
sentence	str	The Japanese sentence to process	required
mode	BundleMode	How aggressively to group the tokens	grammar

Returns:

Type	Description
list[EnrichedJapaneseWord]	A list of EnrichedJapaneseWord (stitched word + dictionary data)

Raises:

Type	Description
FugashiError	If tokenisation fails
KotobaseError	If a dictionary lookup fails

ensure_fugashi()

Performs a simple tokenisation operation using fugashi to ensure that it's functional, raising an exception on any failures

Raises:

Type	Description
FugashiError	If any error occurs during tokenisation

ensure_kotobase()

Performs a simple lookup operation using kotobase to ensure that it's functional, raising an exception on any failures

Raises:

Type	Description
KotobaseError	If any error occurs during the lookup

get_audio_clip(literal, clip)

Fetches the raw bytes of one pronunciation clip

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal the clip belongs to	required
clip	str	The clip id from get_kanji_audio	required

Returns:

Type	Description
tuple[str, bytes] | None	The clip's file name and raw bytes, or None when no clip matches

Raises:

Type	Description
KotobaseError	If the lookup fails

get_kanji(literal)

Fetches the full profile of a single Kanji

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal	required

Returns:

Type	Description
KanjiInfo | None	The flat KanjiInfo, or None when the Kanji is unknown

Raises:

Type	Description
KotobaseError	If the lookup fails

get_kanji_audio(literal)

Lists the pronunciation clips available for a Kanji (Kanji Alive)

Clips

• Kanji Alive records example-word pronunciations per Kanji, so each clip id names one recorded example word

• The clips list is empty when the optional audio pack is not installed or the Kanji has no clips

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal	required

Returns:

Type	Description
KanjiAudio	The KanjiAudio clip listing

Raises:

Type	Description
KotobaseError	If the lookup fails

get_sentences_with_kanji(literal, limit=10)

Lists example sentences that contain a Kanji

Backs the Kanji view's examples section

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal	required
limit	int	Maximum number of sentences to return	10

Returns:

Type	Description
list[Example]	The matching sentences with their translations

Raises:

Type	Description
KotobaseError	If the lookup fails

get_stroke_svg(literal)

Fetches a Kanji's stroke-order diagram as a self-contained SVG document (KanjiVG)

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal	required

Returns:

Type	Description
str | None	The SVG document, or None when no stroke-order data exists

Raises:

Type	Description
KotobaseError	If the lookup fails

get_words_with_kanji(literal, limit=50)

Lists dictionary entries whose written form uses a Kanji

Backs the Kanji view's Words With section

Parameters:

Name	Type	Description	Default
literal	str	The Kanji literal	required
limit	int	Maximum number of entries to return	50

Returns:

Type	Description
list[JMEntry]	The matching entries as flat JMEntry models

Raises:

Type	Description
KotobaseError	If the lookup fails

kanji_by_radicals(radicals, match='all')

Finds Kanji that contain one, or every one of the given radicals

Parameters:

Name	Type	Description	Default
radicals	tuple[str, ...]	The radical glyphs to require	required
match	str	When all, every one of radicals must be present in the kanji. When any, kanjis that contain at least one of radicals are matched	'all'

Returns:

Type	Description
list[KanjiInfo]	The matching Kanji as flat KanjiInfo models

Raises:

Type	Description
KotobaseError	If the search fails

list_radicals() cached

Lists every search radical with its stroke count (RADKFILE)

Cached for the process lifetime, since the radical inventory is static

Returns:

Type	Description
list[RadicalInfo]	All search radicals

Raises:

Type	Description
KotobaseError	If the lookup fails

query_kotobase(query, wildcard=False, include_names=True, sentence_limit=5, entry_limit=None, reading=None, pos=None) cached

Wraps kotobase.Kotobase.lookup to provide a lru-cache for queries and flatten the nested lookup DTOs into the KotobaseData model

Readable Tags

The lookup runs with with_labels=True and every tag code (POS, field, misc, name type) is expanded into its human-readable label here, so the frontend never handles raw codes

Token-Aware Relevance

• When the caller knows the token's reading and pos (from fugashi), entries are re-ranked so kana-matching and part-of-speech-matching entries come first, which also drives the meanings summary taken from the top entry

• A lookup that finds nothing retries once with the hiragana reading, so tokens whose written form is not a dictionary key still resolve

Parameters:

Name	Type	Description	Default
query	str	word or wildcard pattern to query	required
wildcard	bool	When True, allows wildcards to be passed to query in order to match multiple words	False
include_names	bool	When True, also includes proper-name entries from the JMNe Dictionary	True
sentence_limit	int	Defines how many Tatoeba example sentences to fetch	5
entry_limit	int | None	Defines the maximum number of JMDict entries to fetch. Fetches all entries when set to None	None
reading	str | None	The token's reading (any kana), when known	None
pos	str | None	The token's UniDic top-level part of speech, when known	None

Returns:

Type	Description
KotobaseData	Pydantic model containing all information extracted from kotobase for the query word

Raises:

Type	Description
KotobaseError	If the kotobase lookup fails

resolve_reference(ref)

Resolves a sense cross-reference or antonym code into its entries

Lets the UI turn an xref or antonym string (e.g. 見る・1) into the actual referenced dictionary entries on demand, without resolving every reference upfront during a lookup

Parameters:

Name	Type	Description	Default
ref	str	The reference code from a sense's xrefs or antonyms	required

Returns:

Type	Description
list[JMEntry]	The referenced entries as flat JMEntry models

Raises:

Type	Description
KotobaseError	If the resolution fails

search_english(query, limit=50)

Searches dictionary entries by English meaning (reverse lookup)

Parameters:

Name	Type	Description	Default
query	str	The English search text	required
limit	int	Maximum number of entries to return	50

Returns:

Type	Description
list[JMEntry]	The matching entries as flat JMEntry models

Raises:

Type	Description
KotobaseError	If the search fails

segment(sentence, mode=BundleMode.grammar)

Tokenizes and stitches a sentence into useful words (no dictionary lookups)

Usage

• This is the fast path used to render clickable text

• Since it skips the (relatively slow) dictionary lookups, it is suited to tokenising whole subtitles/transcripts

• The dictionary data is fetched later by enrich or on a word click

Parameters:

Name	Type	Description	Default
sentence	str	The Japanese sentence to segment	required
mode	BundleMode	How aggressively to group the tokens	grammar

Returns:

Type	Description
list[JapaneseWord]	The stitched JapaneseWord bundles

Raises:

Type	Description
FugashiError	If tokenisation fails

segment_batch(sentences, mode=BundleMode.grammar)

Segments many sentences in one call (see segment)

Used to tokenize a whole subtitle file up front in a single request, so the player never tokenizes per-cue mid-playback

Parameters:

Name	Type	Description	Default
sentences	list[str]	The sentences to segment, in order	required
mode	BundleMode	How aggressively to group the tokens	grammar

Returns:

Type	Description
list[list[JapaneseWord]]	One stitched-word list per input sentence, in the same order

Raises:

Type	Description
FugashiError	If tokenisation fails

stitch(tokens, mode=BundleMode.grammar)

Stitches UniDic short-unit tokens into useful, learner-facing words

The grouping is controlled by mode (see BundleMode)

私は図書館で本を読みました (grammar mode)

UniDic short units (10):
    私 | は | 図書 | 館 | で | 本 | を | 読み | まし | た

Stitched words (8):
    私 | は | 図書館 | で | 本 | を | 読み | ました

図書 + 館 -> 図書館 (library)
読み stays on its own, the polite まし + た splits off -> 読み | ました
Particles は/で/を stay on their own

Reliability

• Splitting verbs, auxiliaries and particles follows directly from UniDic's grammatical labels, so it is essentially deterministic

• Noun compounding is a heuristic. UniDic returns a run of nouns, not whether they form one word or several (that lives in its separate "long unit word" layer, which the short-unit output does not expose), so consecutive nouns are merged by rule and may over- or under-merge

Parameters:

Name	Type	Description	Default
tokens	list[Token]	Short-unit tokens, in order	required
mode	BundleMode	How aggressively to group the tokens	grammar

Returns:

Type	Description
list[JapaneseWord]	The stitched JapaneseWord bundles, in order

tokenize(sentence)

Tokenizes a Japanese sentence using fugashi and extracts all token information into a pydantic model

Parameters:

Name	Type	Description	Default
sentence	str	Sentence to tokenize	required

Returns:

Type	Description
list[Token]	list of Token models containing extracted token information

Raises:

Type	Description
FugashiError	If the tagger can't be initialised or tokenisation fails