Skip to content

Worker

worker

Defines the server's deployed GPU-offload Modal app

The offload backend runs Whisper transcription and video conversion on a Modal GPU. This is the server-owned half of the Modal integration (the launcher owns the full-stack host app). The shared deployment lifecycle lives in mirumoji.modal

Why A Deployed Cls

  • An ephemeral app per job (app.run()) always spins up a fresh container and reloads the multi-GB Whisper model

  • A bigger scaledown_window can't help because a single job was the only call in the app's lifetime

  • OffloadWorker is a deployed modal.Cls that loads the model once per container (@modal.enter) and stays warm for scaledown_window, so back-to-back jobs skip the cold start while min_containers=0 still lets it scale to zero (no idle GPU billing)

Auto-Deployed

The server deploys this app to the user's workspace on first use via ensure_offload_deployed and tracks it via the mirumoji.modal ownership tags to ensure that it is never duplicated and is updated in place during version upgrades

Modal Container File Transfers

  • When using the server's modal transcribe backend, the server needs to perform network file transfers between the local file system and the Modal containers which run the GPU processing work

  • These file transfers include either sending an input media file located under HOST_MEDIA_PATH in the local file system for the container to process, or receiving a processed file from the container and saving it under HOST_MEDIA_PATH

  • Since these files are often large, multi-GB media files, the server creates a transient ephemeral Modal Volume for every job, which exists only for the duration of the job and can be accessed either inside the container or locally via a unique identifier

  • For this reason, Jobs receive both the volume's ID and the path in which the local runtime saved the input file inside the volume so that the container can find that file during the job

Modal Volume Lifecycle

  • Before the job starts, the local runtime creates the volume and uploads the input media file to it

  • The container then locates the volume via its ID, saves the file to its local storage, does any necessary GPU processing, and writes the resulting processed file (if any) back to that same volume

  • If there are any resulting processed files, the local runtime then accesses the volume again, saves them to the local file system and serves it to the user

  • Finally, the job is completed and the ephemeral volume is deleted

OFFLOAD_APP_NAME = 'mirumoji-offload' module-attribute

The deployed offload app's name on the user's Modal workspace

WORKER_CLS_NAME = 'OffloadWorker' module-attribute

The offload worker class name, used for Cls.from_name lookups

OffloadWorker

Warm GPU worker that transcribes and converts media on Modal

Attributes:

Name Type Description
_model WhisperModel

The WhisperModel loaded once per container at entry

convert(vol_fp, vol_id, to_mp4_kwargs=None)

Convert a video to MP4 inside a Modal GPU container

Encoder

  • The container probes NVENC and uses the on-device GPU pipeline only when an encoder is present, falling back to CPU libx264 otherwise

  • The data center compute GPUs (A100 / H100 / B200) have no NVENC, so they convert on CPU

File Transfer

  • The input video is read out of the per-job ephemeral volume into a container-local temp directory

  • The container converts the video and the resulting MP4 is written back into the same volume

  • The job returns the output key so that the local runtime can stream the result out and save it

  • The temp directory is removed once the job finishes

Parameters:

Name Type Description Default
vol_fp str

Path of the input video inside the per-job ephemeral volume

 required
vol_id str

ID of the per-job ephemeral volume

 required
to_mp4_kwargs dict | None

Additional arguments for mirumoji.server.processing.audio.to_mp4

 None

Returns:

Type Description
str

The path of the converted MP4 inside the per-job ephemeral volume

Raises:

Type Description
ModalVolumeError

If the input or output can't be transferred through the volume
FFmpegError

If any of the FFMPEG commands return a non-zero exit code
ValueError

If the input doesn't exist or an invalid resolution is provided
MissingFFmpegError

If the FFMPEG executable couldn't be located
MissingFFprobeError

If the FFPROBE executable couldn't be located
RuntimeError

If conversion produces no output

transcribe(vol_fp, vol_id, output_format='srt', *, w_transcribe_args=None)

Transcribe media on a Modal GPU and return raw transcription

Transcription-Only

  • Jobs return raw transcription

  • LLM post-processing (SRT-Fixing) is applied by the Processor afterwards through the provider-agnostic LLM layer, so the same path works for both local and Modal transcription

output_format

  • When output_format="srt", sentence-level SRT content is composed from transcription segments, returning a string ready to be saved as a .srt file

  • When output_format="joined", transcription segment texts are joined with the Japanese full stop into a single string without any timing information

File Transfer

  • The input media is read out of the per-job ephemeral volume into a container-local temp directory before being handed to Whisper

  • That directory is removed once the job finishes

Parameters:

Name Type Description Default
vol_fp str

Path of the input media inside the per-job ephemeral volume

 required
vol_id str

ID of the per-job ephemeral volume

 required
output_format Literal['srt', 'joined']

srt for sentence-level SRT content, joined for a single joined string. Defaults to srt

 'srt'
w_transcribe_args dict | None

Additional arguments for WhisperModel.transcribe. Overrides the ones set in mirumoji.server.processing.whisper.DEFAULT_TRANSCRIBE_OPTS

 None

Returns:

Type Description
str

The raw transcription in the requested format

Raises:

Type Description
ModalVolumeError

If the input can't be read from the volume
TranscriptionError

If transcription fails

ensure_offload_deployed()

Deploys the offload app to the user's workspace unless already current

Idempotent and tracked through mirumoji.modal, so it never duplicates the app and rolls it forward when the package version changes

Raises:

Type Description
ModalError

If credentials are missing or the deploy fails

get_offload_app() cached

Builds the offload app and registers the worker on first use

Import-Safe
  • The app is created here, not at module import, so importing this module has no side effects
Container Reconstruction

  • The worker is registered by reference (not serialized)

  • include_source ships the current source and the deployed container rebuilds the worker by re-importing OffloadWorker from this module

Returns:

Type Description
App

The configured mirumoji-offload app with OffloadWorker registered

offload_worker()

Returns a handle to the deployed OffloadWorker for remote calls

The instantiated Modal Cls resolves .transcribe / .convert through a dynamic __getattr__, so Any is the correct return type

Returns:

Type Description
Any

An OffloadWorker instance handle whose transcribe / convert methods expose .remote.aio(...) and .spawn.aio(...)