Backend reference

Backends

Support matrix

Note: The llamacpp rocm row lists linux, windows for the family as a whole, but MI350X (gfx950) is currently gated to Linux + stable channel only — the Windows TheRock distribution and the ROCm nightly build for gfx950 are not yet published, so gfx950 installs are rejected on Windows and on the nightly channel. The OS column reflects the row's overall reach; the per-architecture restriction is enforced by the backend's install gate.

Recipe options

acestep — ACE-Step

ds4 — DwarfStar4 (experimental)

flm — FastFlowLM NPU

llamacpp — Llama.cpp GPU

llamacpp-hrx — HRX GPU (experimental)

moonshine — Moonshine

onnxruntime — ONNX Runtime

openmoss — OpenMOSS TTS

sd-cpp — StableDiffusion.cpp

thenoise — TheNoise ROCm

thinksound — ThinkSound

trellis — TRELLIS.2

vllm — vLLM ROCm (experimental)

whispercpp — Whisper.cpp

Implementation notes

ACE-Step (acestep)

ace-server exposes an asynchronous job API: POST /lm or POST /synth returns a job id immediately, GET /job?id=N polls the status, and GET /job?id=N&result=1 fetches the finished result. AceStepServer::run_job wraps this submit/poll/fetch cycle with a ceiling of roughly 20 minutes per stage at a 1-second poll cadence. Synth results arrive as multipart/mixed (an audio part plus a latent part); Lemonade extracts the first audio part.

Vocals are a two-stage pipeline. POST /lm with lm_mode: "generate" runs the ACE-Step language model, which turns the caption and lyrics into audio codes plus LM-filled metadata, returned as a JSON array of enriched requests. That array is accepted by POST /synth verbatim, so Lemonade feeds it through unchanged. POST /synth on its own is the DiT-only instrumental path — it has no language model and cannot sing, so a lyrics value (other than the sentinel below) is what routes a request through /lm first.

[Instrumental] — any case, surrounding whitespace ignored — is ACE-Step's sentinel for the no-vocals path, matching the Python reference implementation. Instrumental requests send the sentinel explicitly rather than an empty string because the synth stage also feeds the lyrics text into its conditioning.

Errors from audio_generations are written into the response sink as a JSON error payload; the endpoint handler turns that into an HTTP error instead of shipping it as audio.

The model download fetches the DiT checkpoint variant plus three companions when present in the repo: the language model (acestep-5Hz-lm-4B-Q8_0.gguf, required for vocals and auto-lyrics), the Qwen3 text encoder, and the VAE. The checkpoint path handed to --models is the directory of GGUFs; ace-server scans it by architecture, and --keep-loaded keeps models resident across requests.

OpenMOSS (openmoss)

moss-tts-server hosts exactly one --model per process, so the voice generator that ships as a component of the speech model cannot be served by the same process. OpenMossServer runs the cascade one model at a time: the speech process is stopped, a transient process is spawned on the voice-generator checkpoint to render a single reference sample, and the speech process is brought back. Holding both would require a card that fits the pair, a strictly harder requirement than running the model that was actually asked for. For the same reason, server_models.json keeps size as the peak resident requirement of the main speech model rather than adding the non-resident VoiceGen download size. Rendered samples are cached per description for the life of the load, so repeating a description costs nothing. The speech process is restarted even when design fails, so an unsuccessful design cannot leave a loaded model with no process behind it.

The legacy MOSS-VoiceGen registry entry remains as a compatibility model for existing clients; integrated voice design on OpenMOSS-TTS and MOSS-TTS-Local does not depend on that standalone entry.

request_mutex_ serialises a process-changing voice-design request against normal OpenMOSS inference, load(), and unload(). Speech and SFX hold it shared; voice design and lifecycle changes hold it exclusively. During the intentional speech -> VoiceGenerator -> speech swap, process_swap_in_progress_ keeps is_backend_alive() true so the router does not mistake the temporarily empty speech-process handle for a crash. A request arriving in that window is accepted and then waits on request_mutex_ until speech is restored. start_speech_process() therefore calls stop_speech_process() rather than unload() on a failed readiness wait, because unload() takes the same exclusive lock.

The transient design process is polled for readiness through its own /health rather than the shared wait_for_ready(), since it does not own port_; the poll also checks ProcessManager::is_running() each round and reports the child's exit code, so a subprocess that dies during model loading returns that error instead of polling to the timeout. spawn() uses find_free_port() instead of choose_port() for the same reason — choose_port() assigns port_, and the caller decides which process port_ addresses.

Reference-conditioned TTS can exceed OpenMOSS's default 8192-token context. Lemonade intentionally keeps the upstream context default instead of forcing a larger allocation, preserving compatibility with memory-constrained GPUs. OpenMOSS v0.3 chunks prefill batches, but a complete prompt still has to fit n_ctx; unusually long references may therefore be rejected by upstream with guidance to shorten the reference or raise the context size.

Lemonade does not inject max_audio_frames into OpenMOSS speech requests. OpenMOSS v0.3 owns this policy: its VoiceGenerator derives a text-based duration window when needed, while MOSS-TTS/MOSS-TTSD keep their reference-aware/native stop behavior. Keeping that distinction in the backend avoids truncating conditioned or multi-speaker speech.

Voice design is opt-in through the voice_design_description extension and is never inferred from voice, which keeps its OpenAI-compatible meaning and is forwarded as an instruction. A client sending "voice": "default" gets speech rather than a design run for a voice literally named "default". The field is ignored when the request already carries reference_wav_b64. Request fields are read with a type-checking accessor rather than json::value(), which throws on a type mismatch instead of falling back to the default and would turn a client's wrong-typed field into a 500.

MOSS-SoundEffect uses the same recipe but is an audio-generation model: audio_generations() forwards to the backend's /sfx endpoint, accepting duration/cfg as aliases for seconds/cfg_scale.

llama.cpp (llamacpp)

Lemonade launches llama-server with --parallel 1 and leaves the upstream --cache-ram host prompt cache at its default. LlamaCppServer does not override downsize(): slot erase frees no device memory (the KV buffer is allocated once at model load) and discards the slot's KV state without going through the host-cache save path, so soft idle leaves the slot resident and a resumed conversation reuses its cached prefix.

Model downloads

A checkpoint file can be reached twice during a registry download: once because the backend's select_checkpoint_files claimed it alongside the main weight, and again because it is also declared as its own checkpoint role (the OpenMOSS .extras.gguf sidecars are both). The same bytes either way, so download_from_registry collapses duplicates before counting, or the progress total overshoots.

Backend auto-selection

When a recipe's backend is not pinned in config.json (the backend key is absent or "auto"), the default backend reported in system-info — and used by RecipeOptions when resolving *_backend options — is chosen as follows: the first supported backend in RECIPE_DEFS preference order wins, unless a later supported backend is already locally installed (state installed, update_available, or update_required) while the earlier candidates are merely installable. In that case the first installed one wins. This makes explicitly installing a variant (e.g. the Vulkan build of a GPU backend) an effective override: auto-selection uses what is on disk instead of downloading the preference-order winner. An explicit backend value in config.json always takes precedence over both rules. The llamacpp system variant is never auto-selected unless prefer_system is set.