Lemonade API

We have designed a set of Lemonade-specific endpoints to enable client applications by extending the existing cloud-focused APIs (e.g., OpenAI). These extensions allow for a greater degree of UI/UX responsiveness in native applications by allowing applications to:

• Download models at setup time.
• Pre-load models at UI-loading-time, as opposed to completion-request time.
• Unload models to save memory space.
• Understand system resources and state to make dynamic choices.

POST /v1/pull

Register and install models for use with Lemonade Server.

Parameters

The Lemonade Server built-in model registry has a collection of model names that can be pulled and loaded. The pull endpoint can install any registered model, and it can also register-then-install any model available on Hugging Face.

Common Parameters

Install a Model that is Already Registered

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Response format:

{
  "status":"success",
  "message":"Installed model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

Register and Install a Model

Registration will place an entry for that model in the user_models.json file, which is located in the user's Lemonade cache (default: ~/.cache/lemonade). Then, the model will be installed. Once the model is registered and installed, it will show up in the models endpoint alongside the built-in models and can be loaded.

The recipe field defines which software framework and device will be used to load and run the model.

Note: the model_name for registering a new model must use the user namespace, to prevent collisions with built-in models. For example, user.Phi-4-Mini-GGUF.

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.Phi-4-Mini-GGUF",
    "checkpoint": "unsloth/Phi-4-mini-instruct-GGUF:Q4_K_M",
    "recipe": "llamacpp"
  }'

Response format:

{
  "status":"success",
  "message":"Installed model: user.Phi-4-Mini-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

Register an Omni-Model

An omni collection is a collection type that bundles several already-registered models into a single entry that can be loaded, pulled, or deleted as a unit. Use recipe: "collection.omni" with a components array instead of checkpoint.

Components do not need to be downloaded already — any not-yet-downloaded components are pulled by the same call. Deleting the collection removes only the collection entry; components stay on disk.

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.MyKit",
    "recipe": "collection.omni",
    "components": ["Qwen3-0.6B-GGUF", "Whisper-Tiny", "SD-Turbo"]
  }'

Streaming Response (stream=true)

When stream=true, the endpoint returns Server-Sent Events with real-time download progress:

event: progress
data: {"file":"model.gguf","file_index":1,"total_files":2,"bytes_downloaded":1073741824,"bytes_total":2684354560,"percent":40}

event: progress
data: {"file":"config.json","file_index":2,"total_files":2,"bytes_downloaded":1024,"bytes_total":1024,"percent":100}

event: complete
data: {"file_index":2,"total_files":2,"percent":100}

Event Types:

Server-owned download mode (stream=true, subscribe=false)

By default, stream=true keeps the /v1/pull HTTP response subscribed to Server-Sent Events until the download finishes. Clients that need download state to survive a renderer reload, tab close, or reconnect can also send subscribe=false.

When stream=true and subscribe=false, /v1/pull starts a server-owned model download job and returns a JSON snapshot immediately. The job continues on the server. Clients can poll GET /v1/downloads to restore progress and can use POST /v1/downloads/control to pause, cancel, or remove the job.

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "stream": true,
    "subscribe": false
  }'

Example response:

{
  "id": "model:Qwen3-0.6B-GGUF",
  "type": "model",
  "model_name": "Qwen3-0.6B-GGUF",
  "status": "downloading",
  "running": true,
  "file": "",
  "file_index": 0,
  "total_files": 0,
  "bytes_downloaded": 0,
  "bytes_total": 0,
  "total_download_size": 0,
  "bytes_previously_downloaded": 0,
  "completed_files_bytes": 0,
  "cumulative_bytes_downloaded": 0,
  "overall_bytes_downloaded": 0,
  "percent": 0,
  "complete": false
}

GET /v1/downloads

List server-owned model download jobs that were started with POST /v1/pull using stream=true and subscribe=false.

This endpoint is intended for clients that need to restore download-manager state after a reload or reconnect. Active, paused, cancelled, and errored jobs remain visible until the client removes them. Completed jobs remain visible briefly so clients can observe completion and refresh model state.

Example request

curl http://localhost:13305/v1/downloads

Response format

[
  {
    "id": "model:Qwen3-0.6B-GGUF",
    "type": "model",
    "model_name": "Qwen3-0.6B-GGUF",
    "status": "downloading",
    "running": true,
    "file": "model.gguf",
    "file_index": 1,
    "total_files": 2,
    "bytes_downloaded": 1073741824,
    "bytes_total": 2684354560,
    "total_download_size": 2684355584,
    "bytes_previously_downloaded": 0,
    "completed_files_bytes": 0,
    "cumulative_bytes_downloaded": 1073741824,
    "overall_bytes_downloaded": 1073741824,
    "percent": 40,
    "complete": false
  }
]

Download job fields

POST /v1/downloads/control

Control a server-owned model download job.

Parameters

Actions

Example request

curl -X POST http://localhost:13305/v1/downloads/control \
  -H "Content-Type: application/json" \
  -d '{
    "id": "model:Qwen3-0.6B-GGUF",
    "action": "pause"
  }'

Response format

For pause and cancel, the endpoint returns the latest job snapshot:

{
  "id": "model:Qwen3-0.6B-GGUF",
  "type": "model",
  "model_name": "Qwen3-0.6B-GGUF",
  "status": "paused",
  "running": false,
  "file": "model.gguf",
  "file_index": 1,
  "total_files": 2,
  "bytes_downloaded": 1073741824,
  "bytes_total": 2684354560,
  "percent": 40,
  "complete": false
}

For remove, the endpoint returns:

{"status":"ok"}

If the job is already missing and action is remove, the endpoint returns:

{"status":"ok","missing":true}

GET /v1/pull/variants

Inspect a Hugging Face GGUF repository and enumerate the variants (quantizations and sharded folder groups) available for installation. Used by the lemonade pull <owner/repo> CLI flow and by the desktop app's model search to auto-populate the install form. The endpoint reads only public Hugging Face metadata; if the HF_TOKEN environment variable is set on the server, it is forwarded as a bearer token to access gated repositories.

Parameters

Example request:

curl 'http://localhost:13305/v1/pull/variants?checkpoint=unsloth/Qwen3-8B-GGUF'

Response

{
  "checkpoint": "unsloth/Qwen3-8B-GGUF",
  "recipe": "llamacpp",
  "suggested_name": "Qwen3-8B-GGUF",
  "suggested_labels": ["vision"],
  "mmproj_files": ["mmproj-model-f16.gguf"],
  "variants": [
    {
      "name": "Q4_K_M",
      "primary_file": "Qwen3-8B-Q4_K_M.gguf",
      "files": ["Qwen3-8B-Q4_K_M.gguf"],
      "sharded": false,
      "size_bytes": 4920000000
    },
    {
      "name": "Q8_0",
      "primary_file": "Q8_0/Qwen3-8B-Q8_0-00001-of-00002.gguf",
      "files": ["Q8_0/Qwen3-8B-Q8_0-00001-of-00002.gguf", "Q8_0/Qwen3-8B-Q8_0-00002-of-00002.gguf"],
      "sharded": true,
      "size_bytes": 8500000000
    }
  ]
}

Error responses

POST /v1/delete

Delete a model by removing it from local storage. If the model is currently loaded, it will be unloaded first.

Note: deleting a collection (recipe: "collection.omni") removes only the collection entry from user_models.json; its components stay on disk. Delete the components individually if you want to free their disk space.

Parameters

Example request:

curl -X POST http://localhost:13305/v1/delete \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Response format:

{
  "status":"success",
  "message":"Deleted model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

POST /v1/load

Explicitly load a registered model into memory. This is useful to ensure that the model is loaded before you make a request. Installs the model if necessary.

Note: loading a collection (recipe: "collection.omni") loads each of its components in turn. Per-model options like ctx_size or llamacpp_backend are not forwarded to components — set them on each component's own recipe_options.json entry instead.

Parameters

Setting Priority:

When loading a model, settings are applied in this priority order: 1. Values explicitly passed in the load request (highest priority) 2. Per-model values configurable in recipe_options.json (see below for details) 3. Values from environment variables or server startup arguments (see Server Configuration) 4. Default hardcoded values in lemond (lowest priority)

Per-model options

You can configure recipe-specific options on a per-model basis. Lemonade manages a file called recipe_options.json in the user's Lemonade cache (default: ~/.cache/lemonade). The available options depend on the model's recipe:

{
  "user.Qwen2.5-Coder-1.5B-Instruct": {
    "ctx_size": 16384,
    "llamacpp_backend": "vulkan",
    "llamacpp_args": "-np 2 -kvu"
  },
  "Qwen3-Coder-30B-A3B-Instruct-GGUF" : {
    "llamacpp_backend": "rocm"
  },
  "whisper-large-v3-turbo-q8_0.bin": {
    "whispercpp_backend": "npu",
    "whispercpp_args": "--convert"
  }
}

Note that model names include any applicable prefix, such as user. and extra..

Example requests

Basic load:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Load with custom settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "ctx_size": 8192,
    "llamacpp_backend": "rocm",
    "llamacpp_args": "--flash-attn on --no-mmap"
  }'

Load and save settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "ctx_size": 8192,
    "llamacpp_backend": "vulkan",
    "llamacpp_args": "--no-context-shift --no-mmap",
    "save_options": true
  }'

Load a Whisper model with NPU backend and conversion enabled:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "whisper-large-v3-turbo-q8_0.bin",
    "whispercpp_backend": "npu",
    "whispercpp_args": "--convert"
  }'

Load an image generation model with custom settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "sd-turbo",
    "steps": 4,
    "cfg_scale": 1.0,
    "width": 512,
    "height": 512
  }'

Response format

{
  "status":"success",
  "message":"Loaded model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

POST /v1/unload

Explicitly unload a model from memory. This is useful to free up memory while still leaving the server process running (which takes minimal resources but a few seconds to start).

Parameters

Example requests

Unload a specific model:

curl -X POST http://localhost:13305/v1/unload \
  -H "Content-Type: application/json" \
  -d '{"model_name": "Qwen3-0.6B-GGUF"}'

Unload all models:

curl -X POST http://localhost:13305/v1/unload

Response format

Success response:

{
  "status": "success",
  "message": "Model unloaded successfully"
}

Error response (model not found):

{
  "status": "error",
  "message": "Model not found: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

GET /v1/health

Check the health of the server. This endpoint returns information about loaded models.

Parameters

This endpoint does not take any parameters.

Example request

curl http://localhost:13305/v1/health

Response format

{
  "status": "ok",
  "version":"9.3.3",
  "websocket_port":9000,
  "model_loaded": "Llama-3.2-1B-Instruct-Hybrid",
  "all_models_loaded": [
    {
      "model_name": "Llama-3.2-1B-Instruct-Hybrid",
      "checkpoint": "amd/Llama-3.2-1B-Instruct-awq-g128-int4-asym-fp16-onnx-hybrid",
      "last_use": 1732123456.789,
      "type": "llm",
      "device": "gpu npu",
      "recipe": "ryzenai-llm",
      "pid": 12345,
      "recipe_options": {
        "ctx_size": 4096
      },
      "backend_url": "http://127.0.0.1:8001/v1"
    },
    {
      "model_name": "nomic-embed-text-v1-GGUF",
      "checkpoint": "nomic-ai/nomic-embed-text-v1-GGUF:Q4_K_S",
      "last_use": 1732123450.123,
      "type": "embedding",
      "device": "gpu",
      "recipe": "llamacpp",
      "pid": 12346,
      "recipe_options": {
        "ctx_size": 8192,
        "llamacpp_args": "--no-mmap",
        "llamacpp_backend": "rocm"
      },
      "backend_url": "http://127.0.0.1:8002/v1"
    }
  ],
  "max_models": {
    "transcription":1,
    "embedding":1,
    "image":1,
    "llm":1,
    "reranking":1,
    "tts":1
  }
}

Field Descriptions:

• status - Server health status, always "ok"
• version - Version number of Lemonade Server
• model_loaded - Model name of the most recently accessed model
• all_models_loaded - Array of all currently loaded models with details:
• model_name - Name of the loaded model
• checkpoint - Full checkpoint identifier
• last_use - Unix timestamp of last access (load or inference)
• type - Model type: "llm", "embedding", "reranking", "transcription", "image", or "tts"
• device - Space-separated device list: "cpu", "gpu", "npu", or combinations like "gpu npu"
• backend_url - URL of the backend server process handling this model (useful for debugging)
• pid - The Process ID (PID) of the backend engine handling this model
• recipe - Backend/device recipe used to load the model (e.g., "ryzenai-llm", "llamacpp", "flm")
• recipe_options - Options used to load the model (e.g., "ctx_size", "llamacpp_backend", "llamacpp_args", "whispercpp_args")
• max_models - Maximum number of models that can be loaded simultaneously per type (set via max_loaded_models in Server Configuration):
• llm - Maximum LLM/chat models
• embedding - Maximum embedding models
• reranking - Maximum reranking models
• transcription - Maximum speech-to-text models
• image - Maximum image models
• tts - Maximum text-to-speech models
• websocket_port - (optional) Port of the WebSocket server for the Realtime Audio Transcription API and Log Streaming API. Only present when the WebSocket server is running. The port is OS-assigned or set via --websocket-port.

GET /v1/stats

Performance statistics from the last request.

Parameters

This endpoint does not take any parameters.

Example request

curl http://localhost:13305/v1/stats

Response format

{
  "time_to_first_token": 2.14,
  "tokens_per_second": 33.33,
  "input_tokens": 128,
  "output_tokens": 5,
  "prompt_tokens": 9
}

Field Descriptions:

• time_to_first_token - Time in seconds until the first token was generated
• tokens_per_second - Generation speed in tokens per second
• input_tokens - Number of tokens processed
• output_tokens - Number of tokens generated
• prompt_tokens - Total prompt tokens including cached tokens

GET /metrics

Prometheus scrape endpoint for Lemonade Server. The endpoint returns Prometheus text exposition format and is intended to be scraped by Prometheus, not by Grafana directly.

Unlike most Lemonade API endpoints, /metrics is root-level only. It is not mounted under /api/v0/, /api/v1/, /v0/, or /v1/.

HEAD /metrics is also supported and returns 200 OK with an empty body.

Authentication

If LEMONADE_API_KEY is set, /metrics requires bearer authentication. Either the regular API key or LEMONADE_ADMIN_API_KEY is accepted.

If only LEMONADE_ADMIN_API_KEY is set and LEMONADE_API_KEY is unset, /metrics is accessible without authentication, matching regular API endpoint behavior.

Polling and Refresh Rate

The /metrics endpoint has no internal refresh timer. It renders the latest server state at the moment it is scraped.

Polling frequency is configured in Prometheus via scrape_interval, for example:

global:
  scrape_interval: 10s

Grafana queries Prometheus. Grafana's dashboard refresh controls how often panels query Prometheus, but it does not control how often Prometheus scrapes Lemonade.

Example request

curl http://localhost:13305/metrics

With API-key auth:

curl http://localhost:13305/metrics \
  -H "Authorization: Bearer $LEMONADE_API_KEY"

Response format

The response uses Prometheus text exposition format:

# HELP lemonade_server_up Whether the Lemonade server is running.
# TYPE lemonade_server_up gauge
lemonade_server_up 1
# HELP lemonade_server_info Lemonade server build information.
# TYPE lemonade_server_info gauge
lemonade_server_info{version="10.4.0"} 1

Content type:

text/plain; version=0.0.4; charset=utf-8

Lemonade Metric Families

The authoritative metric-family list is generated by the /metrics implementation in src/cpp/server/server.cpp. Search for handle_metrics and metrics.describe(...) to see the current names, types, labels, and descriptions.

Unsupported, unavailable, null, NaN, and infinity values are omitted rather than emitted as samples.

llama.cpp Backend Metrics

When a loaded model uses the llamacpp recipe, Lemonade makes a best-effort scrape of the loaded backend process's private /metrics endpoint. Backend scrape failures do not fail the Lemonade /metrics response.

Scraped llama.cpp metrics are normalized under the lemonade_llamacpp_* prefix and labeled with the same Lemonade model metadata used by lemonade_model_info.

Lemonade starts llama.cpp backends with metrics enabled so these backend metrics are available whenever the backend supports them.

GET /v1/system-info

System information endpoint that provides complete hardware details and device enumeration.

Example request

curl "http://localhost:13305/v1/system-info"

Response format

{
  "OS Version": "Windows-10-10.0.26100-SP0",
  "Processor": "AMD Ryzen AI 9 HX 375 w/ Radeon 890M",
  "Physical Memory": "32.0 GB",
  "OEM System": "ASUS Zenbook S 16",
  "BIOS Version": "1.0.0",
  "CPU Max Clock": "5100 MHz",
  "Windows Power Setting": "Balanced",
  "devices": {
    "cpu": {
      "name": "AMD Ryzen AI 9 HX 375 w/ Radeon 890M",
      "cores": 12,
      "threads": 24,
      "available": true,
      "family": "x86_64"
    },
    "amd_gpu": [
      {
        "name": "AMD Radeon(TM) 890M Graphics",
        "vram_gb": 0.5,
        "available": true,
        "family": "gfx1150"
      }
    ],
    "amd_npu": {
      "name": "AMD Ryzen AI 9 HX 375 w/ Radeon 890M",
      "power_mode": "Default",
      "available": true,
      "family": "XDNA2"
    }
  },
  "recipes": {
    "llamacpp": {
      "default_backend": "vulkan",
      "backends": {
        "vulkan": {
          "devices": ["cpu", "amd_gpu"],
          "state": "installed",
          "message": "",
          "action": "",
          "version": "b7869"
        },
        "rocm": {
          "devices": ["amd_gpu"],
          "state": "installable",
          "message": "Backend is supported but not installed.",
          "action": "lemonade backends install llamacpp:rocm"
        },
        "metal": {
          "devices": [],
          "state": "unsupported",
          "message": "Requires macOS",
          "action": ""
        },
        "cpu": {
          "devices": ["cpu"],
          "state": "update_required",
          "message": "Backend update is required before use.",
          "action": "lemonade backends install llamacpp:cpu"
        }
      }
    },
    "whispercpp": {
      "default_backend": "default",
      "backends": {
        "default": {
          "devices": ["cpu"],
          "state": "installable",
          "message": "Backend is supported but not installed.",
          "action": "lemonade backends install whispercpp:default"
        }
      }
    },
    "sd-cpp": {
      "default_backend": "default",
      "backends": {
        "default": {
          "devices": ["cpu"],
          "state": "installable",
          "message": "Backend is supported but not installed.",
          "action": "lemonade backends install sd-cpp:default"
        }
      }
    },
    "flm": {
      "default_backend": "default",
      "backends": {
        "default": {
          "devices": ["amd_npu"],
          "state": "installed",
          "message": "",
          "action": "",
          "version": "1.2.0"
        }
      }
    },
    "ryzenai-llm": {
      "default_backend": "default",
      "backends": {
        "default": {
          "devices": ["amd_npu"],
          "state": "installed",
          "message": "",
          "action": ""
        }
      }
    }
  }
}

Field Descriptions:

• System fields:
• OS Version - Operating system name and version
• Processor - CPU model name
• Physical Memory - Total RAM
• OEM System - System/laptop model name (Windows only)
• BIOS Version - BIOS information (Windows only)
• CPU Max Clock - Maximum CPU clock speed (Windows only)

• Windows Power Setting - Current power plan (Windows only)

• devices - Hardware devices detected on the system (no software/support information)

• cpu - CPU information (name, cores, threads)
• amd_gpu - Array of AMD GPUs, both integrated and discrete (if present)
• nvidia_gpu - Array of NVIDIA GPUs (if present)

• amd_npu - AMD NPU device (if present)

• recipes - Software recipes and their backend support status

• Each recipe (e.g., llamacpp, whispercpp, flm) contains:
  • default_backend - Preferred backend selected by server policy for this system (present when at least one backend is not unsupported)
  • backends - Available backends for this recipe
  • Each backend contains:
    • devices - List of devices on this system that support this backend (empty if not supported)
    • state - Backend lifecycle state: unsupported, installable, update_required, or installed
    • message - Human-readable status text for GUI and CLI users. Required for unsupported, installable, and update_required; empty for installed.
    • action - Actionable user instruction string. For install/update cases this is typically an exact CLI command; for other states it may be empty or another actionable value (for example, a URL).
    • version - Installed or configured backend version (when available)

POST /v1/install

Install or update a backend for a specific recipe/backend pair. If the backend is already installed but outdated, this endpoint updates it to the configured version.

Parameters

Example request

curl -X POST http://localhost:13305/v1/install \
  -H "Content-Type: application/json" \
  -d '{
    "recipe": "llamacpp",
    "backend": "vulkan",
    "stream": false
  }'

Response format

{
  "status":"success",
  "recipe":"llamacpp",
  "backend":"vulkan"
}

In case of an error, returns an error field with details.

POST /v1/uninstall

Uninstall a backend for a specific recipe/backend pair. If loaded models are using that backend, they are unloaded first.

Parameters

Example request

curl -X POST http://localhost:13305/v1/uninstall \
  -H "Content-Type: application/json" \
  -d '{
    "recipe": "llamacpp",
    "backend": "vulkan"
  }'

Response format

{
  "status":"success",
  "recipe":"llamacpp",
  "backend":"vulkan"
}

In case of an error, returns an error field with details.

Log Streaming API (WebSocket)

Stream server logs over WebSocket. Clients connect, send a subscribe message, and receive a snapshot of recent log history followed by live log entries as they occur.

Connection

The WebSocket server shares the same port as the Realtime Audio Transcription API. Discover the port via the /v1/health endpoint (websocket_port field), then connect:

ws://localhost:<websocket_port>/logs/stream

After connecting, send a logs.subscribe message to start receiving logs.

Client → Server Messages

Server → Client Messages

Example: Subscribe to Logs

Subscribe from the beginning (full backlog):

{
  "type": "logs.subscribe",
  "after_seq": null
}

Resume after a known sequence number (e.g., on reconnect):

{
  "type": "logs.subscribe",
  "after_seq": 1042
}

Example: Snapshot Response

{
  "type": "logs.snapshot",
  "entries": [
    {
      "seq": 1,
      "timestamp": "2025-03-30 14:22:01.123",
      "severity": "Info",
      "tag": "Server",
      "line": "2025-03-30 14:22:01.123 [Info] (Server) Starting Lemonade Server..."
    }
  ]
}

Example: Live Entry

{
  "type": "logs.entry",
  "entry": {
    "seq": 1043,
    "timestamp": "2025-03-30 14:22:05.456",
    "severity": "Info",
    "tag": "Router",
    "line": "2025-03-30 14:22:05.456 [Info] (Router) Model loaded successfully"
  }
}

Log Entry Fields

Integration Notes

• Reconnection: Track the last seq received and pass it as after_seq on reconnect to avoid duplicate entries.
• Backlog: The server retains up to 5000 recent log entries. The snapshot may be smaller if fewer entries exist.
• Platform availability: WebSocket log streaming is available on all platforms (Windows, Linux, and macOS).

GET /live

Lightweight liveness probe for load balancers and orchestrators. Unlike /v1/health, this endpoint does no work beyond confirming the process is up — it does not inspect loaded models or backends — so it is safe to poll at high frequency. HEAD /live is also supported and returns 200 OK with an empty body.

Unlike the other endpoints on this page, /live is not versioned and is not mounted under the /api/v0/, /api/v1/, /v0/, /v1/ prefixes.

Example request

curl http://localhost:13305/live

Response format

{"status":"ok"}