Skip to content

Embeddable Lemonade: Runtime

This guide will show you how to operate Embeddable Lemonade in your app at runtime.

Contents:

Launching

We recommend that your app launches lemond as a subprocess, using a command like this:

set LEMONADE_API_KEY=KEY && lemond.exe ./ --port PORT
LEMONADE_API_KEY=KEY lemond ./ --port PORT

Breaking this down: - LEMONADE_API_KEY=KEY sets an API key for lemond known only to your app. This locks out other apps, as well as users, from interfacing directly with lemond's endpoints. - The positional ./ is the working directory for lemond, where it will look for config.json, bin/, etc. - --port PORT ensures that lemond launches on a specific port where your app will find it.

Authenticating Requests

If you launch lemond with LEMONADE_API_KEY set, your app must send that same key on every HTTP request to Lemonade endpoints. Do this by setting an Authorization header with a Bearer token:

Authorization: Bearer KEY

For example, with curl:

curl http://localhost:8000/v1/health ^
  -H "Authorization: Bearer KEY"
curl http://localhost:8000/v1/health \
  -H "Authorization: Bearer KEY"

For JSON POST requests:

curl -X POST http://localhost:8000/internal/set ^
  -H "Authorization: Bearer KEY" ^
  -H "Content-Type: application/json" ^
  -d "{\"log_level\": \"debug\"}"
curl -X POST http://localhost:8000/internal/set \
  -H "Authorization: Bearer KEY" \
  -H "Content-Type: application/json" \
  -d '{"log_level": "debug"}'

In JavaScript:

await fetch("http://localhost:8000/v1/models", {
  headers: {
    Authorization: `Bearer ${apiKey}`,
  },
});

This matches the existing CLI, tray, app, and test implementations in this repo. If the header is missing or the key is wrong, lemond will reject the request with 401 Unauthorized.

Runtime Model and Backend Management

lemond provides a full set of endpoints for managing models and backends at runtime.

Endpoint Description
POST /v1/pull Download a model
POST /v1/delete Delete a downloaded model
POST /v1/load Load a model into memory
POST /v1/unload Unload a model from memory
POST /v1/install Install or update a backend
POST /v1/uninstall Remove a backend
GET /v1/models List available models
GET /v1/health Server status and loaded models

See the Endpoints Spec for full request/response details.

Runtime Settings and Shutdown

Your app can manage its lemond instance at runtime by using /internal endpoints. See the Internal API for their request and response formats.

Endpoint Description
POST /internal/set Change config.json settings without restarting lemond
GET /internal/config Read the current settings
GET /internal/config/defaults Read the factory default settings
POST /internal/pin Pin or unpin a loaded model so eviction skips it
POST /internal/shutdown Unload every model and stop lemond

See the Configuration Guide for every setting.