Assistant API

The POST /api/assistant endpoint is the streaming backend for an AI chat UI. It takes a list of chat messages, calls any OpenAI-compatible /v1/chat/completions endpoint with streaming enabled, and relays the model's output to the browser as Server-Sent Events — token by token, plus assembled tool calls. It is built on the framework's own pure-std akurai-llm client and its existing SSE support; there are no external dependencies and no bundler.

Request

POST /api/assistant
Content-Type: application/json

{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "tools": [
    {
      "type": "function",
      "function": { "name": "get_weather", "description": "…", "parameters": {} }
    }
  ]
}

• messages (required) — a non-empty array of { "role", "content" } objects.

Both fields must be strings. The roles are passed straight through to the model (system, user, assistant, …).

• tools (optional) — an OpenAI tools array, forwarded verbatim in the

upstream request. Omit it (or pass []) to disable tool calling.

A non-POST method returns 405. A malformed body does not fail the request — it streams a single error event (see below) so the client only ever has to handle one response shape.

Response — Server-Sent Events

The response is text/event-stream. Each event has a named event: type and a data: payload:

| Event | data: payload | Meaning | | ----------- | ------------------------------------------------ | -------------------------------------------------- | | delta | the token text (raw string) | A streamed chunk of assistant text. Concatenate. | | tool_call | JSON { "id", "name", "arguments" } | A fully-assembled tool call (one per call). | | done | [DONE] | The stream finished; no more events will arrive. | | error | JSON { "error": "<message>" } | A graceful failure; the stream then ends. |

Ordering: zero or more delta events, then zero or more tool_call events, then exactly one terminal done — unless an error is emitted instead. Tool calls are reassembled server-side from the upstream's streamed fragments, so each tool_call event carries the complete id, name, and arguments JSON string.

Example stream:

event: delta
data: Hello

event: delta
data: , world

event: tool_call
data: {"id":"call_1","name":"get_weather","arguments":"{\"location\":\"NYC\"}"}

event: done
data: [DONE]

Consume it from the browser with EventSource (for GET) or a fetch + ReadableStream reader (for this POST), dispatching on the event name.

Configuration

The endpoint and model are read from the environment at request time — nothing is hardcoded:

| Variable | Example | Purpose | | ------------------ | ----------------------------- | ---------------------------------------- | | AKURAI_LLM_URL | http://localhost:8080 | Base URL of the OpenAI-compatible server | | AKURAI_LLM_MODEL | gemma-3-12b | Model name sent in the request body |

If either variable is unset (or empty), the endpoint streams a single error event — LLM endpoint not configured (set AKURAI_LLM_URL and AKURAI_LLM_MODEL) — and ends. This keeps the chat UI functional and debuggable even before a backend is wired up.

AKURAI_LLM_URL=http://localhost:8080 \
AKURAI_LLM_MODEL=gemma-3-12b \
akurai serve

No TLS — point at a plain-HTTP origin

The akurai-llm client is built on std::net::TcpStream and speaks plain HTTP/1.1 only. The Rust standard library has no TLS, and the framework will not pull in a dependency for it. An https:// value for AKURAI_LLM_URL is rejected with a clear error event rather than silently failing.

This is by design and matches the rest of the framework: terminate TLS at the edge (Caddy, nginx, a cloud load balancer) and point AKURAI_LLM_URL at the plain-HTTP origin behind it — typically a local model server (http://localhost:8080) or an internal address on a trusted network. The endpoint understands both unchunked and Transfer-Encoding: chunked streaming responses, which is what local OpenAI-compatible servers normally send.