LLM
Render a prompt per record and store the model's completion on it.
Render a prompt per record and store the model's completion on it.
When to use it
Render a prompt for each record and save the response in the chosen field. Keep source text distinct from your instructions. Choose a model that supports any supplied images and inspect per-record errors and budget outcomes. A plausible completion still needs checking against its source.
Add and configure
- Add LLM from the workflow step picker.
- Give the step a recognisable name and connect its required inputs.
- Set the options below. Nested names identify fields within an object or a list item.
- Validate the recipe, inspect the test result, then run it on the intended data.
The step identifier used in the editor’s code view is llm_complete.
Settings
| Setting | Accepted value | Required | Default | What it controls |
|---|---|---|---|---|
Concurrency (concurrency) | integer; minimum 1; maximum 16 | No | 2 | Maximum items or requests processed concurrently within this step. More concurrency can increase resource use and provider pressure. |
Handler Timeout Seconds (handler_timeout_seconds) | number; minimum 1.0; maximum 1800.0 or null | No | null | Wins over timeout_seconds when set, so a long fan-out can give each call its own bound while timeout_seconds sizes the whole step. Unset → timeout_seconds keeps its historical double duty. |
images (images) | object or null | No | null | Page images supplied by an earlier step, for a model offering image input. Missing, invalid, or excessive images produce a per-record error. Images are omitted from output unless explicitly retained. |
Detail (images.detail) | "auto" / "low" / "high" or null | No | null | Explicit-only OpenAI image_url.detail hint, sent verbatim when set (a vLLM-served route ignores it). |
Field (images.field) | string; minimum characters 1 | Yes | Not specified | Record field (dotted path allowed) holding the image(s): a data:image/png|jpeg|webp;base64,... URL, a list of them, or a file envelope (mime_type + bytes_base64). Nothing is fetched: an http(s) URL is refused per record. |
Keep In Output (images.keep_in_output) | boolean | No | false | Off (default), the image field is stripped from the output record — a page render is never copied into the run store or Knowledge. |
Max Image Bytes (images.max_image_bytes) | integer; minimum 1024; maximum 8388608 | No | 2097152 | Decoded size ceiling per image; a larger one is a per-record error. |
Max Images (images.max_images) | integer; minimum 1; maximum 8 | No | 1 | — |
Required (images.required) | boolean | No | true | A record without an image is a per-record image_missing error (no call is made). Off, such a record is sent as plain text. |
Max Tokens (max_tokens) | integer; minimum 1; maximum 128000 | No | 1024 | Maximum tokens in a model reply. A cap that is too small can truncate structured output. |
Model (model) | string | No | "openai:gpt-6-luna" | Model id, provider-prefixed — e.g. openai:gpt-6-luna or anthropic:claude-sonnet-5. |
Output Path (output_path) | string | No | "completion" | Top-level field on each output record where the LLM response text is stored. |
Prompt Template (prompt_template) | string | No | "Summarize the following record in one short paragraph:\n\n${record}" | Per-record prompt template. ${field} placeholders are replaced with values from the record (dotted paths supported, e.g. ${user.name}). ${record} expands to the whole record as compact JSON — unless the record has a real record field, which always wins. Defaults to a one-paragraph summary of ${record}. |
Quota Degrade Contract (quota_degrade_contract) | string or null | No | null | Required with quota_policy: degrade: the declared downstream consumer of the degraded records — a step id or a short contract name (e.g. 'fallback_in filter -> propose'). Stamped on every degraded record and named in the step's log line. It is a DECLARATION: the author names the intended reader; the handler does not verify that such a step exists in the DAG. |
Quota Policy (quota_policy) | "wait" / "degrade" | No | "wait" | What a reservation the daily ceiling cannot admit does. wait (default): the step raises QuotaExhaustedError and the runner waits for the budget then retries the step. degrade: EVERY record of the step becomes {**record, error: 'quota_exhausted', error_kind: 'quota_exhausted', quota_degrade_contract: } (all-or-nothing per step, no HTTP call is made) so the sibling filter/fallback step named by quota_degrade_contract can react; requires that contract to be declared. |
Reasoning Effort (reasoning_effort) | "none" / "minimal" / "low" / "medium" / "high" / "xhigh" or null | No | null | Explicit-only. When set, sent VERBATIM as reasoning_effort on the OpenAI Chat Completions body — accepted ONLY where the provider takes it, per model line: gpt-5/-mini/-nano minimal|low|medium|high; gpt-5.1 none|low|medium|high; gpt-5.2/5.4/5.5 none|low|medium|high|xhigh; the o-series low|medium|high; o1-mini/o1-preview none at all; an unlisted GPT-5/o-series line only low|medium|high. The local Qwen router rejects every explicit value, including none; omit this field to choose provider-default reasoning. Any other combination is rejected at create time. Unset, the key is absent and every body is byte-identical to before. A reasoning model spends its reasoning tokens INSIDE max_tokens, so a low effort leaves the cap to the visible reply. |
Reserve Prompt Tokens (reserve_prompt_tokens) | boolean | No | true | Reserve, against the tenant's DAILY token ceiling, the sum over the step's records of (a conservative token count of the rendered prompt + system prompt: (cl100k count x margin) + 32 per call, margin 1.1 for openai:* and 1.35 for anthropic:* (its tokenizer runs denser), or (UTF-8 bytes x margin) + 32 without a tokenizer) + max_tokens — what the step can actually spend — before the first call. False keeps the legacy flat guess of (max_tokens + 2000) x records, which under-reserves a long prompt and over-reserves a short one. Daily-ceiling arithmetic: the default ceiling is 100k tokens/day, so e.g. 24 records x (a 15k-token prompt = 16.5k + 4k max_tokens) reserves ~493k and cannot run under the default; the tenant setting (llm_daily_token_limit) must admit the step's reservation for it to start at all. |
Response Format (response_format) | "text" / "json_object" or null | No | null | Explicit-only. When set, sent as response_format: {"type": <value>} on the OpenAI Chat Completions body: json_object turns on the provider's JSON mode (the reply is constrained to parse as JSON), text pins the provider default. SYNTAX ONLY: the prompt (template or system) must still ask for JSON — OpenAI-compatible servers require the word JSON in the messages and 400 otherwise — and no schema is enforced. Set reasoning_effort only when that route declares support; otherwise leave it unset and use the provider's reasoning default. Accepted for openai: models only (the Anthropic Messages API has no response_format; rejected at create time). Unset, the key is absent and every body is byte-identical to before. |
System (system) | string or null | No | null | Optional system prompt — applies to every record. |
Temperature (temperature) | number; minimum 0.0; maximum 2.0 | No | 0.3 | Sampling setting where the model supports it. Fixed-sampling models do not receive this option. |
Timeout Seconds (timeout_seconds) | number; minimum 1.0; maximum 1800.0 | No | 60.0 | Deprecated alias for the per-request provider timeout — handler_timeout_seconds wins when present — AND the step's wall-clock timeout: the runner reads this key as the step's asyncio.wait_for window, so retries and every record's call (at concurrency) must fit inside it. 1800 s funds a long-prompt map step over tens of records; a value the step cannot spend just leaves the window open. |
Inputs and outputs
| Direction | Port | Content | Required | Multiple records |
|---|---|---|---|---|
| Inputs | in | records | Yes | Yes |
| Outputs | out | records | Yes | Yes |
Effects, testing, and recovery
Calls a model and can consume model budget.
The declared preview behaviour is to execute this step. Reads, processing, or model usage can therefore be real. Read dry runs before using a preview with live connections.
No duplicate-prevention key is declared for this step. Do not assume that manually repeating the step is free of repeated effects.
When step caching is available, the declared cache lifetime is 86400 seconds. Eligible successful results can be reused; bypass caching when the task requires a fresh read.
Example: Summarize each record in one sentence
Step settings:
{
"max_tokens": 120,
"model": "openai:gpt-6-luna",
"output_path": "summary",
"prompt_template": "Summarize in one sentence: ${record}"
}
Example: Classify a ticket into a JSON label
Step settings:
{
"model": "openai:gpt-6-luna",
"output_path": "label",
"prompt_template": "Ticket title: ${title}\nBody: ${body}",
"response_format": "json_object",
"system": "You label support tickets. Reply with JSON {\"label\": ...}.",
"temperature": 0.0
}
Check the result
Inspect the selected run’s status, counts, errors, and output records. An empty collection can be a valid result, and a completed step can contain per-record failures where the step defines that behaviour. Check the next step’s expected record shape before publishing or delivering the result.