Model Configuration

This guide explains how Self-Managed Kindo installations define and maintain global models — the system-wide LLMs (and related models like embeddings or transcription) that power workflows, chat, ingestion, and other core experiences.

In a self-managed environment, you add models to Kindo, map them to Unleash feature variants, and keep those mappings up to date when models are replaced.

Deploying model infrastructure?

Before configuring models in Kindo, you need working inference endpoints. See Prepare AI Models for how to deploy and verify self-hosted models (vLLM, context window sizing, tool-call parsers) or connect cloud API providers.

Unleash Features and Strategy Variants

Each key below is an Unleash feature whose variant payload contains one or more global model IDs. The backend and frontend read these variants to decide which models to use.

Required Features

Feature Key	Description
AUDIO_TRANSCRIPTION	Converts audio to text
CRON_EXPRESSION_GENERATION	Generates cron expressions from natural language schedule descriptions
DEFAULT_WORKFLOW_STEP_MODEL	Default model for workflow step execution
EMBEDDING_MODELS	Models for generating text embeddings
INGESTION_WORKERS	Models for data ingestion and extraction
INTERNAL_AUTO_GENERATION	Internal model for automatic content generation and internal worker tasks
LOCAL_STORAGE_CHAT_ACTIONS_MODELS	Fallback default chat model for the model dropdown and chat action selection. Applies when an org admin has not chosen a default chat model in Settings → Models (or the chosen model is unavailable)
TOOL_CALLING_MODELS	Models that support tool/function calling (used to filter model selection in chat UI)

Optional Features

Feature Key	Description	Notes
API_STEP_GENERATION	Models for generating API steps in workflows
DYNAMIC_API_REQUEST_PARSER	Parses dynamic API requests
SLACK_MESSAGE_GENERATION	Generates Slack messages	Only if Slack integration is enabled
MULTIMODAL_MODELS	Models with image/multimodal input support	Only if multimodal chat is enabled

Minimum Model Requirements

Self-Managed Kindo needs baseline global models to function. Without them, core flows like chat, workflow execution, ingestion, and indexing will fail.

Absolute Minimum Set

Model Type	Purpose	Recommendations
Embedding model	Semantic search, retrieval, and indexing	Open source: BAAI/bge-m3. Hosted: Gemini embedding
Strong/large model	Complex reasoning, workflow execution, long-form generation	Open source: gpt-oss 120B. Hosted: Gemini 2.5 Pro, Claude 4.5, GPT 5.2
Audio transcription	Audio file processing, voice notes, transcription workflows	Open source: Faster Whisper. Hosted: Deepgram

Recommended Additional Models

• Small model (title generation, quick summaries, low-latency steps): Gemma 3 or Llama 3.2 7B
• Security-focused: DeepHat 32B
• Multimodal: Gemini 2.5 Pro (image + text)

Managing Models

Overview of Steps

1. Add the new model in Settings → Models → Add model.
2. Update Unleash variants to reference the new model IDs.
3. Delete the old model with replacement.

If you are operating through the released Helmfile installer, the equivalent workflow is make add-models, then make sync-model-flags. That path updates only the model ID payloads for the model-related flags instead of re-importing the full Unleash export.

Caution

If the model you are deleting is still referenced by Unleash, the response will list the feature flags that need updates. Update those variants first, then retry the delete.

Add a Self-Managed Model in the UI

A self-managed model is a model you already run or pay for somewhere else — a self-hosted vLLM box, a serverless endpoint such as Together, Fireworks, or Groq, an internal inference gateway, or your own OpenAI account — that you want to use inside Kindo for chat and agents. The endpoint must be OpenAI-compatible (it accepts OpenAI-style chat completion requests). The Add Model form asks only for the handful of facts any such endpoint needs, in plain English.

Field	Required?	What to enter
Display name	Yes	The name you’ll see in Kindo when choosing this model, for example Llama 3.3 70B (self-managed).
Model ID	Yes	The model’s name exactly as your endpoint expects it in a request, for example Qwen/Qwen3.7-Max, llama-3.3-70b, or gpt-4o. Use exactly what your provider or server calls it. For Azure OpenAI, use your deployment name.
Endpoint URL	Yes	The base URL of your model’s API, for example https://api.together.xyz/v1, http://my-vllm:8000/v1, or https://your-resource.openai.azure.com. Paste exactly what your provider gives you — Kindo tidies it up (trims it, drops a trailing slash or a mis-pasted /chat/completions, and adds the version segment if your base URL doesn’t already include one).
API version	No	Leave blank for most providers. Only needed when your endpoint requires an api-version query parameter — for example an Azure OpenAI deployment pinned to a dated version such as 2024-10-21.
API key	Yes	The key your endpoint uses to authenticate. Stored encrypted; never shown again.
Context window	Yes	Your model’s maximum context length in tokens, from your provider’s docs, for example 128000. Kindo uses this to keep long conversations within the model’s limit.
Max output tokens	Yes	The maximum number of tokens the model should generate per response, from your provider’s docs, for example 4096.

Endpoint URL examples

Paste the base URL your provider hands you. Common shapes already work as-is:

• OpenAI: https://api.openai.com/v1
• Together: https://api.together.xyz/v1
• Groq: https://api.groq.com/openai/v1
• Fireworks: https://api.fireworks.ai/inference/v1
• Self-hosted vLLM: http://<host>:8000 (Kindo appends /v1 for you)
• Azure OpenAI: https://<your-resource>.openai.azure.com (Kindo routes to Azure’s OpenAI-compatible /openai/v1 surface for you)

Azure OpenAI

Azure OpenAI endpoints work like any other OpenAI-compatible endpoint:

• Most deployments: paste your resource URL (for example https://my-resource.openai.azure.com), set Model ID to your deployment name, and leave API version blank. Kindo talks to Azure’s OpenAI-compatible v1 surface, which needs no api-version parameter.
• Deployments pinned to a dated API version: fill in the API version field (for example 2024-10-21) and Kindo dispatches through Azure’s versioned API instead. Pasting the full target URI from the Azure portal (https://…/openai/deployments/<name>/chat/completions?api-version=…) also works — Kindo picks up the api-version from the URL automatically.

Note

The URL must point at your chat completions endpoint. URLs for other APIs (for example ones ending in /embeddings or /responses) are rejected so a wrong paste can’t silently mis-configure the model.

Test & Save

Test & Save (the primary button) sends one real chat completion to your endpoint before anything is saved. If the test fails, nothing is created — the form stays filled in and the error points at the field to fix: a rejected key highlights API key, an unknown model name highlights Model ID, an unreachable or timed-out endpoint highlights Endpoint URL, and a too-high Max output tokens or a rejected request parameter is called out by name. From the failure banner you can either Edit the form and test again, or Save anyway to store the config without a passing test.

Save without testing skips the test entirely — useful when the endpoint is temporarily down, a self-hosted model is cold-starting, or you don’t want to spend a request on a quota-limited endpoint.

The model list shows the outcome as a chip next to each self-managed model: Verified (last test passed), Untested (saved without a passing test), or Failed (a later re-test failed — hover the chip for the reason). Kindo’s built-in global models are managed for you and carry no chip.

Models with Multiple Providers (Model Groups)

A model in Kindo is not tied to one endpoint. You can attach several provider endpoints to the same model — for example the same model served from two regions of your cloud provider, or a self-hosted model behind two vLLM endpoints. In the Add Model form, pick an existing model in the Model field instead of typing a new name, and the endpoint you are configuring is added as another provider for it. Attaching and reordering a model’s providers is an organization-admin action on the Enterprise tier, in installations where multi-provider models are enabled; the endpoints you attach belong to the model’s own provider.

People always pick the model, never a specific provider. Which of the model’s providers serves a given request is not decided until the request is already on its way, by the model’s fallback chain and its session affinity.

Because of that, a model group behaves as the lowest common denominator of its providers:

Control	How the group resolves it	Effect
Context window	Smallest across the model’s enabled providers	A 32k provider under a 200k model makes the whole model 32k.
Tool limit	Smallest across the model’s enabled providers	An OpenAI or Azure OpenAI endpoint accepts at most 128 tools per request, so one of them makes 128 tools the model’s effective limit.
DLP filters	Strictest across the model’s providers	A filter that redacts for any provider redacts for the whole model.
Provider enablement	Strictest across the model’s providers	A provider your organization has disabled in its security controls makes the whole model unavailable, even where that provider is attached to the model only through a disabled endpoint. (Turning off one endpoint in the model’s provider list is a different thing — see Fallback chain.)

Why the group uses the minimum

Two reasons, both about the request being decided before the provider is:

• Routing is blind and failover is silent. Kindo has to fix the capacity budget for a request before it knows which provider will serve it — and a request that fails over mid-flight can land on a different provider than it started with. The only budget that is valid for every provider the request could reach is the smallest one.
• Runs have to replay identically. The budget is recorded when the request starts so that a resumed or retried run reproduces exactly the same conversation. If the budget tracked whichever provider happened to answer, a retry that fell back to a smaller provider would overflow it.

Group providers with like-for-like capacity

Pick providers that are close to each other in capability, so nothing is stranded at the floor:

• Match context windows. Attaching a 32k endpoint to a 200k model does not add 32k of capacity — it removes 168k from the model. If a provider is much smaller, give it its own model instead, so people can choose it deliberately.
• Match tool support. If tool-heavy agents use the model, an OpenAI or Azure OpenAI endpoint accepts at most 128 tools per request — a request carrying more can fail whenever that endpoint serves it, so the model’s effective tool limit becomes 128.
• Match governance. A provider your organization has disabled, or one with stricter DLP filters, applies its rules to the whole model, not just to the requests it serves.

The Add Model form tells you about this before you save: once you pick a model to attach to, it shows the context window the new endpoint should clear and warns you when the endpoint would newly limit the model’s tool use.

Fallback chain

The providers attached to a model form a fallback chain: an ordered list, lowest priority number first, and that is the order the provider list shows in Settings → Models. Move a provider up or down to change the chain. Turning a provider off takes its endpoint out of the chain — the model’s other endpoints keep serving — and leaves its configuration in place so you can turn it back on.

Two things move a request off the provider that was chosen for it:

• Retries. A transient failure — a timeout, a rate limit, or a provider-side error — is retried inside the model, up to twice. Failures a retry cannot fix, such as a malformed request, are not retried.
• Descent to the next priority level. When the retries are used up, the request is re-aimed at the next priority level down the chain. This is also how an interrupted streamed response continues: it resumes on the next level, and text already streamed to the reader is not repeated. Most errors take this path once, not only transient ones — a request whose content the model rejected outright, or one that overflowed the context window, is returned to the caller instead.

Descending the chain needs at least two distinct priority levels. An endpoint you attach starts at the top priority, which ties it with the provider the model already had, and providers that share a priority number share traffic rather than one preferring the other — a retry can still move a failed request to the other, but there is no lower level to descend to. So if you want a deliberate primary with a backup behind it, move the rows up or down once: the list renumbers into distinct levels and the chain becomes strict.

A provider that fails persistently is put in cooldown: it is dropped from the chain for about half a minute and then becomes eligible again on its own, so a hard-down endpoint is not re-tried by every request in the meantime and recovery needs no administrator action. Cooldown only ever removes a provider that has an alternative — a model with a single provider is never cooled out of its own chain, because there would be nothing left to serve it.

Session affinity

Providers cache the part of a prompt they have already seen, and that cache lives on the endpoint that served the request — it is not shared between two endpoints of the same model. Because a conversation resends its whole history on every turn, a cache hit is the difference between the provider re-reading the entire conversation and re-reading only the newest turn. That drives both latency (how fast a long conversation starts responding) and cost (cached input is billed at a fraction of fresh input). A long conversation bounced between two providers would pay full price for the same history again on every switch.

So once a conversation is under way, Kindo keeps it on the provider that served its earlier turns where it can — session affinity:

• The pin is per conversation, and it is best effort. It exists to keep a cache warm; it is not a promise about which provider answers you. It covers the conversations Kindo drives — chat and agent runs. Requests sent straight to the Kindo APIs are not pinned.
• The pin expires. It is upper-bounded by a time-to-live (one hour by default), so a conversation picked back up much later is routed fresh. The lifetime is deliberately longer than the provider cache windows it protects: a pin whose cache has already lapsed simply re-warms the same endpoint.
• Health beats affinity. If the pinned provider is in cooldown or otherwise out of the healthy set, the conversation falls through to a surviving provider and pins to whichever one serves it. Affinity never holds a conversation on a dead endpoint.
• Until it is cooled out, though, the pinned provider is still preferred — including over a healthy provider at a lower priority level. A conversation can therefore keep attempting a failing endpoint for the turn or two it takes cooldown to remove it, which is what recovery looks like from the inside rather than a stuck conversation.
• A single-provider model is unaffected — with one endpoint there is only one choice.

What this means when you plan a model’s providers: adding a second provider to a busy model does not cool the caches you already have, because conversations in flight stay where they are and only later ones spread out. It also means the providers worth grouping are the ones that are close in capability (see above) — a chain whose members are interchangeable is the chain that keeps caches warm, since every fallback lands on a cold cache.

One provider is a complete configuration

A model has exactly one provider configured unless an administrator attaches another, and that is the state every model starts in. A single-provider model has no chain to descend and no pin worth observing, so everything above describes behavior you opt into per model, one provider at a time — there is nothing to migrate, nothing to re-configure, and nothing required of an administrator who never attaches a second provider.

Add a Global Model via API

curl -X POST <API_URL>/internal/openapi/admin/model/new \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <UM_INTERNAL_API_KEY>' \
  -d '{
    "orgId": "<YOUR_ORG_ID>",
    "userId": "<YOUR_USER_ID>",
    "displayName": "Model Display Name",
    "modelProviderDisplayName": "Provider Name",
    "type": "CHAT",
    "contextWindow": 200000,
    "metadata": {
      "type": "Text Generation",
      "costTier": "HIGH",
      "usageTag": "Chat + Agents",
      "description": "Model description",
      "modelCreator": "Provider Name"
    },
    "litellmModelName": "model-name",
    "litellmParams": {
      "model": "provider/model-name",
      "api_key": "<API_KEY>",
      "max_tokens": 64000
    }
  }'

Note

Ensure modelProviderDisplayName matches the display name in the ModelProvider table. Otherwise, a new ModelProvider record will be created. Also verify that LiteLLM params are correct for the specific model, as defaults vary by provider.

Delete with Replacement

curl -X POST <API_URL>/internal/openapi/admin/model/delete-with-replacement \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <UM_INTERNAL_API_KEY>' \
  -d '{
    "deletingModelId": "<OLD_MODEL_ID>",
    "replacementModelId": "<NEW_MODEL_ID>",
    "orgId": "<YOUR_ORG_ID>",
    "userId": "<YOUR_USER_ID>"
  }'

Update Parameters on an Existing Model

To modify LiteLLM parameters for an existing model, update the LiteLLM database directly:

UPDATE "public"."LiteLLM_ProxyModelTable"
SET "litellm_params" = jsonb_set(
  "litellm_params",
  '{max_tokens}',
  '64000'::jsonb
)
WHERE "model_name" = 'your-model-name';

Tip

When adding models to dev/prod, be mindful of which region you are using to manage quota efficiently. Separate dev and prod to avoid rate-limit outages during local testing.