Intelligence / model routing
P1 · Intelligence
LLM Plane
Reach multiple model providers through one interface, with routing, budgets, and fallbacks in one place.
The unit of configuration is a capability, not a provider model. An application asks for summarize-document. Ultrahost decides which model runs it.
Paths in the tables are relative to https://api.ultrahost.com/v1/projects/{projectId}. Pagination, idempotency, and errors are described in Conventions.
Create
| SDK | REST | Description |
|---|---|---|
llm.createCapability(name, config) | POST /llm/capabilities | Define a capability |
llm.getCapability(name) | GET /llm/capabilities/{name} | Retrieve configuration |
llm.listCapabilities() | GET /llm/capabilities | List capabilities |
llm.updateCapability(name, config) | PATCH /llm/capabilities/{name} | Update routing and limits |
llm.deleteCapability(name) | DELETE /llm/capabilities/{name} | Remove a capability |
llm.listModels() | GET /llm/models | List supported models |
llm.getModel(modelId) | GET /llm/models/{modelId} | Model capabilities and pricing metadata |
await ultrahost.llm.createCapability("summarize-document", {
model: "balanced",
fallback: ["fast", "economy"],
maxOutputTokens: 2000,
responseFormat: "text",
budget: { monthlyUSD: 25 }
});model here is a routing class such as "balanced", not a provider-specific model id. Discovery of concrete models goes through listModels() and getModel().
Configure
| SDK | REST | Description |
|---|---|---|
llm.setBudget(name, budget) | PUT /llm/capabilities/{name}/budget | Set spending limits |
llm.setFallbacks(name, models) | PUT /llm/capabilities/{name}/fallbacks | Set fallback models |
Capabilities also cover:
- Provider-independent request and response shapes
- Model selection and routing
- Structured JSON output
- Streaming
- Prompt versioning
- Retry and fallback policy
- Token and cost accounting
- Usage per user, per project, and per capability
- Hard and soft spending limits
- Request tracing
- Discovery of which models are available and what they can do
Use
| SDK | REST | Description |
|---|---|---|
llm.generate(options) | POST /llm/generate | Text or structured output |
llm.stream(options) | POST /llm/generate/stream | Stream the output |
llm.chat(options) | POST /llm/chat | A conversational request |
llm.embed(options) | POST /llm/embeddings | Generate embeddings |
llm.countTokens(options) | POST /llm/tokens/count | Estimate token usage |
llm.usage(options) | POST /llm/usage/query | Query usage and cost |
llm.history(options) | GET /llm/requests | Request history |
llm.getRequest(requestId) | GET /llm/requests/{requestId} | Inspect one request |
const result = await ultrahost.llm.generate({
capability: "summarize-document",
input: documentText
});React callers use useLLM(capability) and useLLMStream(capability). Both take the capability name, not a provider model.