LLM Endpoint

LLM endpoints connect Eve to model providers such as OpenAI, Anthropic, OpenRouter, Ollama, and Azure OpenAI so traffic can be governed through one model path.

Endpoint Kind

Use llm as the endpoint kind when creating this endpoint through the API or dashboard.

What Eve Uses This Endpoint For

Route model requests through Eve with policy, telemetry, and cost controls.
Create governed model access for applications and agents.
Apply PII, budget, routing, and caching controls before requests reach providers.
Compare provider latency, errors, and cost across traffic.

Basic Endpoint Shape

json

{
  "endpoint": "openai-prod",
  "kind": "llm",
  "config": {
    "read_conn": {
      "provider": "OpenAI",
      "inline_api_key": "secret",
      "defaults": {
        "model": "gpt-4o"
      }
    }
  }
}

Create it with the endpoint API:

bash

curl -sS -X POST "$EDEN/endpoints" \
  -H "$AUTH_HEADER" \
  -H "Content-Type: application/json" \
  -d '{
  "endpoint": "openai-prod",
  "kind": "llm",
  "config": {
    "read_conn": {
      "provider": "OpenAI",
      "inline_api_key": "secret",
      "defaults": {
        "model": "gpt-4o"
      }
    }
  }
}'

Configuration Fields

Field	Purpose
`provider`	Model provider, such as OpenAI, Anthropic, Ollama, OpenRouter, or Azure OpenAI
`inline_api_key`	Provider API key when required
`model`	Default model
`base_url_override`	Optional compatible API base URL

Operating Notes

Use separate endpoints for materially different providers, environments, or routing policies.
Avoid embedding provider secrets in client applications; route clients through Eve-issued access.
Set model allowlists and budget controls before broad rollout.