Gatez

Appearance

LLM Provider Management API ​

Endpoints for managing LLM provider configurations, API keys, and provider testing.

Base URL: Control Plane API (default: http://localhost:4001)

Authentication: These endpoints are part of the Operator Portal and require platform admin authentication.

Overview ​

The Provider Management API allows platform administrators to:

  • Configure LLM providers (OpenAI, Anthropic, Gemini, etc.)
  • Add custom OpenAI-compatible providers
  • Manage API key sources (environment variables, Vault, K8s Secrets, AWS/Azure secret managers)
  • Test provider connectivity and discover available models
  • Enable/disable providers dynamically

Key Features:

  • Model Passthrough: No need to register individual models. Any model matching a provider's prefix pattern routes automatically.
  • Secret Reference Store: API keys are never stored in plaintext. Keys are referenced from env vars or external secret managers.
  • Built-in Providers: 10 pre-configured providers (OpenAI, Anthropic, Gemini, Groq, Mistral, DeepSeek, Together, Fireworks, Cohere, Ollama).

Endpoints ​

GET /api/providers ​

List all configured LLM providers with masked API keys.

Response:

json
{
  "providers": [
    {
      "id": "openai",
      "display_name": "OpenAI",
      "type": "openai",
      "base_url": "https://api.openai.com",
      "auth_type": "bearer",
      "key_source": {
        "type": "env_var",
        "var_name": "OPENAI_API_KEY"
      },
      "model_patterns": ["gpt-*", "o1-*", "o3-*", "chatgpt-*", "dall-e-*"],
      "default_models": ["gpt-4o", "gpt-4o-mini", "o3-mini"],
      "enabled": true,
      "status": "valid",
      "last_tested": "2026-03-27T10:30:00Z",
      "discovered_models": ["gpt-4o", "gpt-4o-mini", "gpt-4-turbo", "o3-mini"]
    },
    {
      "id": "anthropic",
      "display_name": "Anthropic",
      "type": "anthropic",
      "base_url": "https://api.anthropic.com",
      "auth_type": "x-api-key",
      "key_source": {
        "type": "env_var",
        "var_name": "ANTHROPIC_API_KEY"
      },
      "model_patterns": ["claude-*"],
      "default_models": ["claude-sonnet-4-20250514", "claude-haiku-4-20250514"],
      "enabled": true,
      "status": "valid",
      "last_tested": "2026-03-27T10:30:00Z",
      "discovered_models": []
    }
  ]
}

Field Descriptions:

  • id: Unique provider identifier (lowercase, no spaces)
  • display_name: Human-readable provider name
  • type: Provider type (openai, anthropic, gemini, cohere, ollama, openai_compatible)
  • base_url: API base URL
  • auth_type: Authentication header type (bearer, x-api-key, api-key, none)
  • key_source: How the API key is stored/retrieved (see Key Source Types below)
  • model_patterns: Prefix patterns for routing (e.g., gpt-* matches gpt-4o, gpt-4o-mini, etc.)
  • default_models: Well-known models shown in UI
  • enabled: Whether the provider is active
  • status: Test status (untested, valid, invalid, error)
  • last_tested: ISO 8601 timestamp of last test
  • discovered_models: Models found via /v1/models endpoint (populated after test)

Key Source Types:

typescript
// Environment Variable
{
  "type": "env_var",
  "var_name": "OPENAI_API_KEY"
}

// HashiCorp Vault
{
  "type": "vault",
  "path": "secret/data/llm/openai",
  "field": "api_key"
}

// Kubernetes Secret
{
  "type": "k8s_secret",
  "name": "gatez-llm-secrets",
  "key": "openai-api-key"
}

// AWS Secrets Manager
{
  "type": "aws_secrets_manager",
  "secret_id": "prod/llm/openai"
}

// Azure Key Vault
{
  "type": "azure_key_vault",
  "vault_url": "https://gatez-vault.vault.azure.net",
  "secret_name": "openai-api-key"
}

// Manual (Development Only - NOT ALLOWED IN PRODUCTION)
{
  "type": "manual",
  "masked_value": "****xyz"  // Only last 4 chars shown
}

// None (for local providers like Ollama)
{
  "type": "none"
}

POST /api/providers ​

Add a new provider or update an existing one.

Request Body:

json
{
  "id": "perplexity",
  "display_name": "Perplexity",
  "type": "openai_compatible",
  "base_url": "https://api.perplexity.ai",
  "auth_type": "bearer",
  "key_source": {
    "type": "env_var",
    "var_name": "PERPLEXITY_API_KEY"
  },
  "model_patterns": ["pplx-*", "sonar-*"],
  "default_models": ["pplx-70b-online"],
  "enabled": true,
  "status": "untested"
}

Response (Success):

json
{
  "status": "saved",
  "provider": "perplexity"
}

Response (Error - Manual Key in Production):

json
{
  "status": "error",
  "message": "Manual key entry is not allowed in production. Use environment variables (OPENAI_API_KEY, etc.) or configure a secret manager (Vault, K8s Secrets, AWS Secrets Manager)."
}

Notes:

  • If id already exists, the provider will be updated (upsert behavior).
  • manual key source is rejected in production for security reasons. Set ENVIRONMENT=development to allow manual keys for local testing.

DELETE /api/providers/:id ​

Remove a provider configuration.

Path Parameters:

  • id: Provider ID (e.g., openai, anthropic)

Response:

json
{
  "status": "deleted",
  "id": "perplexity"
}

Notes:

  • Built-in providers can be deleted but will reappear on restart (loaded from defaults). To permanently disable a built-in provider, set enabled: false instead.

POST /api/providers/:id/test ​

Test a provider's API key by calling its /v1/models endpoint (or equivalent).

Path Parameters:

  • id: Provider ID to test

Response (Valid Key):

json
{
  "status": "valid",
  "models_discovered": 15,
  "models": [
    "gpt-4o",
    "gpt-4o-mini",
    "gpt-4-turbo",
    "o3-mini",
    "dall-e-3",
    ...
  ]
}

Response (Invalid Key):

json
{
  "status": "invalid",
  "http_status": 401,
  "message": "Incorrect API key provided..."
}

Response (Error - Provider Not Reachable):

json
{
  "status": "error",
  "message": "error sending request for url (https://api.openai.com/v1/models): connection timeout"
}

Response (Error - Key Source Not Supported):

json
{
  "status": "error",
  "message": "Key source type not yet supported for testing. Set the key via environment variable for now."
}

Notes:

  • Updates the provider's status, last_tested, and discovered_models fields.
  • For Anthropic, tests with a minimal /v1/messages request since they don't have a /v1/models endpoint.
  • Timeout: 10 seconds.
  • Only env_var, manual, and none key sources are currently supported. Vault/K8s/AWS/Azure secret resolution is not yet implemented.

Provider Types ​

openai ​

  • Base URL: https://api.openai.com
  • Auth: Bearer token
  • Models endpoint: /v1/models

anthropic ​

  • Base URL: https://api.anthropic.com
  • Auth: x-api-key header
  • Models endpoint: None (uses /v1/messages for testing)

gemini ​

  • Base URL: https://generativelanguage.googleapis.com/v1beta/openai
  • Auth: Bearer token
  • Models endpoint: /v1/models

cohere ​

  • Base URL: https://api.cohere.com
  • Auth: Bearer token
  • Models endpoint: /v2/models

ollama ​

  • Base URL: http://host.docker.internal:11434 (Docker), http://localhost:11434 (Native)
  • Auth: None
  • Models endpoint: /v1/models

openai_compatible ​

Generic type for any OpenAI-compatible API (Groq, Mistral, DeepSeek, Together, Fireworks, Perplexity, Cerebras, etc.)

  • Auth: Usually Bearer token
  • Models endpoint: /v1/models

Model Passthrough Routing ​

Gatez uses prefix pattern matching to route model IDs to providers. You don't need to register individual models.

Example:

  • Request: POST /v1/chat/completions with "model": "gpt-4o-2024-08-06"
  • AI Gateway checks: Does gpt-4o-2024-08-06 match any provider's patterns?
  • openai provider has pattern gpt-* → Match!
  • Request is routed to OpenAI with model gpt-4o-2024-08-06

Benefits:

  • New models from providers work immediately (no code changes)
  • Fine-tuned models work (ft:gpt-4o-mini-2024-07-18:org:suffix)
  • Custom local models work with Ollama (mistral:7b-instruct-v0.3)

Pattern Matching Rules:

  1. Exact match in hardcoded models (backward compatibility)
  2. Prefix pattern match (gpt-*, claude-*, etc.)
  3. Ollama catch-all (models containing : or not matching cloud patterns)

Security Notes ​

Production Key Storage ​

  • NEVER use manual key source in production. It is blocked by default (returns 400 error).
  • Always use:
    • env_var for simple deployments (Docker, K8s env vars)
    • vault for HashiCorp Vault
    • k8s_secret for Kubernetes native secrets
    • aws_secrets_manager for AWS
    • azure_key_vault for Azure

Key Masking ​

  • API keys are never returned in full via GET /api/providers.
  • Manual keys show only last 4 characters: ****xyz
  • Other key sources show metadata (env var name, Vault path) but never the actual secret.

Redis Storage ​

  • Provider configs are stored in Redis at key platform:providers.
  • Only metadata and key references are stored, not raw keys (except for manual type in dev mode).

Example: Adding a Custom Provider ​

bash
curl -X POST http://localhost:4001/api/providers \
  -H "Content-Type: application/json" \
  -d '{
    "id": "custom-llama",
    "display_name": "Custom Llama Endpoint",
    "type": "openai_compatible",
    "base_url": "https://llama.internal.company.com",
    "auth_type": "bearer",
    "key_source": {
      "type": "k8s_secret",
      "name": "gatez-llm-secrets",
      "key": "custom-llama-key"
    },
    "model_patterns": ["llama-*", "custom-*"],
    "default_models": ["llama-3.3-70b"],
    "enabled": true,
    "status": "untested"
  }'

Then test it:

bash
curl -X POST http://localhost:4001/api/providers/custom-llama/test

Built-in Providers ​

The following providers are pre-configured and available by default:

  1. OpenAI (openai) — gpt-*, o1-*, o3-*
  2. Anthropic (anthropic) — claude-*
  3. Google Gemini (gemini) — gemini-*
  4. Groq (groq) — llama-*, mixtral-*, gemma-*
  5. Mistral AI (mistral) — mistral-*, codestral-*, pixtral-*
  6. DeepSeek (deepseek) — deepseek-*
  7. Together AI (together) — meta-llama/*, Qwen/*
  8. Fireworks AI (fireworks) — accounts/fireworks/*
  9. Cohere (cohere) — command-*, c4ai-*
  10. Ollama (ollama) — Local models (catch-all for models containing :)

All built-in providers use env_var key source by default. Set the corresponding environment variable (e.g., OPENAI_API_KEY) to enable.

Enterprise API + AI + Agent Gateway

Copyright 2026 Gatez