Custom Models and Model Capabilities

When using custom models or overriding model capabilities in kagent, you need to specify what features your model supports. This is done through the modelInfo configuration.

Model Capabilities

Provider Requirements

Configuring Custom Models

Often you may want to use a custom model with an existing provider, for example if you're using a self-hosted model exposed via the OpenAI API. Or if you're using a proxy to the OpenAI API.

The example below will add an API key, but given that this is a custom provider, the key is optional.

1. Create a Kubernetes Secret that stores the API key, replace <your_api_key> with an actual API key:

export OPENAI_API_KEY=<your_api_key>
kubectl create secret generic kagent-openai -n kagent --from-literal OPENAI_API_KEY=$OPENAI_API_KEY

2. Create a ModelConfig resource that references the secret and key name, and specify the model you want to use:

apiVersion: kagent.dev/v1alpha1
kind: ModelConfig
metadata:
  name: custom-openai-model-config
  namespace: kagent
spec:
  apiKeySecretRef: kagent-openai
  apiKeySecretKey: OPENAI_API_KEY
  model: <insert-model-name> # This should match the model name passed to the provider.
  provider: OpenAI
  openAI:
    baseUrl: https://your-custom-openai-provider.com
  # Required: Specify model capabilities
  modelInfo:
    vision: false
    functionCalling: true
    jsonOutput: true
    family: "unknown"
    structuredOutput: true
    multipleSystemMessages: false

If you don't need an API key you can elide the apiKeySecretRef and apiKeySecretKey fields.

3. Apply the above resource to the cluster.

Once the resource is applied, you can select the model from the Model dropdown in the UI when creating or updating agents.

See provider documentation for specific configuration:

• OpenAI
• Anthropic
• Ollama