Providers

The provider system is what makes the SDK truly model-agnostic. A provider translates the SDK's unified interface into the specific API calls for each model. Your agent code never changes — only the provider does.

Built-in Providers

Claude (Anthropic)

import { claude } from 'assistme-agent-sdk-provider-claude'

const agent = new Agent({
  name: 'assistant',
  model: claude('claude-sonnet-4-6'),
  // ...
})

// With options
const agent2 = new Agent({
  name: 'assistant',
  model: claude('claude-opus-4-6', {
    apiKey: process.env.ANTHROPIC_API_KEY, // defaults to ANTHROPIC_API_KEY env var
    baseUrl: 'https://api.anthropic.com',   // custom endpoint
    maxRetries: 3,
  }),
})

Available models: claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5

OpenAI

import { openai } from 'assistme-agent-sdk-provider-openai'

const agent = new Agent({
  name: 'assistant',
  model: openai('gpt-4o'),
})

Available models: gpt-4o, gpt-4o-mini, o3, o3-mini, o4-mini

Google Gemini

import { gemini } from 'assistme-agent-sdk-provider-gemini'

const agent = new Agent({
  name: 'assistant',
  model: gemini('gemini-2.5-pro'),
})

Available models: gemini-2.5-pro, gemini-2.5-flash

OpenAI-Compatible (Ollama, vLLM, Together, etc.)

For any API that follows the OpenAI chat completions format:

import { openaiCompatible } from 'assistme-agent-sdk-provider-openai-compatible'

// Ollama (local)
const agent = new Agent({
  name: 'local-assistant',
  model: openaiCompatible('llama3.3:70b', {
    baseUrl: 'http://localhost:11434/v1',
  }),
})

// Together AI
const agent2 = new Agent({
  name: 'together-assistant',
  model: openaiCompatible('meta-llama/Llama-3.3-70B-Instruct', {
    baseUrl: 'https://api.together.xyz/v1',
    apiKey: process.env.TOGETHER_API_KEY,
  }),
})

// vLLM
const agent3 = new Agent({
  name: 'vllm-assistant',
  model: openaiCompatible('my-fine-tuned-model', {
    baseUrl: 'http://vllm-server:8000/v1',
  }),
})

Switching Providers

The same agent logic works across any provider:

const config = {
  name: 'assistant',
  instructions: 'You are a helpful assistant.',
  tools: [webSearch, calculator],
}

// Same agent, different models
const claudeAgent = new Agent({ ...config, model: claude('claude-sonnet-4-6') })
const gptAgent = new Agent({ ...config, model: openai('gpt-4o') })
const geminiAgent = new Agent({ ...config, model: gemini('gemini-2.5-pro') })
const localAgent = new Agent({ ...config, model: openaiCompatible('llama3.3:70b', { baseUrl: 'http://localhost:11434/v1' }) })

// All four produce results in the same format
const result1 = await Runner.run(claudeAgent, { messages })
const result2 = await Runner.run(gptAgent, { messages })

Model Selection by Task

Use different models for different tasks based on capability and cost:

// Cheap, fast model for routing
const router = new Agent({
  name: 'router',
  model: claude('claude-haiku-4-5'), // $0.25/M input, fast
  instructions: 'Route requests to the right specialist.',
  handoffs: [researcher, writer],
})

// Powerful model for complex reasoning
const researcher = new Agent({
  name: 'researcher',
  model: claude('claude-opus-4-6'), // Most capable
  instructions: 'Research deeply and thoroughly.',
  tools: [webSearch],
})

// Balanced model for most tasks
const writer = new Agent({
  name: 'writer',
  model: claude('claude-sonnet-4-6'), // Good balance
  instructions: 'Write clear, well-structured content.',
})

Custom Providers

Build a provider for any LLM by implementing the ModelProvider interface:

import { ModelProvider, ModelRequest, ModelResponse } from 'assistme-agent-sdk'

class CustomProvider implements ModelProvider {
  name = 'custom'

  constructor(
    private modelId: string,
    private options: { baseUrl: string; apiKey: string },
  ) {}

  async generate(request: ModelRequest): Promise<ModelResponse> {
    const response = await fetch(`${this.options.baseUrl}/generate`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.options.apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: this.modelId,
        messages: this.convertMessages(request.messages),
        tools: this.convertTools(request.tools),
        temperature: request.temperature,
        max_tokens: request.maxTokens,
      }),
    })

    const data = await response.json()
    return this.convertResponse(data)
  }

  async *stream(request: ModelRequest): AsyncGenerator<ModelStreamEvent> {
    // Implement streaming...
  }

  // Convert between SDK format and provider format
  private convertMessages(messages: Message[]): any[] { /* ... */ }
  private convertTools(tools: ToolDefinition[]): any[] { /* ... */ }
  private convertResponse(data: any): ModelResponse { /* ... */ }
}

// Usage
function custom(modelId: string, options: CustomOptions): ModelProvider {
  return new CustomProvider(modelId, options)
}

const agent = new Agent({
  model: custom('my-model', { baseUrl: '...', apiKey: '...' }),
})

Provider Interface

The full interface that every provider must implement. See the complete ModelProvider interface in the API Reference.

Key types:

ModelProvider — The core interface with generate(), stream(), and optional supports() methods
ModelRequest — Input to the provider: messages, tools, output, temperature, maxTokens, etc.
ModelResponse — Provider output: content, toolCalls, usage: { inputTokens, outputTokens }, stopReason
ProviderFeature — Feature flags: 'tool_use', 'streaming', 'structured_output', 'vision', 'extended_thinking'

Provider Feature Detection

Not all models support all features. The SDK handles this gracefully:

const provider = claude('claude-sonnet-4-6')

provider.supports?.('tool_use')          // true
provider.supports?.('streaming')         // true
provider.supports?.('structured_output') // true
provider.supports?.('vision')            // true
provider.supports?.('extended_thinking') // true

// If a feature isn't supported, the SDK falls back or throws a clear error
const localModel = openaiCompatible('small-model', { baseUrl })
localModel.supports?.('tool_use') // false → SDK will error if agent has tools

Best Practices

Default to the best model you can afford — Start with the most capable model, then optimize down to cheaper models once you understand performance requirements.
Use environment variables for API keys — Never hardcode keys. Providers default to standard env vars (ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.).
Match model to task — Routing/classification → cheap model. Complex reasoning → capable model. Code generation → code-optimized model.
Test across providers — If you plan to support multiple providers, test your agents with each one. Tool-calling behavior varies between models.
Implement supports() in custom providers — This lets the SDK fail fast with clear errors when a model doesn't support a required feature.

Providers

On this page