> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Vercel AI SDK integration

> Use the AI Gateway with Vercel AI SDK for streaming LLM responses. Build real-time AI chat interfaces with React Server Components and edge runtime.

> [card] [AI Gateway](#ai-gateway)
>
> Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

> [card] [Observability](#observability)
>
> Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

## AI Gateway

### Overview

The Vercel AI SDK provides TypeScript and Python toolkits for building AI-powered applications with streaming, tools, and multi-model support. The TypeScript SDK can also connect to Toqan Router's AI Gateway through `@ai-sdk/openai-compatible` for access to 500+ models with a single provider setup.

### Key Benefits

Toqan Router's AI Gateway enhances your Vercel AI applications with:

> [card] Complete Observability
>
> Track every generation, stream, and structured output with detailed traces

> [card] Built-in Reliability
>
> Automatic fallbacks, retries, and load balancing for production resilience

> [card] Cost Optimization
>
> Real-time cost tracking and spend management across all your AI operations

> [card] Multi-Provider Access
>
> Access 500+ LLMs and 30+ providers through a single, unified integration

### Prerequisites

Before integrating Vercel AI with Toqan Router, ensure you have:

* A Toqan Router account and API Key
* Node.js 18 or higher

> [!NOTE]
>
> To setup your API key, see API keys & Endpoints.

### Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm install @ai-sdk/openai-compatible ai
```

### Configuration

Configure an OpenAI-compatible provider with your API key:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const toqan = createOpenAICompatible({
  name: "toqan",
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});
```

> **base\_url**: `https://router.toqan.ai/eu/v3/router`

### Text Generation

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const toqan = createOpenAICompatible({
  name: "toqan",
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const { text } = await generateText({
  model: toqan("openai/gpt-5.6-sol"),
  prompt: "Write a haiku about programming",
});

console.log(text);
```

### Streaming Responses

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const toqan = createOpenAICompatible({
  name: "toqan",
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const { textStream } = await streamText({
  model: toqan("openai/gpt-5.6-sol"),
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Explain quantum computing in two sentences." },
  ],
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}
```

### Structured Output

Use a JSON system prompt and parse the response:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const toqan = createOpenAICompatible({
  name: "toqan",
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const { text } = await generateText({
  model: toqan("openai/gpt-5.6-sol"),
  messages: [
    {
      role: "system",
      content: "You are a data assistant. Always respond with valid JSON only, no markdown.",
    },
    {
      role: "user",
      content: "Generate information about France with fields: name, capital, population, languages.",
    },
  ],
});

const country = JSON.parse(text);
console.log(country);
```

### Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const toqan = createOpenAICompatible({
  name: "toqan",
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

// Use Claude
const claudeResult = await generateText({
  model: toqan("anthropic/claude-sonnet-5"),
  prompt: "What is the largest planet?",
});

// Use Gemini
const geminiResult = await generateText({
  model: toqan("google/gemini-2.5-flash"),
  prompt: "What is the largest planet?",
});

// Use Groq
const groqResult = await generateText({
  model: toqan("groq/qwen/qwen3.8-27b"),
  prompt: "What is the largest planet?",
});
```

### Reasoning Effort with the Responses API

When calling the **AI Gateway** Responses endpoint through `@ai-sdk/openai`, the SDK only sends `reasoning` if it recognizes the model id as an OpenAI reasoning model. A `provider/` prefix such as `openai/gpt-5.6-sol` fails that check, so `reasoningEffort` is dropped without an error and the model runs at its default effort.

Set `forceReasoning: true` (`@ai-sdk/openai` 3.x and later), or pass the bare model id. The **AI Gateway** routes a bare id to the model developer's own offering, so `gpt-5.6-sol` is routed as `openai/gpt-5.6-sol`. On `@ai-sdk/openai` 2.x (AI SDK v5), only the bare id works.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const oai = createOpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const { text } = await generateText({
  model: oai.responses("openai/gpt-5.6-sol"),
  prompt: "What is 2 + 2?",
  providerOptions: {
    openai: { reasoningEffort: "low", forceReasoning: true },
  },
});
```

## Observability

### Getting Started

Both Vercel AI SDK implementations expose OpenTelemetry support that captures agent runs, model calls, tool executions, token usage, and errors. Export those traces to **Toqan Router** by pointing a standard OpenTelemetry OTLP exporter at the **Toqan Router** collector: the Python SDK through its `experimental_telemetry` OpenTelemetry adapter, and the TypeScript SDK through the `@ai-sdk/otel` package.

> [!WARNING]
>
> AI SDK for TypeScript v7 moved OpenTelemetry collection out of the `ai` package and into the separate `@ai-sdk/otel` package. Spans are only emitted once a telemetry integration is registered with `registerTelemetry`. Setting `experimental_telemetry: { isEnabled: true }` on its own emits nothing on v7, so traces never reach **Toqan Router**. See [TypeScript AI SDK](#typescript-ai-sdk) for the v7 setup.

> [!WARNING]
>
> Vercel currently marks Python telemetry as experimental. Its API and emitted span attributes may change between releases.

### Prerequisites

Before you begin, ensure you have:

* A Toqan Router account and an API Key.
* Vercel AI SDK for TypeScript v7+, or v3.1 through v6 for the legacy setup. Vercel AI SDK for Python 0.4+.
* Node.js 18+ (or Bun 1.3.5+) for TypeScript, or Python 3.12+ for Python.
* API keys for your LLM providers (OpenAI, Anthropic, etc.).

### Install Dependencies

> [tab] Python
>
> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> pip install "ai[otel,anthropic]" opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
> ```
>
> The `otel` extra adds Vercel's OpenTelemetry adapter. Replace `anthropic` with the provider extra the application uses, such as `openai`.

> [tab] TypeScript
>
> ```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Core Vercel AI SDK with latest version
> npm install ai@latest
>
> # Telemetry collection (v7+; omit on v6 and earlier)
> npm install @ai-sdk/otel
>
> # OpenTelemetry packages
> npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http
> npm install @opentelemetry/instrumentation @opentelemetry/resources
> npm install @opentelemetry/semantic-conventions
>
> # Next.js OpenTelemetry wiring (skip for standalone Node.js)
> npm install @vercel/otel
>
> # Provider SDKs (choose what you need)
> npm install @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google
>
> # Optional: For React applications
> npm install @ai-sdk/react
> ```

### Configure the TypeScript telemetry environment

Set up your environment variables to connect to Toqan Router's OpenTelemetry collector:

**Unix/Linux/macOS:**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <TOQAN_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=vercel-ai-app,service.version=1.0.0"
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
```

**Windows (PowerShell):**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://router.toqan.ai/eu/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <TOQAN_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=vercel-ai-app,service.version=1.0.0"
$env:OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>"
```

**Using .env file:**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
OTEL_EXPORTER_OTLP_ENDPOINT=https://router.toqan.ai/eu/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <TOQAN_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=vercel-ai-app,service.version=1.0.0
OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
```

### Integrations

#### Python AI SDK

The Python SDK maps its telemetry onto OpenTelemetry through an `OtelAdapter`. Configure a standard OpenTelemetry `TracerProvider` that exports to the **Toqan Router** collector, then register the adapter once at startup:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import asyncio
import os

import ai
from ai.experimental_telemetry import register
from ai.experimental_telemetry.otel import OtelAdapter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = TracerProvider(resource=Resource.create({"service.name": "weather-agent"}))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="https://router.toqan.ai/eu/v2/otel/v1/traces",
            headers={"Authorization": f"Bearer {os.environ['TOQAN_API_KEY']}"},
        )
    )
)

register(OtelAdapter(tracer_provider=provider, capture_content=True))


@ai.tool
async def get_weather(city: str) -> str:
    """Get the current weather."""
    return f"72°F and sunny in {city}"


async def main() -> None:
    agent = ai.Agent(tools=[get_weather])
    model = ai.get_model("anthropic:claude-sonnet-5")

    async with agent.run(
        model,
        [ai.user_message("Weather in SF?")],
    ) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="")


try:
    asyncio.run(main())
finally:
    provider.force_flush()
```

`register()` installs the adapter globally, so every agent run, model call, and tool execution after it is traced. `capture_content=True` includes message content in spans. Call `provider.force_flush()` before a short-lived script exits so buffered spans are sent.

Model ids in `provider:model` form call the provider directly using its API key, such as `ANTHROPIC_API_KEY`. Ids in `provider/model` form route through the Vercel AI Gateway and require `AI_GATEWAY_API_KEY` instead.

#### TypeScript AI SDK

Telemetry setup differs by major version. On v7 and later, OpenTelemetry collection lives in the `@ai-sdk/otel` package and is registered once at startup. On v6 and earlier, it is enabled per call through `experimental_telemetry`.

> [!WARNING]
>
> On v7, `experimental_telemetry` alone emits no spans. `isEnabled` gates a registered integration rather than installing one, so without `registerTelemetry` there is nothing to gate and the call produces no telemetry at all.

> [!NOTE]
>
> The examples in this section call the `openai` provider from `@ai-sdk/openai` directly. This routes requests to OpenAI without going through the **AI Gateway**. To route through Toqan Router, replace `openai` with the provider from `@ai-sdk/openai-compatible` as shown in the [Text Generation](#text-generation) section above.

##### v7 and later

Register the OTLP exporter, then register the AI SDK telemetry integration once at startup. This is the Next.js setup, where the framework calls `register()` from its instrumentation hook. For standalone Node.js, wire the exporter with `NodeSDK` as shown in [Agent and tool capture](#agent-and-tool-capture).

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// instrumentation.ts
import { registerOTel, OTLPHttpJsonTraceExporter } from '@vercel/otel';
import { trace } from '@opentelemetry/api';
import { registerTelemetry } from 'ai';
import { OpenTelemetry } from '@ai-sdk/otel';

export function register() {
  registerOTel({
    serviceName: 'your-project-name',
    traceExporter: new OTLPHttpJsonTraceExporter({
      url: 'https://router.toqan.ai/eu/v2/otel/v1/traces',
      headers: {
        'Authorization': `Bearer ${process.env.TOQAN_API_KEY}`,
      },
    }),
  });

  registerTelemetry(new OpenTelemetry({ tracer: trace.getTracer('ai') }));
}
```

> [!WARNING]
>
> Pass `tracer: trace.getTracer('ai')`. `@ai-sdk/otel` otherwise names its tracer `gen_ai`, which is the generic OpenTelemetry namespace rather than an AI SDK marker. **Toqan Router** identifies the framework from that instrumentation scope, so without it the traces are attributed to plain OpenTelemetry and the agents and tools they create are not linked to their runs, tokens, and cost.

Registration is global and opt-out: every AI SDK call emits spans afterwards, with no per-call flag.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// index.ts
import './instrumentation.js';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-5.5'),
  prompt: 'Write a short story about a robot',
});
```

Per-call options still apply through `experimental_telemetry` (or its `telemetry` alias). Set `isEnabled: false` to suppress a single call, `recordInputs` or `recordOutputs` to `false` to keep prompts or responses out of the spans, and `functionId` to name the operation. `functionId` becomes `gen_ai.agent.name` on the `invoke_agent` span, which is what **Toqan Router** registers as the agent.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const result = await generateText({
  model: openai('gpt-5.5'),
  prompt: 'Explain quantum computing',
  experimental_telemetry: {
    functionId: 'quantum-explanation',
    recordInputs: false,
    recordOutputs: false,
  },
});
```

`OpenTelemetry` emits the OpenTelemetry GenAI semantic conventions (`gen_ai.*` attributes). `@ai-sdk/otel` also exports `LegacyOpenTelemetry`, which emits the v6 `ai.*` attributes instead. **Toqan Router** ingests both. For the full set of integration options, see the [AI SDK telemetry documentation](https://ai-sdk.dev/docs/ai-sdk-core/telemetry).

##### v6 and earlier

Enable telemetry per call through the `experimental_telemetry` option:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// instrumentation.js
import { registerOTel, OTLPHttpJsonTraceExporter } from '@vercel/otel';

export function register() {
  registerOTel({
    serviceName: 'your-project-name',
    traceExporter: new OTLPHttpJsonTraceExporter({
      url: 'https://router.toqan.ai/eu/v2/otel/v1/traces',
      headers: {
        'Authorization': `Bearer ${process.env.TOQAN_API_KEY}`,
      },
    }),
  });
}
```

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// index.js
import './instrumentation.js'
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

// Simple usage with telemetry enabled
const result = await generateText({
  model: openai("gpt-5.5"),
  prompt: "Write a short story about a robot",
  experimental_telemetry: {
    isEnabled: true,
  },
});

// Advanced configuration with custom metadata
const resultWithMetadata = await generateText({
  model: openai("gpt-5.5"),
  prompt: "Explain quantum computing",
  experimental_telemetry: {
    isEnabled: true,
    functionId: "quantum-explanation",
    metadata: {
      userId: "user-123",
      requestId: "req-456",
      environment: "production",
    },
  },
});

// Control what data is recorded
const userPrompt = "What is the meaning of life, the universe, and everything?";
const resultWithPrivacy = await generateText({
  model: openai("gpt-5.5"),
  prompt: userPrompt,
  experimental_telemetry: {
    isEnabled: true,
    recordInputs: false, // Don't record prompts
    recordOutputs: false, // Don't record responses
  },
});
```

Here the main factor to enable telemetry is to include the following payload when generating text. This can be used across the board.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
  experimental_telemetry: {
    isEnabled: true,
    recordInputs: false, // Don't record prompts
    recordOutputs: false, // Don't record responses
  },
```

### Agent and tool capture

Agents, tools, and models are derived from the spans and rendered in [Agent Graphs](/docs/ai-studio/observability/agent-graphs.md).

#### Installation

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm install ai @ai-sdk/otel @ai-sdk/openai \
               @opentelemetry/api \
               @opentelemetry/sdk-node \
               @opentelemetry/exporter-trace-otlp-http
```

```bash bun theme={"theme":{"light":"github-light","dark":"github-dark"}}
bun add ai @ai-sdk/otel @ai-sdk/openai \
           @opentelemetry/api \
           @opentelemetry/sdk-node \
           @opentelemetry/exporter-trace-otlp-http
```

#### Configuration

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { trace } from "@opentelemetry/api";
import { registerTelemetry } from "ai";
import { OpenTelemetry } from "@ai-sdk/otel";

const exporter = new OTLPTraceExporter({
  url: "https://router.toqan.ai/eu/v2/otel/v1/traces",
  headers: {
    Authorization: `Bearer ${process.env.TOQAN_API_KEY}`,
  },
});

const sdk = new NodeSDK({
  spanProcessor: new BatchSpanProcessor(exporter),
  serviceName: "my-ai-app",
});

sdk.start();
registerTelemetry(new OpenTelemetry({ tracer: trace.getTracer("ai") }));
```

> [!NOTE]
>
> `registerTelemetry` is required on AI SDK v7 and later. On v6 and earlier, omit it and pass `experimental_telemetry: { isEnabled: true }` on each call instead.

#### Agent Detection

> [!NOTE]
>
> The example below calls the `openai` provider from `@ai-sdk/openai` directly, bypassing the **AI Gateway**. Replace `openai` with the provider from `@ai-sdk/openai-compatible` to route through Toqan Router.

The Vercel AI SDK does not have a built-in way to mark a span as an agent. To capture agents, use manual OpenTelemetry instrumentation with `tracer.startActiveSpan()`. The span name (e.g., `"translator-agent"`) becomes the agent name. This approach captures tools and models alongside the agent span.

> [!NOTE]
>
> The example below targets AI SDK v7. On v6 and earlier, drop `registerTelemetry` and add `isEnabled: true` to the `experimental_telemetry` block instead.

Captures: `agent/translator-agent`, `tool/translate`, `model/gpt-5.6-sol`

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { openai } from "@ai-sdk/openai";
import { context, trace } from "@opentelemetry/api";
import { generateText, jsonSchema, registerTelemetry, stepCountIs, tool } from "ai";
import { OpenTelemetry } from "@ai-sdk/otel";

const exporter = new OTLPTraceExporter({
  url: "https://router.toqan.ai/eu/v2/otel/v1/traces",
  headers: { Authorization: `Bearer ${process.env.TOQAN_API_KEY}` },
});

const sdk = new NodeSDK({
  spanProcessor: new BatchSpanProcessor(exporter),
  serviceName: "my-ai-app",
});

sdk.start();
registerTelemetry(new OpenTelemetry({ tracer: trace.getTracer("ai") }));

const translateTool = tool({
  description: "Translate text to a target language",
  inputSchema: jsonSchema<{ text: string; target_language: string }>({
    type: "object",
    properties: {
      text: { type: "string", description: "Text to translate" },
      target_language: { type: "string", description: "Target language" },
    },
    required: ["text", "target_language"],
  }),
  execute: async ({ text, target_language }) => {
    const translations: Record<string, Record<string, string>> = {
      spanish: { "Hello, how are you?": "¡Hola, ¿cómo estás?" },
      french: { "Hello, how are you?": "Bonjour, comment allez-vous?" },
      japanese: { "Hello, how are you?": "こんにちは、お元気ですか？" },
    };
    return (
      translations[target_language.toLowerCase()]?.[text] ||
      `[${target_language}] ${text}`
    );
  },
});

const tracer = trace.getTracer("ai-sdk-app");

await tracer.startActiveSpan("translator-agent", async (parentSpan) => {
  parentSpan.setAttribute("input", 'Translate "Hello, how are you?" to Spanish');

  await context.with(
    trace.setSpan(context.active(), parentSpan),
    async () => {
      try {
        const { text } = await generateText({
          model: openai("gpt-5.6-sol"),
          system: "You are a translator agent that helps users translate text to different languages.",
          prompt: 'Translate "Hello, how are you?" to Spanish',
          tools: { translate: translateTool },
          stopWhen: stepCountIs(3),
          experimental_telemetry: { functionId: "translator_agent" },
        });

        console.log(text);
        parentSpan.setAttribute("output", text);
      } finally {
        parentSpan.end();
      }
    },
  );
});

await sdk.shutdown();
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.