OpenAI SDK drop-in integration

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Gateway

Overview

The OpenAI SDK provides powerful tools for building AI applications with GPT models. By connecting the SDK to Toqan Router’s AI Gateway, you transform your OpenAI integration into a production-ready system with enterprise-grade capabilities, complete observability, and access to 500+ models across 30+ providers.

Key Benefits

Toqan Router’s AI Gateway enhances your OpenAI SDK with:

Complete Observability

Track every API call, token usage, and model interaction with detailed traces and analytics

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 500+ LLMs and 30+ providers through a single, unified integration

Prerequisites

Before integrating OpenAI SDK with Toqan Router, ensure you have:

A Toqan Router account and API Key
Python 3.8+ or Node.js 18+ with TypeScript support
OpenAI SDK installed

To setup your API key, see API keys & Endpoints.

To use libraries with private models, see Onboarding Private Models.

For a step-by-step migration of an existing OpenAI SDK application, see Migrate to Toqan Router.

Installation

pip install openai

npm install openai

Configuration

While using the OpenAI SDK, set the Base URL to the AI Gateway to feed calls through our API without changing any other part of your code. Using the Toqan Router AI Gateway, you benefit from Platform Traces and Cost and Usage Monitoring, keeping full compatibility and a unified API with all models while using the OpenAI SDK.

base_url: https://router.toqan.ai/eu/v3/router

Text Generation

Basic text generation with the OpenAI SDK through Toqan Router:

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const response = await openai.responses.create({
  model: "openai/gpt-5.6-sol",
  instructions: "You are a helpful assistant.",
  input: "Hello!",
});

console.log(response.output_text);

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

response = client.responses.create(
    model="openai/gpt-5.6-sol",
    instructions="You are a helpful assistant.",
    input="Hello!",
)

print(response.output_text)

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const completion = await openai.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello!" },
  ],
});

console.log(completion.choices[0].message.content);

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

completion = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
)

print(completion.choices[0].message.content)

Streaming Responses

Stream responses for real-time output:

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const stream = await openai.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "Write a short story about robots.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

stream = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="Write a short story about robots.",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const stream = await openai.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [{ role: "user", content: "Write a short story about robots." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

stream = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[{"role": "user", "content": "Write a short story about robots."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const claudeResponse = await openai.responses.create({
  model: "anthropic/claude-sonnet-5",
  input: "Explain machine learning",
});

const geminiResponse = await openai.responses.create({
  model: "google/gemini-2.5-flash",
  input: "Explain machine learning",
});

const groqResponse = await openai.responses.create({
  model: "groq/qwen/qwen3.8-27b",
  input: "Explain machine learning",
});

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

claude_response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain machine learning",
)

gemini_response = client.responses.create(
    model="google/gemini-2.5-flash",
    input="Explain machine learning",
)

groq_response = client.responses.create(
    model="groq/qwen/qwen3.8-27b",
    input="Explain machine learning",
)

Observability

Getting Started

Integrate OpenAI SDK with Toqan Router’s observability to gain complete insights into model performance, token usage, API latency, and conversation flows using OpenTelemetry.

Prerequisites

Before you begin, ensure you have:

A Toqan Router account and API Key
Python 3.8+ or Node.js 18+
OpenAI SDK installed

Install Dependencies

# Core OpenTelemetry packages
pip install opentelemetry-sdk opentelemetry-instrumentation opentelemetry-exporter-otlp

# OpenAI SDK
pip install openai

# OpenAI Auto-Instrumentation
pip install opentelemetry-instrumentation-openai

# OpenTelemetry SDK (includes all core packages)
npm install @opentelemetry/sdk-node @opentelemetry/api @opentelemetry/exporter-trace-otlp-http

# OpenAI SDK
npm install openai

# OpenAI Auto-Instrumentation
npm install @opentelemetry/instrumentation-openai

Configure Toqan Router

Set up your environment variables to connect to Toqan Router’s OpenTelemetry collector: Unix/Linux/macOS:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $TOQAN_API_KEY"
export OTEL_RESOURCE_ATTRIBUTES="service.name=openai-app,service.version=1.0.0"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/json"

Windows (PowerShell):

$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://router.toqan.ai/eu/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <TOQAN_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=openai-app,service.version=1.0.0"
$env:OTEL_EXPORTER_OTLP_TRACES_PROTOCOL = "http/json"

Using .env file:

OTEL_EXPORTER_OTLP_ENDPOINT=https://router.toqan.ai/eu/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <TOQAN_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=openai-app,service.version=1.0.0
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json

Setup

Configure OpenTelemetry once at application startup:

from openai import OpenAI
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
import os

# Configure OpenTelemetry
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(tracer_provider)

# Instrument OpenAI
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# Create OpenAI client
client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY")
)

import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { OpenAIInstrumentation } from "@opentelemetry/instrumentation-openai";
import OpenAI from "openai";

// Initialize SDK
const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [new OpenAIInstrumentation()],
});

sdk.start();

// Graceful shutdown
process.on("SIGTERM", () => {
  sdk.shutdown()
    .then(() => console.log("Tracing terminated"))
    .catch((error) => console.log("Error terminating tracing", error))
    .finally(() => process.exit(0));
});

// Create OpenAI client
const client = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

Examples

The examples below use the Chat Completions endpoint. OpenTelemetry instrumentation works identically with the Responses API: replace client.chat.completions.create(...) with client.responses.create(...).

Basic Example

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

response = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is quantum computing in one sentence?"}
    ]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is quantum computing in one sentence?" },
  ],
});

console.log(response.choices[0].message.content);

Streaming Example

from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("streaming-completion") as span:
    stream = client.chat.completions.create(
        model="openai/gpt-5.6-sol",
        messages=[{"role": "user", "content": "Write a haiku about code."}],
        stream=True
    )

    full_response = ""
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            content = chunk.choices[0].delta.content
            full_response += content
            print(content, end="", flush=True)

    span.set_attribute("response.length", len(full_response))

import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

await tracer.startActiveSpan("streaming-completion", async (span) => {
  try {
    const stream = await client.chat.completions.create({
      model: "openai/gpt-5.6-sol",
      messages: [{ role: "user", content: "Write a haiku about code." }],
      stream: true,
    });

    let fullResponse = "";
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) {
        fullResponse += content;
        process.stdout.write(content);
      }
    }

    span.setAttribute("response.length", fullResponse.length);
  } finally {
    span.end();
  }
});

Custom Spans Example

from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)

def analyze_document(document: str):
    with tracer.start_as_current_span("document-analysis") as span:
        span.set_attribute("document.length", len(document))

        # Prepare prompt
        prompt = f"Analyze this text: {document}"
        with tracer.start_as_current_span("prepare-prompt") as prep_span:
            prep_span.set_attribute("prompt.length", len(prompt))

        # Model inference
        with tracer.start_as_current_span("model-inference") as inf_span:
            inf_span.set_attribute("model", "gpt-5.6-sol")

            response = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": prompt}]
            )

            inf_span.set_attribute("tokens.total", response.usage.total_tokens)

        # Process result
        with tracer.start_as_current_span("process-result") as proc_span:
            result = response.choices[0].message.content
            proc_span.set_attribute("result.length", len(result))

        return result

analysis = analyze_document("Machine learning is a subset of AI.")
print(analysis)

import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

async function analyzeDocument(document: string) {
  return await tracer.startActiveSpan("document-analysis", async (span) => {
    try {
      span.setAttribute("document.length", document.length);

      // Prepare prompt
      const prompt = `Analyze this text: ${document}`;
      await tracer.startActiveSpan("prepare-prompt", async (prepSpan) => {
        try {
          prepSpan.setAttribute("prompt.length", prompt.length);
        } finally {
          prepSpan.end();
        }
      });

      // Model inference
      let response: Awaited<ReturnType<typeof client.chat.completions.create>>;
      await tracer.startActiveSpan("model-inference", async (infSpan) => {
        try {
          infSpan.setAttribute("model", "gpt-5.6-sol");

          response = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: prompt }],
          });

          infSpan.setAttribute("tokens.total", response.usage?.total_tokens || 0);
        } finally {
          infSpan.end();
        }
      });

      // Process result
      await tracer.startActiveSpan("process-result", async (procSpan) => {
        try {
          const result = response!.choices[0].message.content || "";
          procSpan.setAttribute("result.length", result.length);
        } finally {
          procSpan.end();
        }
      });

      return response!.choices[0].message.content;
    } finally {
      span.end();
    }
  });
}

analyzeDocument("Machine learning is a subset of AI.")
  .then((analysis) => console.log(analysis));

Advanced Workflows Example

from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)

def content_generation_pipeline(topic: str):
    with tracer.start_as_current_span("content-pipeline") as pipeline_span:
        pipeline_span.set_attribute("pipeline.topic", topic)
        pipeline_span.set_attribute("pipeline.stages", 3)

        # Stage 1: Research
        with tracer.start_as_current_span("stage-research") as research_span:
            research_span.set_attribute("stage.name", "research")

            research = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"List 3 key facts about {topic}."}]
            )

            facts = research.choices[0].message.content
            research_span.set_attribute("facts.count", 3)
            research_span.set_attribute("tokens.used", research.usage.total_tokens)

        # Stage 2: Writing
        with tracer.start_as_current_span("stage-writing") as writing_span:
            writing_span.set_attribute("stage.name", "writing")

            writing = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"Write a brief introduction using these facts: {facts}"}]
            )

            content = writing.choices[0].message.content
            writing_span.set_attribute("content.length", len(content))

        # Stage 3: Review
        with tracer.start_as_current_span("stage-review") as review_span:
            review_span.set_attribute("stage.name", "review")

            review = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"Rate this content quality 1-10: {content}"}]
            )

            rating = review.choices[0].message.content
            review_span.set_attribute("quality.rating", rating)

        pipeline_span.set_attribute("pipeline.success", True)
        return {"content": content, "rating": rating}

result = content_generation_pipeline("neural networks")
print(f"Content: {result['content']}")
print(f"Rating: {result['rating']}")

import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

async function contentGenerationPipeline(topic: string) {
  return await tracer.startActiveSpan("content-pipeline", async (pipelineSpan) => {
    try {
      pipelineSpan.setAttribute("pipeline.topic", topic);
      pipelineSpan.setAttribute("pipeline.stages", 3);

      let facts = "";
      let content = "";
      let rating = "";

      // Stage 1: Research
      await tracer.startActiveSpan("stage-research", async (researchSpan) => {
        try {
          researchSpan.setAttribute("stage.name", "research");

          const research = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `List 3 key facts about ${topic}.` }],
          });

          facts = research.choices[0].message.content || "";
          researchSpan.setAttribute("facts.count", 3);
          researchSpan.setAttribute("tokens.used", research.usage?.total_tokens || 0);
        } finally {
          researchSpan.end();
        }
      });

      // Stage 2: Writing
      await tracer.startActiveSpan("stage-writing", async (writingSpan) => {
        try {
          writingSpan.setAttribute("stage.name", "writing");

          const writing = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `Write a brief introduction using these facts: ${facts}` }],
          });

          content = writing.choices[0].message.content || "";
          writingSpan.setAttribute("content.length", content.length);
        } finally {
          writingSpan.end();
        }
      });

      // Stage 3: Review
      await tracer.startActiveSpan("stage-review", async (reviewSpan) => {
        try {
          reviewSpan.setAttribute("stage.name", "review");

          const review = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `Rate this content quality 1-10: ${content}` }],
          });

          rating = review.choices[0].message.content || "";
          reviewSpan.setAttribute("quality.rating", rating);
        } finally {
          reviewSpan.end();
        }
      });

      pipelineSpan.setAttribute("pipeline.success", true);
      return { content, rating };
    } finally {
      pipelineSpan.end();
    }
  });
}

contentGenerationPipeline("neural networks")
  .then((result) => {
    console.log(`Content: ${result.content}`);
    console.log(`Rating: ${result.rating}`);
  });

View Traces

View your traces in the AI Studio in the Traces tab.

Visit your AI Studio to view real-time analytics and traces.

Overview

Instrument your app

Explore

Monitor

Export

Review

Data privacy & security

Framework integrations

AI Gateway

Observability

AI Gateway

Overview

Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

Prerequisites

Installation

Configuration

Text Generation

Streaming Responses

Model Selection

Observability

Getting Started

Prerequisites

Install Dependencies

Configure Toqan Router

Setup

Examples

Basic Example

Streaming Example

Custom Spans Example

Advanced Workflows Example

View Traces

AI Gateway

Observability

​AI Gateway

​Overview

​Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

​Prerequisites

​Installation

​Configuration

​Text Generation

​Streaming Responses

​Model Selection

​Observability

​Getting Started

​Prerequisites

​Install Dependencies

​Configure Toqan Router

​Setup

​Examples

​Basic Example

​Streaming Example

​Custom Spans Example

​Advanced Workflows Example

​View Traces

AI Gateway

Overview

Key Benefits

Prerequisites

Installation

Configuration

Text Generation

Streaming Responses

Model Selection

Observability

Getting Started

Prerequisites

Install Dependencies

Configure Toqan Router

Setup

Examples

Basic Example

Streaming Example

Custom Spans Example

Advanced Workflows Example

View Traces