> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Instructor structured output integration

> Combine the AI Gateway with Instructor for type-safe LLM responses. Generate Pydantic models and structured JSON outputs with validation and retries.

> [card] [AI Gateway](#ai-gateway)
>
> Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

> [card] [Observability](#observability)
>
> Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

## AI Gateway

### Overview

Instructor is a library for extracting structured outputs from LLMs using Pydantic models. By connecting Instructor to Toqan Router's AI Gateway, you get type-safe structured extraction with access to 500+ models through a single configuration change.

### Key Benefits

Toqan Router's AI Gateway enhances your Instructor applications with:

> [card] Complete Observability
>
> Track every extraction, validation, and retry with detailed traces

> [card] Built-in Reliability
>
> Automatic fallbacks, retries, and load balancing for production resilience

> [card] Cost Optimization
>
> Real-time cost tracking and spend management across all your AI operations

> [card] Multi-Provider Access
>
> Access 500+ LLMs and 30+ providers through a single, unified integration

### Prerequisites

Before integrating Instructor with Toqan Router, ensure you have:

* A Toqan Router account and API Key
* Python 3.8 or higher

> [!NOTE]
>
> To setup your API key, see API keys & Endpoints.

### Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install instructor openai
```

### Configuration

Configure Instructor to use Toqan Router's AI Gateway by patching an OpenAI client with a custom `base_url`:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from openai import OpenAI
import os

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))
```

> **base\_url**: `https://router.toqan.ai/eu/v3/router`

### Basic Extraction

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel
from openai import OpenAI
import os

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

class UserInfo(BaseModel):
    name: str
    age: int

user_info = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)   # John Doe
print(user_info.age)    # 30
```

### Classification

Use Instructor with `Literal` types for classification:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from typing import Literal
import os

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

class Sentiment(BaseModel):
    label: Literal["positive", "negative", "neutral"]
    confidence: float = Field(ge=0.0, le=1.0)

result = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    response_model=Sentiment,
    messages=[{"role": "user", "content": "Classify: 'This product is absolutely amazing!'"}],
)
print(f"Sentiment: {result.label}, Confidence: {result.confidence:.2f}")
```

### Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel
from openai import OpenAI
import os

class UserInfo(BaseModel):
    name: str
    age: int

# Use Claude
claude_client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

result = claude_client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Jane Smith is 25 years old."}],
)
print(f"{result.name}, {result.age}")

# Use Gemini
gemini_client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

result = gemini_client.chat.completions.create(
    model="google-ai/gemini-2.5-flash",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Bob Johnson is 40 years old."}],
)
print(f"{result.name}, {result.age}")
```

## Observability

### Getting Started

Instructor enables structured outputs from language models using Pydantic schemas. Tracing Instructor with Toqan Router provides comprehensive insights into data extraction patterns, validation success rates, retry mechanisms, and structured output performance to optimize your LLM-powered data processing pipelines.

### Prerequisites

Before you begin, ensure you have:

* A Toqan Router account and API Key
* Python 3.8+
* Instructor library installed in your project
* OpenAI API key (or other supported LLM provider credentials)

### Install Dependencies

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Core Instructor and OpenTelemetry packages
pip install instructor openai opentelemetry-sdk opentelemetry-exporter-otlp

# OpenInference instrumentation for Instructor
pip install openinference-instrumentation-instructor
```

### Configure Toqan Router

Set up your environment variables to connect to Toqan Router's OpenTelemetry collector:

**Unix/Linux/macOS:**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <TOQAN_API_KEY>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=instructor-app,service.version=1.0.0"
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
```

**Windows (PowerShell):**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://router.toqan.ai/eu/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <TOQAN_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=instructor-app,service.version=1.0.0"
$env:OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>"
```

**Using .env file:**

```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
OTEL_EXPORTER_OTLP_ENDPOINT=https://router.toqan.ai/eu/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <TOQAN_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=instructor-app,service.version=1.0.0
OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
```

### Integration

Instructor uses OpenInference instrumentation for automatic OpenTelemetry tracing.

> Set up the instrumentation in your application:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
import os

# Configure tracer provider
tracer_provider = TracerProvider(
    resource=Resource({"service.name": "instructor-app"})
)

# Set up OTLP exporter
otlp_exporter = OTLPSpanExporter(
    endpoint=f"{os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT')}/v1/traces",
    headers={"Authorization": os.getenv('OTEL_EXPORTER_OTLP_HEADERS').split('=', 1)[1]}
)

tracer_provider.add_span_processor(BatchSpanProcessor(otlp_exporter))

# Instrument Instructor
from openinference.instrumentation.instructor import InstructorInstrumentor

InstructorInstrumentor().instrument(tracer_provider=tracer_provider)
```

> Use Instructor with automatic tracing:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel
from openai import OpenAI
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
import os

# Configure OpenTelemetry
tracer_provider = TracerProvider(
    resource=Resource({"service.name": "instructor-app"})
)

otlp_exporter = OTLPSpanExporter(
    endpoint=f"{os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT')}/v1/traces",
    headers={"Authorization": os.getenv('OTEL_EXPORTER_OTLP_HEADERS').split('=', 1)[1]}
)

tracer_provider.add_span_processor(BatchSpanProcessor(otlp_exporter))

# Instrument Instructor
from openinference.instrumentation.instructor import InstructorInstrumentor

InstructorInstrumentor().instrument(tracer_provider=tracer_provider)

# Define response schema
class UserInfo(BaseModel):
    name: str
    age: int
    occupation: str

# Create Instructor client
client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

# Extract structured data (automatically traced)
user_info = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    response_model=UserInfo,
    messages=[{
        "role": "user",
        "content": "John Smith is a 32-year-old software engineer."
    }]
)

print(f"Extracted: {user_info.name}, {user_info.age}, {user_info.occupation}")
```

> [!TIP]
>
> All Instructor structured output extractions will be automatically instrumented and exported to Toqan Router through the OTLP exporter. For more details, see [Traces](/docs/ai-studio/observability/traces.md).

### Advanced Examples

**Complex Nested Schemas**

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from typing import List
import os

# Setup done as shown in Integration section above

class Address(BaseModel):
    street: str
    city: str
    country: str
    postal_code: str

class OrderItem(BaseModel):
    product_name: str
    quantity: int
    price: float

class Invoice(BaseModel):
    invoice_number: str = Field(description="Unique invoice identifier")
    customer_name: str
    billing_address: Address
    items: List[OrderItem]
    subtotal: float
    tax: float
    total: float

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

# Extract complex nested data (automatically traced)
invoice_text = """
Invoice #INV-2024-001
Customer: Acme Corporation
Address: 123 Main St, New York, NY, USA, 10001

Items:
- Premium Widget x5 @ $50.00
- Standard Gadget x10 @ $25.00

Subtotal: $500.00
Tax: $40.00
Total: $540.00
"""

invoice = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    response_model=Invoice,
    messages=[{"role": "user", "content": f"Extract invoice data: {invoice_text}"}]
)

print(f"Invoice {invoice.invoice_number} for {invoice.customer_name}")
print(f"Total: ${invoice.total}")
```

**Batch Processing with Validation**

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel, Field, field_validator
from openai import OpenAI
from typing import List
import os

class ProductReview(BaseModel):
    reviewer_name: str
    rating: int = Field(ge=1, le=5, description="Rating from 1-5 stars")
    sentiment: str = Field(description="positive, negative, or neutral")
    key_points: List[str]

    @field_validator('sentiment')
    @classmethod
    def validate_sentiment(cls, v):
        if v.lower() not in ['positive', 'negative', 'neutral']:
            raise ValueError('Invalid sentiment')
        return v.lower()

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

reviews_text = [
    "Amazing product! Best purchase ever. 5 stars from John.",
    "Terrible quality. Broke after one day. Very disappointed. - Sarah",
    "It's okay, nothing special but works as expected. 3 stars - Mike"
]

extracted_reviews = []

# Process batch with validation (automatically traced)
for review in reviews_text:
    try:
        result = client.chat.completions.create(
            model="openai/gpt-5.6-sol",
            response_model=ProductReview,
            messages=[{"role": "user", "content": f"Extract review data: {review}"}]
        )
        extracted_reviews.append(result)
    except Exception as e:
        print(f"Failed to process review: {e}")

print(f"Successfully processed {len(extracted_reviews)} reviews")
```

**Retry with Custom Logic**

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import instructor
from pydantic import BaseModel, Field, field_validator
from openai import OpenAI
from datetime import datetime
import os

class EventInfo(BaseModel):
    title: str = Field(min_length=5, max_length=100)
    date: str = Field(description="Date in YYYY-MM-DD format")
    time: str = Field(description="Time in HH:MM format")
    attendees: int = Field(ge=1, le=10000)

    @field_validator('date')
    @classmethod
    def validate_date(cls, v):
        try:
            datetime.strptime(v, '%Y-%m-%d')
            return v
        except ValueError:
            raise ValueError('Date must be in YYYY-MM-DD format')

client = instructor.from_openai(OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.getenv("TOQAN_API_KEY"),
))

event_description = """
AI in Healthcare Summit 2024
Date: April 15, 2024
Time: 9:00 AM
Expected Attendees: 500
"""

# Extract with automatic retries on validation errors (automatically traced)
event = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    response_model=EventInfo,
    messages=[{"role": "user", "content": f"Extract event info: {event_description}"}],
    max_retries=3
)

print(f"Event: {event.title} on {event.date} at {event.time}")
```

> [!TIP]
>
> Instructor is also compatible with our [AI Gateway](#ai-gateway), to learn more, see [Instructor](#ai-gateway).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.