LangGraph framework integration

AI Gateway

Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

Observability

Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

AI Gateway

Overview

LangGraph is a framework for building stateful, multi-actor AI applications with LLMs. It extends LangChain with graph-based agent orchestration, cycles, and controllability. By connecting LangGraph to Toqan Router’s AI Gateway, you get production-ready agentic workflows with access to 500+ models.

Key Benefits

Toqan Router’s AI Gateway enhances your LangGraph applications with:

Complete Observability

Track every agent step, tool use, and graph transition with detailed traces

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all your AI operations

Multi-Provider Access

Access 500+ LLMs and 30+ providers through a single, unified integration

Prerequisites

Before integrating LangGraph with Toqan Router, ensure you have:

A Toqan Router account and API Key
Python 3.8 or higher

To setup your API key, see API keys & Endpoints.

Installation

pip install langgraph langchain langchain-openai langchain-core

Configuration

Configure LangGraph to use Toqan Router’s AI Gateway by passing a ChatOpenAI instance with a custom base_url:

Python

from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

base_url: https://router.toqan.ai/eu/v3/router

Basic Agent Example

Here’s a complete example using create_agent with a tool:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [("user", "What's the weather in San Francisco?")]})
print(result["messages"][-1].content)

Agent with Multiple Tools

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

agent = create_agent(
    llm,
    tools=[get_weather, add, multiply],
    system_prompt="You are a helpful assistant with access to weather and math tools.",
)

result = agent.invoke({
    "messages": [("user", "What is 15 * 4? Also check the weather in Tokyo.")]
})
print(result["messages"][-1].content)

Streaming

Stream agent steps as they happen:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

for chunk in agent.stream(
    {"messages": [("user", "What's the weather in Paris?")]},
    stream_mode="updates",
):
    print(chunk)

Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

Python

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

# Use Claude
claude_agent = create_agent(
    ChatOpenAI(
        model="anthropic/claude-sonnet-5",
        api_key=os.getenv("TOQAN_API_KEY"),
        base_url="https://router.toqan.ai/eu/v3/router",
    ),
    tools=[get_weather],
)

# Use Gemini
gemini_agent = create_agent(
    ChatOpenAI(
        model="google-ai/gemini-2.5-flash",
        api_key=os.getenv("TOQAN_API_KEY"),
        base_url="https://router.toqan.ai/eu/v3/router",
    ),
    tools=[get_weather],
)

result = claude_agent.invoke({"messages": [("user", "What's the weather in London?")]})
print(result["messages"][-1].content)

Observability

Model calls through the AI Gateway appear in Traces. To capture the surrounding application workflow, use OpenTelemetry with a framework instrumentor and the regional OTLP endpoint. For Python, install openinference-instrumentation-langchain and follow the OpenInference LangChain setup. It supports LangChain and LangGraph. For Node.js, use OpenInference LangChain.js. Configure the exporter using the Observability quickstart:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $TOQAN_API_KEY"

Use the provider API directly when exporting framework spans for the same model calls, or choose which layer to trace so one call is not recorded twice.

Overview

Instrument your app

Explore

Monitor

Export

Review

Data privacy & security

Framework integrations

AI Gateway

Observability

AI Gateway

Overview

Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

Prerequisites

Installation

Configuration

Basic Agent Example

Agent with Multiple Tools

Streaming

Model Selection

Observability

AI Gateway

Observability

​AI Gateway

​Overview

​Key Benefits

Complete Observability

Built-in Reliability

Cost Optimization

Multi-Provider Access

​Prerequisites

​Installation

​Configuration

​Basic Agent Example

​Agent with Multiple Tools

​Streaming

​Model Selection

​Observability

AI Gateway

Overview

Key Benefits

Prerequisites

Installation

Configuration

Basic Agent Example

Agent with Multiple Tools

Streaming

Model Selection

Observability