> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Request timeouts

> Set a maximum LLM request duration with the timeout parameter on the AI Gateway to prevent hanging calls and trigger automatic fallback models.

**Use Cases**

* Preventing slow models from blocking user-facing requests indefinitely.
* Setting different limits for interactive (short) vs. batch (long) workloads.
* Triggering fallback logic when a provider exceeds an acceptable wait time.
* Enforcing response-time SLAs on latency-sensitive features.

***

## Quick Start

Set maximum request duration to prevent hanging requests.

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X POST https://router.toqan.ai/eu/v3/router/responses \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4-mini",
    "input": "Summarize AI trends for 2024",
    "timeout": {"call_timeout": 30000}
  }'
```

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const response = await client.responses.create({
  model: "openai/gpt-5.4-mini",
  input: "Summarize AI trends for 2024",
  timeout: {
    call_timeout: 30000,
  },
});

console.log(response.output_text);
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

response = client.responses.create(
    model="openai/gpt-5.4-mini",
    input="Summarize AI trends for 2024",
    extra_body={"timeout": {"call_timeout": 30000}},
)

print(response.output_text)
```

```bash cURL (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X POST https://router.toqan.ai/eu/v3/router/chat/completions \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4-mini",
    "messages": [{ "role": "user", "content": "Summarize AI trends for 2024" }],
    "timeout": {"call_timeout": 30000}
  }'
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.4-mini",
  messages: [{ role: "user", content: "Summarize AI trends for 2024" }],
  timeout: {
    call_timeout: 30000,
  },
});
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

response = client.chat.completions.create(
    model="openai/gpt-5.4-mini",
    messages=[{"role": "user", "content": "Summarize AI trends for 2024"}],
    extra_body={"timeout": {"call_timeout": 30000}},
)
```

## Configuration

| Parameter | Type | Required | Description |
| - | - | - | - |
| `call_timeout` | number | No | Maximum time for the model call, in milliseconds. Defaults to `720000` (12 minutes) when `timeout` is omitted. |

**Timeout applies to:**

* Model generation time for non-streaming calls. The whole call must finish within `call_timeout`.
* Time to first token for streaming calls. See [Timeout with streaming](#timeout-with-streaming).
* Every retry and fallback attempt. Each attempt gets its own `call_timeout`, except on the streaming Responses API, where the attempts share the request deadline.

`call_timeout` bounds the model call only. It does not bound Guardrails or Evaluators: each Guardrail reference on the request has its own `timeout`, defaulting to the Evaluator's timeout, or 60 seconds if it has none, and configurable up to 10 minutes. See [Guardrail timeout](/docs/ai-gateway/configuration/guardrails.md#timeout).

### Timeout error

When `call_timeout` fires, the gateway cancels the attempt and answers with HTTP `408`:

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "error": {
    "message": "Request timed out. The operation took longer than the configured timeout.",
    "type": "timeout_error",
    "param": null,
    "code": "timeout_error"
  }
}
```

`408` is in the default retry codes and the default fallback codes, so the caller receives this error only after retries and fallbacks are exhausted.

## Recommended Values

| Use Case | Timeout (ms) | Reason |
| - | - | - |
| **Chat applications** | `15000` (15s) | User expectation for responses |
| **Real-time features** | `5000` (5s) | Immediate feedback required |
| **Batch processing** | `60000` (60s) | Complex analysis tasks |
| **Streaming responses** | `30000` (30s) | Longer generation time |
| **Development/testing** | `10000` (10s) | Fast iteration cycles |

## Error Handling

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

try {
  const response = await client.responses.create({
    model: "openai/gpt-5.6-sol",
    input: "Explain quantum computing",
    timeout: { call_timeout: 15000 },
  });
  console.log(response.output_text);
} catch (error) {
  if (error instanceof OpenAI.APIConnectionTimeoutError) {
    console.log('Request timed out - try increasing timeout or using faster model');
  }
}
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

try {
  const response = await client.chat.completions.create({
    model: "openai/gpt-5.6-sol",
    messages: [{ role: "user", content: "Explain quantum computing" }],
    timeout: { call_timeout: 15000 }
  });
} catch (error) {
  if (error instanceof OpenAI.APIConnectionTimeoutError) {
    console.log('Request timed out - try increasing timeout or using faster model');
    // Implement fallback behavior
  }
}
```

## Best Practices

**Timeout selection:**

* Set based on user experience requirements.
* Consider model complexity and prompt length.
* Factor in network latency (add 2-5s buffer).
* Test with realistic prompts and data.

**Environment-specific timeouts:**

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const timeouts = {
  development: 10000, // Fast feedback during dev
  staging: 20000, // Realistic testing
  production: 30000, // Conservative for reliability
};
```

**Progressive timeouts:**

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Start with short timeout, increase for retries
const attempts = [
  { timeout: 10000, model: "fast-model" },
  { timeout: 20000, model: "standard-model" },
  { timeout: 30000, model: "comprehensive-model" },
];
```

## Fallback Integration

Timeouts work seamlessly with fallbacks:

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "timeout": { "call_timeout": 15000 },
  "fallbacks": [
    { "model": "openai/gpt-5.6-sol" },
    { "model": "openai/gpt-5.4-mini" }
  ]
}
```

**Total possible time, chat completions and non-streaming calls:** `timeout × (1 + retry_count) × (1 + fallback_count)`

* Primary plus 2 fallbacks with a 15s timeout and the default 2 retries: 15s × 3 attempts × 3 models, up to 135s plus retry backoff delays.
* The whole request is also capped by an internal request deadline: 12 minutes for requests that are not streaming or that run an Agent, and 30 minutes for other streaming model requests. With `timeout` omitted, the 12-minute default already equals the non-streaming deadline.

## Troubleshooting

**Frequent timeouts**

* Increase timeout value.
* Use faster models (gpt-5.4-mini vs gpt-5.6-sol).
* Reduce prompt complexity/length.
* Check provider status for slowdowns.

**User experience issues**

* Set timeout based on user expectations.
* Show loading states for longer operations.
* Implement progressive enhancement.
* Consider async processing for long tasks.

**Performance optimization**

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Monitor timeout patterns
const timeoutMetrics = {
  averageResponseTime: 0,
  timeoutRate: 0,
  responseTimesByModel: {},
  optimalTimeout: 0, // 95th percentile + buffer
};
```

## Advanced Patterns

### Dynamic timeout adjustment

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const getDynamicTimeout = (promptLength, modelComplexity) => {
  const baseTimeout = 10000;
  const promptFactor = Math.min(promptLength / 1000, 3); // Max 3x for long prompts
  const modelFactor = modelComplexity === "simple" ? 1 : 2;

  return baseTimeout * promptFactor * modelFactor;
};
```

### Timeout with streaming

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "stream": true,
  "timeout": {
    "call_timeout": 30000
  }
}
```

With `stream: true`, `call_timeout` bounds the wait for the first token, not the total duration of the stream. The timer stops when the first chunk arrives, so a stream that keeps producing tokens can run longer than `call_timeout`.

After the first token, the gateway cancels the stream only when the provider stays silent for longer than the stream idle timeout, which is 12 minutes.

If the timeout fires before the first token, `/v3/router/chat/completions` returns HTTP `408` with the `timeout_error` body from [Timeout error](#timeout-error). On `/v3/router/responses`, a stream that is already open ends with a `response.failed` event whose `error.code` is `timeout_error`.

On the streaming Responses API, attempts share the overall request deadline, so a retry's first-token budget is capped at the remaining request budget divided by the attempts left.

### Circuit breaker pattern

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
class CircuitBreaker {
  timeout: number;
  failureCount: number;
  failureThreshold: number;
  state: "CLOSED" | "OPEN" | "HALF_OPEN";
  openedAt: number | null;

  constructor(timeout: number, failureThreshold = 5) {
    this.timeout = timeout;
    this.failureCount = 0;
    this.failureThreshold = failureThreshold;
    this.state = "CLOSED";
    this.openedAt = null;
  }

  async call<T>(requestFn: () => Promise<T>): Promise<T> {
    if (this.state === "OPEN") {
      if (this.openedAt !== null && Date.now() - this.openedAt < this.timeout) {
        throw new Error("Circuit breaker is OPEN");
      }
      this.state = "HALF_OPEN";
    }

    try {
      const result = await requestFn();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  onSuccess() {
    this.failureCount = 0;
    this.state = "CLOSED";
    this.openedAt = null;
  }

  onFailure() {
    this.failureCount += 1;
    if (this.failureCount >= this.failureThreshold) {
      this.state = "OPEN";
      this.openedAt = Date.now();
    }
  }
}
```

## Limitations

* **Fixed timeout**: `call_timeout` does not adapt to prompt length or model.
* **Fallback multiplication**: Every fallback attempt gets the same `call_timeout`, so worst-case latency multiplies by the number of attempts.
* **Provider variations**: Different providers have different baseline response times.
* **Streaming**: `call_timeout` bounds only the first token; the stream idle timeout bounds the rest.

## Monitoring

Key metrics to track:

* **Timeout rate**: % of requests that timeout.
* **Average response time**: Baseline performance.
* **95th percentile**: For setting optimal timeouts.
* **Timeout impact**: User experience degradation.
* **Model performance**: Response times by model.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Example monitoring
const metrics = {
  totalRequests: 0,
  timeouts: 0,
  responseTimes: [] as number[],
  recommendedTimeout: 0,
};

const calculatePercentile = (arr: number[], p: number): number => {
  const sorted = [...arr].sort((a, b) => a - b);
  return sorted[Math.floor((p / 100) * sorted.length)] ?? 0;
};

const monitorTimeouts = (responseTime: number, wasTimeout: boolean) => {
  metrics.totalRequests++;
  if (wasTimeout) {
    metrics.timeouts++;
  } else {
    metrics.responseTimes.push(responseTime);
  }

  // Calculate optimal timeout (95th percentile + 5s buffer)
  const p95 = calculatePercentile(metrics.responseTimes, 95);
  metrics.recommendedTimeout = p95 + 5000;
};
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.