> ## Documentation Index
> Fetch the complete documentation index at: https://docs.expectedparrot.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Token limits and rate limits

> Separate request size, throughput, and usage costs when configuring model runs.

A token is a unit a language model uses to process text, often a word or part of a word. A request includes the question, supplied material, instructions, and any earlier answers used as memory. Models also limit how much they can generate and how many requests an account can make over time.

These limits affect different research choices: a long transcript can exceed a single request's capacity, while many short interviews can exceed an account's processing rate. Use the table to distinguish them before changing your design or execution settings.

| Limit | What it constrains | What to check |
| - | - | - |
| Context window | Tokens in a single request and response, according to the model's rules | Prompt length, attached content, prior-answer memory, and reserved output budget. |
| Output limit | Tokens the model may generate for a response | Selected model's output parameter and any reasoning-token budget. |
| Requests per minute (RPM) | Request throughput | Account/provider quota and concurrent workload. |
| Tokens per minute (TPM) | Token throughput across requests | Account/provider quota and aggregate request sizes. |

TPM is a rate limit, not a price. Billing depends on the provider's usage accounting and [pricing](https://docs.expectedparrot.com/en/latest/costs.md).

## Inspect the configured model

After selecting a model through [Provider configuration](https://docs.expectedparrot.com/en/latest/guides/providers.md), inspect its parameters and EDSL's scheduling limits:

```python theme={null}
from edsl import Model

model = Model()
{"model": model.model, "parameters": model.parameters,
 "rpm": model.rpm, "tpm": model.tpm}
```

This may retrieve model metadata. Values depend on the model and account; no universal default RPM, TPM, or output limit applies.

You can set `model.rpm` and `model.tpm` to tell EDSL about your applicable limits. These settings do not raise the provider's quota. For a prepared job, bound local concurrency in Python:

```python theme={null}
results = job.run(disable_remote_inference=True, max_concurrency=10)
# Choose a limit appropriate to your provider quota and workload.
```

The CLI alternative is `ep run jobs.ep --local --max-concurrency 10 --output results.ep`. Concurrency controls how many interviews can run at once; it is separate from the model's RPM and TPM limits.

## Reduce the request deliberately

Inspect [rendered prompts](https://docs.expectedparrot.com/en/latest/prompts.md) before reducing their size. Remove irrelevant scenario fields from the question text and avoid unnecessary prior-answer memory. Preserve information required by the research design.

When a task can be answered passage by passage, use [scenario chunking](https://docs.expectedparrot.com/en/latest/scenarios.md#join-reshape-and-summarize) or [PDF pages](https://docs.expectedparrot.com/en/latest/scenarios.md#pdf-text) to create smaller inputs. Each then receives its own answer; that changes the unit of analysis from a whole document to a passage or page.

For question types that support it, `include_comment=False` omits the request for an explanatory comment. `use_code=True` requests coded choices rather than their full text. These settings change the response instructions, so check the rendered prompt and test the resulting format.

An explanatory comment is model-generated text, not a guaranteed account of the model's internal reasoning. Token log-probabilities, where a provider exposes them, describe token predictions under that request; they are not probabilities of a human respondent selecting an answer.

## Diagnose a failed request

For context or output-limit errors, check the selected model's limits and adapter parameter names. For rate-limit errors, check the account quota and reduce the throughput. Provider parameters and reasoning budgets differ across models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.