> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prem.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate limits

> Understand how Prem enforces limits and how to handle a 429.

Rate limits restrict the number of requests that a client can send to the API in a given period. This page explains the mechanism and the error handling. For the numbers of each tier, see [Tiers](https://docs.prem.io/tiers.md).

## The purpose of rate limits

We apply rate limits for three reasons. They protect the API against abuse, because a malicious actor can send too many requests and make the service unavailable. They make access fair for all users, because one organization cannot make the API slow for the other users. They hold the total load on our infrastructure at a level that gives a constant response time.

## The operation of rate limits

We measure rate limits in three ways:

* **RPS** (requests per second), for every endpoint
* **TPM** (tokens per minute), for the chat completion endpoints
* **Audio seconds per minute**, for the transcription and translation endpoints

You can get to a limit on any of the three. The first limit that you get to stops the request. As an example, take the Base tier: 5 inference requests per second, and 38,000 tokens per minute. Five small requests in one second fill your request bucket, although those requests use very few tokens.

These points are important too:

* **The request limit applies to your organization.** All users and API keys in an organization draw on one request bucket.
* **The token limit and the audio limit apply to each member.** Prem tracks these two buckets for the member that owns the API key, not for the organization as a whole.
* **Each endpoint has its own request allowance.** Requests to one endpoint do not consume the allowance of another.
* **Rate limits change with your tier.** See [Tiers](https://docs.prem.io/tiers.md#request-limits) for the capacity of each tier.

### The token bucket

Each limit is a bucket with a **capacity** and a **refill rate**:

* The capacity is the largest burst that you can send at one time.
* Each request takes 1 token from the request bucket. Each inference request takes its total token count from the token bucket.
* The bucket refills at a constant rate. This lets you send short bursts, but it stops continuous overuse.

## Identifier scope

The rate limiter uses the most specific identifier that is available:

* **Organization ID**: the primary scope. Your API key supplies it.
* **User ID**: the rate limiter uses this if there is no organization context
* **IP Address**: the rate limiter uses this only for unauthenticated requests

So all requests from the API keys of one organization draw on the same request bucket.

## When you get to a limit

An empty request bucket returns a `429 Too Many Requests` status with a `Retry-After` header in seconds:

```json theme={"system"}
{
  "status": 429,
  "data": null,
  "error": "Rate limit exceeded, try again in 1 seconds",
  "message": null,
  "env": "production",
  "log": null,
  "validator": null,
  "support_id": "req_support_019a3f7c8d2e7b41a0c6"
}
```

Record the `support_id`. Send it to support with a timestamp if you need us to trace a request.

> [!NOTE]
>
> The `Retry-After` value on this response is always `1`, because every bucket refills at one request per second or faster. Do not use the value as a fixed back-off budget. Use it as a minimum. Then apply your own exponential back-off if the next attempt is also unsuccessful.

### Token and audio limits return a 400, not a 429

An empty **token** bucket or **audio** bucket does not return a `429`. The API refuses the request with a `400` status and one of these error strings:

| Bucket | Error string |
| - | - |
| Tokens per minute | `TOKEN_LIMIT_EXCEEDED` |
| Audio seconds per minute | `AUDIO_DURATION_LIMIT_EXCEEDED` |

The `TOKEN_LIMIT_EXCEEDED` response holds your current usage in `data.usage`:

```json theme={"system"}
{
  "status": 400,
  "error": "TOKEN_LIMIT_EXCEEDED",
  "data": {
    "usage": {
      "tier": { "code": "TIER_1", "name": "Tier 1" },
      "usage": {
        "tokens": [
          {
            "period_sec": 60,
            "tokens_available": 0,
            "tokens_limit": 250000,
            "tokens_used": 250000,
            "tokens_refill": 250000
          }
        ]
      }
    }
  }
}
```

> [!WARNING]
>
> A client that watches only for `429` misses both of these errors. Branch on the `error` string too. Then wait for the start of the next minute before you send the request again.

An empty bucket stays closed until it refills to 20% of your limit. A retry before that point receives the same `400`.

## Other 429 responses

The rate limiter returns the shape in the example. Not every `429` that you receive looks like it.

| Condition | Error string | `Retry-After` |
| - | - | - |
| Your API key reached its concurrent request limit | `Concurrent request limit reached. Your limit allows 5 concurrent request(s). Wait for ongoing requests to complete, or contact us for higher limits.` | No |
| A traffic surge at the model that you called | `We are experiencing a traffic surge. Try again later.` | Yes |

A rate limit can also happen upstream of Prem. Those responses have a `429` status with a generic body and no `Retry-After` header:

```json theme={"system"}
{
  "error": {
    "message": "Internal server error",
    "type": "server_error"
  }
}
```

This body looks like a `500`, but the status code is still `429`. Automated clients must obey these rules:

* **Branch on the HTTP status code, not the response body.** Any `429` is a rate limit, whatever the `error.type` or the message text says.
* **Do not expect a `Retry-After` header.** Use your own exponential back-off with jitter when the header is not there.

If you see `429` responses without a `Retry-After` header, or with an unexpected body, [contact us](https://docs.prem.io/contact-us.md) with the `support_id` and a timestamp.

## Error mitigation

### Steps to decrease rate limit errors

Be careful when you give users programmatic access, bulk processing features, and automated posting. Enable these features only for trusted customers.

Set a usage limit for each user in a specified period. This protects against automated and high-volume misuse. Add a hard cap or a manual review process for users who go above the limit.

### Retry with exponential backoff

One simple way to prevent rate limit errors is to retry requests automatically with a random exponential backoff. With this method, the client sleeps for a short time when a rate limit error occurs. Then the client retries the unsuccessful request. If the request is again unsuccessful, the client increases the sleep time and retries again. This continues until the request is successful or until the client reaches a maximum number of retries. This approach has many benefits:

* Automatic retries let you recover from rate limit errors without crashes or lost data
* With exponential backoff, the first retries occur quickly, and later retries have longer delays
* Random jitter in the delay prevents all retries from occurring at the same time

> [!TIP]
>
> Unsuccessful requests count against your rate limit. Do not send the same request again and again without a delay.

#### Example:

This implementation uses the `Retry-After` header from the API response. It uses exponential backoff when the response has no such header:

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

async function retryWithExponentialBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 6,
  baseDelay: number = 1000
): Promise<T> {
  let retries = 0;

  while (true) {
    try {
      return await fn();
    } catch (error: any) {
      const isRateLimitError = error?.status === 429;

      if (!isRateLimitError || retries >= maxRetries) {
        throw error;
      }

      // Some 429 responses (e.g. from an upstream backend) omit Retry-After.
      // Fall back to exponential backoff instead of retrying immediately.
      const retryAfterHeader = error?.headers?.["retry-after"];
      const parsedRetryAfter = retryAfterHeader ? parseInt(retryAfterHeader, 10) : NaN;
      const delay = Number.isFinite(parsedRetryAfter)
        ? parsedRetryAfter * 1000
        : baseDelay * 2 ** retries;

      retries++;
      console.log(
        `Rate limited (support_id ${error?.support_id || "unknown"}). ` +
        `Retrying in ${Math.round(delay)}ms... (attempt ${retries}/${maxRetries})`
      );

      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });

  try {
    const response = await retryWithExponentialBackoff(
      () =>
        client.chat.completions.create({
          model: "deepseek-v4.1-flash-1017",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
        }),
      6,
      1000
    );

    console.log(response.choices[0].message.content);
  } catch (error) {
    console.error("Request failed after retries:", error);
  }
}

main().catch(console.error);
```

### Control how many requests run at one time

Prem limits the number of inference requests that one API key can process at the same time. The limit applies to every inference endpoint, and it is separate from the request and token buckets. If your application sends many requests at one time, put them in a queue. A queue processes the requests in sequence or in controlled batches.

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

class RequestQueue {
  private maxConcurrent: number;
  private running: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrent: number = 3) {
    this.maxConcurrent = maxConcurrent;
  }

  async execute<T>(requestFn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }

    this.running++;
    try {
      return await requestFn();
    } finally {
      this.running--;
      const resolve = this.queue.shift();
      if (resolve) resolve();
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  const queue = new RequestQueue(3); // max 3 concurrent requests

  const prompts = [
    "Explain photosynthesis",
    "What is machine learning?",
    "Describe the water cycle",
    "Explain gravity",
    "What is DNA?",
  ];

  try {
    const results = await Promise.all(
      prompts.map((prompt) =>
        queue.execute(() =>
          client.chat.completions.create({
            model: "deepseek-v4.1-flash-1017",
            messages: [
              { role: "system", content: "You are a helpful assistant." },
              { role: "user", content: prompt },
            ],
          })
        )
      )
    );

    results.forEach((response, index) => {
      console.log(`Response ${index + 1}:`, response.choices[0].message.content);
    });
  } catch (error) {
    console.error("One or more requests failed:", error);
  }
}

main().catch(console.error);
```

## Tips for developers

* **Group related operations** to decrease the number of requests.
* **Cache frequently accessed data** so that you do not request the same data again and again.
* **Branch on the status code**, not on the shape of the error body.
* **Use the `Retry-After` header** when the response has one. Apply your own back-off when the response has none.
* **Watch for `TOKEN_LIMIT_EXCEEDED` and `AUDIO_DURATION_LIMIT_EXCEEDED`** on `400` responses, not only for `429`.
* **Log the `support_id`** for each unsuccessful request.

## Higher rate limits

Your limits increase when you move to a higher tier. See [Tiers](https://docs.prem.io/tiers.md#how-to-move-to-a-higher-tier).

If you need higher limits, [contact us](https://docs.prem.io/contact-us.md) for an enterprise plan with custom rate limits.

## Related

> [card] [Tiers](https://docs.prem.io/tiers.md)
>
> The request, token, and audio limits of each tier.

> [card] [Errors](https://docs.prem.io/errors.md)
>
> Every status code that the API returns, and what to do about it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.