Bailian applies rate limiting on application calls at two dimensions: per application and per Alibaba Cloud account. Requests exceeding the thresholds are rejected with HTTP 429 and error code Throttling.AllocationQuota, and typically recover automatically within one minute.
We recommend applying for capacity expansion in advance before scaling your business: If you anticipate a significant increase in concurrent requests or total call volume, contact your Alibaba Cloud business manager early to assess requirements and submit a capacity expansion request. You may also apply for quota increases through the same channel after rate limiting has been triggered.
Large language model (LLM)-related nodes in workflow applications — such as LLM nodes, question classification nodes, parameter extraction nodes, multimodal generation nodes, and retrieval-augmented LLM nodes — directly invoke underlying LLMs. These invocations are governed by the RPM (requests per minute) and TPM (tokens per minute) quotas of the invoked models themselves. This model-level rate limiting operates independently from application-level rate limiting — either limitation being exceeded will cause the request to fail.
The HTTP status code is
Recovery typically occurs automatically within one minute. During this period, clients should pause retries to avoid repeatedly triggering rate limiting.
The error code remains
Rate Limiting Rules
- Per-application concurrency: Requests per second (QPS) counted by application ID (
app_id), with a default limit of 100 QPS. - Account-level total traffic: Requests per minute (QPM) counted across all applications under the same Alibaba Cloud primary account, with a default limit of 15,000 QPM.
- Shared quota for asynchronous tasks and synchronous calls: Workflow asynchronous task submission requests share the same per-application QPS quota as synchronous API calls — submission frequency is subject to the same QPS limit.
Model-Level Rate Limiting for Workflow Applications
Large language model (LLM)-related nodes in workflow applications — such as LLM nodes, question classification nodes, parameter extraction nodes, multimodal generation nodes, and retrieval-augmented LLM nodes — directly invoke underlying LLMs. These invocations are governed by the RPM (requests per minute) and TPM (tokens per minute) quotas of the invoked models themselves. This model-level rate limiting operates independently from application-level rate limiting — either limitation being exceeded will cause the request to fail.
FAQ
How do I know if my request was rate-limited?
The HTTP status code is 429, and the response error code is Throttling.AllocationQuota. SDKs throw corresponding rate-limiting exceptions upon encountering this condition.
How long does it take to recover after hitting rate limiting?
Recovery typically occurs automatically within one minute. During this period, clients should pause retries to avoid repeatedly triggering rate limiting.
How can I avoid rate limiting?
- Exponential backoff retry: Upon receiving
429, wait a short interval before retrying; double the wait time after each failure (e.g., 1s → 2s → 4s → 8s), while enforcing maximum retry attempts and a cap on wait duration. - Client-side traffic smoothing: Introduce request queues or token bucket rate limiters on the client side to distribute bursty traffic evenly over time windows and prevent concentrated spikes.
- Multi-application traffic distribution: Since per-application QPS quotas are independent, split your business logic across multiple applications by use case to distribute concurrency load. Note: Account-level QPM remains aggregated across all applications.
- Fallback to backup applications: Maintain a list of standby applications on the business side. When the primary application hits rate limiting, route traffic to a backup application to reduce failure rates.
- Asynchronous invocation and batch processing: For non-real-time requests, switch to workflow-based asynchronous tasks or batch processing to bypass the instantaneous concurrency pressure of synchronous calls.
- Proactive capacity expansion: Before anticipated business growth, contact your Alibaba Cloud business manager to request quota increases — preventing rate limiting during production ramp-up.
What should I do if my workflow reports model-level rate limiting?
The error code remains Throttling.AllocationQuota, but the limiting is enforced at the model layer (i.e., by the LLM invoked by the workflow node), not at the application level. Investigate and adjust usage according to the specific RPM/TPM quotas of the underlying model.