Skip to main content

Strategy-Based Queue

Overview

Strategy Based Queue introduces a delayed, priority-based queuing system. This plugin allows you to define a consumption rate by configuring window size and quota, as well as priorities for different groups, ensuring that higher-priority requests are processed first. By using this plugin, requests will be delayed until their turn to proceed has arrived, or until the defined TTL is met.

Configuration

/etc/lunar-proxy/policies.yaml
global:
remedies:
- name: Prioritized Bursts Rate Limit
enabled: true
config:
strategy_based_queue:
allowed_request_count: <allowed_request_count>
window_size_in_seconds: <window_size_in_seconds>
ttl_seconds: <ttl_seconds>
response_status_code: <response_status_code>
prioritization:
group_by:
header_name: <header_name>
groups:
<group_name1>:
priority: <priority1>
<group_name2>:
priority: <priority2>

The following configuration options are available for this plugin:

ParameterExample ValuePossible ValuesDescription
allowed_request_count1Any positive integerThe number of requests allowed within each window.
window_size_in_seconds1Any positive integerThe window size in seconds.
ttl_seconds12Any positive integerTime-to-live for a request in the queue in seconds.
response_status_code429Any HTTP status codeThe HTTP status code returned when the TTL is exceeded.
prioritization.group_by.header_nameX-GroupAny header nameThe header name to extract priority group from.
prioritization.groups.group_nameproductionAny group nameThe group name for prioritization.
prioritization.groups.*.priority1Any integer above 0 (including 0)The priority level assigned to the group. Lower numbers indicate higher priority.
note

The prioritization section is optional.

note

Enqueued requests within the same priority are served by the order upon which they arrived. Enqueued requests of more urgent priority will always be served before other enqueued requests.

It is possible to define the same priority for more than one group.

Examples

/etc/lunar-proxy/policies.yaml
endpoints:
- url: api.com/resource/{id}
method: GET
remedies:
- name: Prioritized Bursts Rate Limit
enabled: true
config:
strategy_based_queue:
allowed_request_count: 1
window_size_in_seconds: 1
ttl_seconds: 12
response_status_code: 429
prioritization:
group_by:
header_name: X-Group
groups:
production:
priority: 1
staging:
priority: 2

In this example, requests with the X-Group: production header will be given higher priority over those with X-Group: staging in the queue.

Use Cases

  • Avoiding Hitting Rate Limits: By delaying requests until there are available slots for them, Too Many Requests responses can be avoided.

  • Prioritizing Critical Requests: In scenarios where certain API requests are more critical than others (e.g., production vs. staging), this plugin allows prioritizing these critical requests to ensure business goals are met.

This plugin offers a refined level of control over API request management, catering to the needs of complex systems that require delayed, prioritized handling of API calls.

Click me for guidance 😀