Skip to main content
Version: 1.1.x

Queue Processor

Overview​

The Queue processor manages traffic by placing requests in a queue when the system is overloaded. It helps maintain smooth traffic flow by queuing excess requests instead of rejecting them outright. Users can choose to enforce priority logic within the Queue, determining which requests will be sent to the provider first.

Queue

Input and Output​

This processor operates on the request stream, determining whether the request can proceed or needs to be queued:

Input Stream: Request – The processor intercepts incoming requests and evaluates whether they should be processed immediately or queued.

Output Streams: Request-

  • Allowed – If the Request passes the processing rules, it proceeds to the next step in the flow.

  • Blocked – If the Request cannot be processed due to queue constraints, it is blocked and does not proceed through the flow.

By managing the flow of requests in this way, the Queue processor helps to maintain system stability during peak load times, ensuring that only requests within acceptable limits are processed further.

Parameters​

Each parameter is defined as a key-value pair inside the parameters section.

quota_id​

Type: string
Required: True
Defines the ID of the quota or internal limit to be used for rate limiting. For more details about Lunar.dev Quotas check out the Quota Overview page.

Example:

- key: quota_id
  value: "MyQuota"

ttl_seconds​

Type: number
Required: True
Default: 10
How many seconds a request can stay in the queue.

Example:

- key: ttl_seconds
  value: 12

queue_size​

Type: number
Required: True
The maximum amount of requests the Gateway can hold in its queue.

Example:

- key: queue_size
  value: 10

redis_queue_size​

Type: number
Required: False
Default: -1
The maximum amount of requests the total Gateways can hold in their shared queue. (-1 for unlimited)

Example:

- key: redis_queue_size
  value: -1

priority_group_by_header​

Type: string
Required: False The header name to extract priority group from.

Example:

- key: priority_group_by_header
  value: x-lunar-consumer-tag

priority_groups​

Type: map_of_strings
Required: False
The order of which grouped requests will be released from the Queue. The lower the number, the higher priority that group has.

Example:

- key: priority_groups
  value:
    production: 1
    staging: 2
note

If the you want to have a priorty within your Queue processor, both priority_groups and priority_group_by_header must be defined.

Best Practices​

  • Ensure quota_id matches a valid quota resource.
    • The queue will dispatch as many requests possible to fill current quota window based on the quota strategy used.
  • Avoid using queue on daily/monthly quotas.
  • Avoid using TTLs longer than one hour (Contact us for information about our async queue)
  • The queue size should be a combination of the Quota and Time to Live (TTL). For example, If the quota allows for 10 requests/second and there is a TTL of 5 seconds, the Queue size will be limited to 50. Once it is full, the 51st request will automatically be blocked.

For more details on configuring flows, visit the Lunar.dev Flows Documentation.

Queue Processor Template​

Queue Processor
Queue:
  processor: Queue 
  parameters:
    - key: quota_id
      value: <QuotaID> # Specify the quota ID, e.g., MyQueue
    - key: ttl_seconds
      value: <TTLInSeconds> # Time to live for requests in the queue
    - key: queue_size
      value: <MaxQueueSize> # Maximum number of requests in the queue
    - key: priority_group_by_header
      value: <HeaderForPriorityGroup> # Optional: Header to determine request priority
    - key: priority_groups
      value: 
        <Group1>: <Priority1> # Optional: Priority groups with their respective priorities
        <Group2>: <Priority2>

Use Case​

When your API is handling more requests than it can process, queue up the lower-priority staging requests while giving priority to production:

  QueueLimiter:
   processor: Queue
   parameters:
     - key: quota_id
       value: MyQuota
     - key: ttl_seconds
       value: 12
     - key: queue_size
       value: 10
     - key: priority_group_by_header
       value: x-lunar-consumer-tag
     - key: priority_groups
       value:
         production: 1
         staging: 2