Priority Queue Flow

The Priority Queue Flow manages API requests based on their priority, ensuring high-priority traffic is handled first while maintaining smooth processing for lower-priority requests. Using the Queue processor, it controls the number of requests in the queue and assigns priorities via the x-lunar-consumer-tag header. Requests are delayed until their turn arrives or the time-to-live (TTL) expires. If the queue is full, or limits are exceeded, a 429 "Too Many Requests" response is generated.

This flow optimizes traffic load, prioritizes critical requests, and ensures efficient resource utilization.

---

Scenarios

1. Critical Request Prioritization: Ensure that high-priority requests, such as production traffic, are processed first in environments with mixed priorities.
2. Preventing Overload with Queuing: Delay lower-priority requests in a queue to manage traffic without immediately generating "Too Many Requests" responses, unless the queue is full or TTL expires.
3. Handling High-Traffic APIs: Manage APIs with mixed-priority requests, balancing both staging and production traffic efficiently.
4. Fair Resource Allocation: Assign priority levels to different requests to balance resource usage and prevent low-priority traffic from impacting high-priority operations.
5. Custom Error Handling: Configure responses for exceeding limits to ensure users are informed of delays or retries in a controlled manner.

---

Flow Components

• Queue Processor
• GenerateResponse Processor
• Fixed Window Quota

---

Flow Example

In this example API requests to api.website.com/* are queued with a limit of 10 requests per second. Requests are prioritized based on the x-lunar-consumer-tag header, where production traffic gets higher priority (1) over staging traffic (2). If the queue is full or a request is delayed beyond the defined TTL (12 seconds), the system returns a 429 HTTP status code with the message: "Too many requests. Please try again later."

/etc/lunar-proxy/flows/flow.yaml

name: PriorityQueueFlow

filter:
  url: api.website.com/*

processors:
  QueueLimiter:
    processor: Queue
    parameters:
      - key: quota_id
        value: MyQuota
      - key: ttl_seconds
        value: 12
      - key: queue_size
        value: 10
      - key: priority_group_by_header
        value: x-lunar-consumer-tag
      - key: priority_groups
        value:
          production: 1
          staging: 2
  GenerateResponseTooManyRequests:
    processor: GenerateResponse
    parameters:
      - key: status
        value: 429
      - key: body
        value: "Too many requests. Please try again later."
      - key: Content-Type
        value: text/plain
flow:
  request:
    - from:
        stream:
          name: globalStream
          at: start
      to:
        processor:
          name: QueueLimiter
    - from:
        processor:
          name: QueueLimiter
          condition: blocked
      to:
        processor:
          name: GenerateResponseTooManyRequests
    - from:
        processor:
          name: QueueLimiter
          condition: allowed
      to:
        stream:
          name: globalStream
          at: end
  response:
    - from:
        processor:
          name: GenerateResponseTooManyRequests
      to:
        stream:
          name: globalStream
          at: end

/etc/lunar-proxy/quotas/quota.yaml

quotas:
  - id: MyQuota
    filter:
      url: api.website.com/*
    strategy:
      fixed_window:
        static:
          max: 10
          interval: 1
          interval_unit: second

---