Async Queue Flow

The Async Queue Flow is primarily a reliability feature, designed to ensure robust communication using the Lunar.dev Gateway. The Lunar.dev Gateway handles requests asynchronously and delivers responses once they are ready, decoupling the client from the timing of the response.

note

• The Async Queue Flow exposes and uses a single point :8010 for submission and retrieval leaving the standard 8000 available.
• Upon submission, users will always get an immediate response with status 202 Accepted, a Location header at which users may try to retrieve the result for that request, and the x-lunar-enqueued: true meaning the job is being queued to start.
• The retrieval process tries to fetch a response from lunar::steve::response::{seq-id}, and then materialize it as a real HTTP response. The Redis key is then marked for expiration with a TTL for how long to keep ready responses.
• If the response is not ready yet (i.e. key not found), a 202 Accepted is returned with the x-lunar-enqueued: true header again.

---

Scenarios

1. Long Queueing: Instead of waiting a long time for a request to execute in the queue, requests will close and the Client will be notified once a response is received.

---

Flow Components

• Fixed Window Quota
• Async Retry Processor
• Async Queue Processor

---

Flow Example

This is an example of a fully configured Async API Call Flow.

/etc/lunar-proxy/flows/flow.yaml

name: AsyncPriorityQueueFlow

filter:
  url: test.org/*

processors:
  Queue:
    processor: AsyncQueue
    parameters:
      - key: quota_id
        value: test.org_PrimaryQuota

  RetryProcessor:
    processor: AsyncRetry
    parameters:
      - key: attempts
        value: 3
      - key: cooldown_between_attempts_seconds
        value: 10
    metrics:
      enabled: true
      labels:
        - flow_name
        - processor_key
        - http_method
        - host
        - status_code
        - consumer_tag
  FilterProcessor:
    processor: Filter
    parameters:
      - key: status_code_range
        value: "400-500"
    metrics:
      enabled: true
      labels:
        - flow_name
        - processor_key
        - http_method
        - host
        - status_code
        - consumer_tag

flow:
  request:
    - from:
        stream:
          name: globalStream
          at: start
      to:
        processor:
          name: Queue

    - from:
        processor:
          name: Queue
      to:
        stream:
          name: globalStream
          at: end

  response:
    - from:
        stream:
          name: globalStream
          at: start
      to:
        processor:
          name: FilterProcessor
    - from: 
        processor:
          name: FilterProcessor
          condition: miss
      to:
        stream:
          name: globalStream
          at: end

    - from:
        processor:
          name: FilterProcessor
          condition: hit
      to:
        processor:
          name: RetryProcessor

    - from:
        processor:
          name: RetryProcessor
          condition: retry
        
      to:
        stream:
          name: globalStream
          at: end

    - from:
        processor:
          name: RetryProcessor
          condition: failed
      to:
        stream:
          name: globalStream
          at: end

/etc/lunar-proxy/quota/quota.yaml

quotas:
  - id: test.org_PrimaryQuota
    filter:
      url: test.org/*
    strategy:
      fixed_window:
        static:
          max: 100
          interval: 1
          interval_unit: minute

---

Troubleshooting

• If a /retrieve is called but response is not ready, the same response as upon the initial call will be returned.
• :8010 will always return 202 Accepted with x-lunar-enqueued: true so users can be sure this is coming from Lunar.dev not their provider. The Location header should be used to retrieve responses.