Skip to main content
Version: 1.1.x

Async Queue Flow

The Async Queue Flow is primarily a reliability feature, designed to ensure robust communication using the Lunar.dev Gateway. The Lunar.dev Gateway handles requests asynchronously and delivers responses once they are ready, decoupling the client from the timing of the response.

Flow Diagram

note
  • The Async Queue Flow exposes and uses a single point :8010 for submission and retrieval leaving the standard 8000 available.
  • Upon submission, users will always get an immediate response with status 202 Accepted, a Location header at which users may try to retrieve the result for that request, and the x-lunar-enqueued: true meaning the job is being queued to start.
  • The retrieval process tries to fetch a response from lunar::steve::response::{seq-id}, and then materialize it as a real HTTP response. The Redis key is then marked for expiration with a TTL for how long to keep ready responses.
  • If the response is not ready yet (i.e. key not found), a 202 Accepted is returned with the x-lunar-enqueued: true header again.

Scenarios

  1. Long Queueing: Instead of waiting a long time for a request to execute in the queue, requests will close and the Client will be notified once a response is received.

Flow Components


Flow Example

This is an example of a fully configured Async API Call Flow.

/etc/lunar-proxy/flows/flow.yaml
name: AsyncPriorityQueueFlow

filter:
url: test.org/*

processors:
Queue:
processor: AsyncQueue
parameters:
- key: quota_id
value: test.org_PrimaryQuota

RetryProcessor:
processor: AsyncRetry
parameters:
- key: attempts
value: 3
- key: cooldown_between_attempts_seconds
value: 10
metrics:
enabled: true
labels:
- flow_name
- processor_key
- http_method
- host
- status_code
- consumer_tag
FilterProcessor:
processor: Filter
parameters:
- key: status_code_range
value: "400-500"
metrics:
enabled: true
labels:
- flow_name
- processor_key
- http_method
- host
- status_code
- consumer_tag

flow:
request:
- from:
stream:
name: globalStream
at: start
to:
processor:
name: Queue

- from:
processor:
name: Queue
to:
stream:
name: globalStream
at: end

response:
- from:
stream:
name: globalStream
at: start
to:
processor:
name: FilterProcessor
- from:
processor:
name: FilterProcessor
condition: miss
to:
stream:
name: globalStream
at: end

- from:
processor:
name: FilterProcessor
condition: hit
to:
processor:
name: RetryProcessor

- from:
processor:
name: RetryProcessor
condition: retry

to:
stream:
name: globalStream
at: end

- from:
processor:
name: RetryProcessor
condition: failed
to:
stream:
name: globalStream
at: end

/etc/lunar-proxy/quota/quota.yaml
quotas:
- id: test.org_PrimaryQuota
filter:
url: test.org/*
strategy:
fixed_window:
static:
max: 100
interval: 1
interval_unit: minute

Troubleshooting

  • If a /retrieve is called but response is not ready, the same response as upon the initial call will be returned.
  • :8010 will always return 202 Accepted with x-lunar-enqueued: true so users can be sure this is coming from Lunar.dev not their provider. The Location header should be used to retrieve responses.