Response-Based Throttling

Overview

Response-Based Throttling is a plugin that helps reduce the cooldown period a service must endure after reaching the rate limit set by the API provider.

With Response-Based Throttling, Lunar determines whether to forward the API call to the actual API provider or return a 429 response generated by the proxy. This decision is based on the retry-after value, which indicates the duration the service needs to wait before making another request.

This behavior ensures seamless operation for services, enabling them to comply with the retry-header requirements without requiring any code changes in the application. It effectively minimizes delays caused by rate limiting, allowing for efficient and uninterrupted service delivery.

Configuration

/etc/lunar-proxy/policies.yaml

global:
  remedies:
    - name: Response-Based Throttling
      enabled: true
      config:
        response_based_throttling:
          retry_after_header: <header_name>
          retry_after_type: <one of relative_seconds or absolute_epoch>
          quota_group: <quota_group_identifier>
          relevant_statuses:
            - <status_code>

The following configuration options are available for this remediation plugin:

Parameter	Example Value	Possible Values	Description
retry_after_header	Retry-After	Any header name	The name of the header to use to determine the retry-after value.
retry_after_type	relative_seconds	One of relative_seconds or absolute_epoch	The type of retry-after value to use. Relative seconds will use the value of the header as the number of seconds to wait before retrying. Absolute epoch will use the value of the header as the epoch time to wait for until before retrying.
quota_group	quota_group_1	Any quota group identifier	The quota group to use for the response-based throttling. Remedies with the same quota group will share the same quota.
relevant_statuses	[429]	List of status codes	The list of status codes that will trigger the response-based throttling.

Usage Examples

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: api.com Response-Based Throttling
        enabled: true
        config:
          response_based_throttling:
            retry_after_header: Retry-After
            retry_after_type: relative_seconds
            quota_group: quota_group_1
            relevant_statuses:
              - 429

Use Cases

• Lower cool-off time: Reduce the cool-off times that the API provider enforces on your services.
• Reduce Error Rate: Reduce the error rate of your services by ensuring that they do not exceed the rate limits set by the API provider.
• Save Costs: Reduce the costs associated with API calls by ensuring that your services do not exceed the rate limits set by the API provider.
• Multiple services: Ensure that multiple services that use the same API provider are compliant with the API provider’s rate limits.