Skip to main content
Version: 0.10.x

Response-Based Throttling

Overview​

Response-Based Throttling is a plugin that helps reduce the cooldown period a service must endure after reaching the rate limit set by the API provider.

With Response-Based Throttling, Lunar determines whether to forward the API call to the actual API provider or return a 429 response generated by the proxy. This decision is based on the retry-after value, which indicates the duration the service needs to wait before making another request.

This behavior ensures seamless operation for services, enabling them to comply with the retry-header requirements without requiring any code changes in the application. It effectively minimizes delays caused by rate limiting, allowing for efficient and uninterrupted service delivery.

Configuration​

/etc/lunar-proxy/policies.yaml
global:
  remedies:
    - name: Response-Based Throttling
      enabled: true
      config:
        response_based_throttling:
          retry_after_header: <header_name>
          retry_after_type: <one of relative_seconds or absolute_epoch>
          quota_group: <quota_group_identifier>
          relevant_statuses:
            - <status_code>

The following configuration options are available for this remediation plugin:

ParameterExample ValuePossible ValuesDescription
retry_after_headerRetry-AfterAny header nameThe name of the header to use to determine the retry-after value.
retry_after_typerelative_secondsOne of relative_seconds or absolute_epochThe type of retry-after value to use. Relative seconds will use the value of the header as the number of seconds to wait before retrying. Absolute epoch will use the value of the header as the epoch time to wait for until before retrying.
quota_groupquota_group_1Any quota group identifierThe quota group to use for the response-based throttling. Remedies with the same quota group will share the same quota.
relevant_statuses[429]List of status codesThe list of status codes that will trigger the response-based throttling.

Usage Examples​

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: api.com Response-Based Throttling
        enabled: true
        config:
          response_based_throttling:
            retry_after_header: Retry-After
            retry_after_type: relative_seconds
            quota_group: quota_group_1
            relevant_statuses:
              - 429

Use Cases​

  • Lower cool-off time: Reduce the cool-off times that the API provider enforces on your services.
  • Reduce Error Rate: Reduce the error rate of your services by ensuring that they do not exceed the rate limits set by the API provider.
  • Save Costs: Reduce the costs associated with API calls by ensuring that your services do not exceed the rate limits set by the API provider.
  • Multiple services: Ensure that multiple services that use the same API provider are compliant with the API provider’s rate limits.
CTRL + M