Skip to main content
Version: 0.10.x

Strategy-Based Throttling

Overview​

Strategy Based Throttling enforces a policy of API throttling which involves setting limits and rules on the rate and volume of API requests made by the client application or system. The purpose is to regulate the frequency and quantity of API calls to ensure optimal usage, prevent overloading the API provider's resources, and maintain a reliable and efficient integration.

Configuration​

/etc/lunar-proxy/policies.yaml
global:
  remedies:
    - name: Strategy-Based Throttling
      enabled: true
      config:
        strategy_based_throttling:
          allowed_request_count: <allowed_request_count>
          window_size_in_seconds: <window_size_in_seconds>
          response_status_code: <response_status_code>

          group_quota_allocation: # Optional
            group_by:
              header_name: <header_name>
            groups:
              - group_header_value: <group_header_value1>
                allocation_percentage: <allocation_percentage1>
              - group_header_value: <group_header_value2>
                allocation_percentage: <allocation_percentage2>
            default: <allow|block|use_default_allocation> # Optional
            default_allocation_percentage: <default_allocation_percentage> # Optional

The following configuration options are available for this remediation plugin:

ParameterExample ValuePossible ValuesDescription
allowed_request_count100Any positive integerThe number of requests allowed within the window size.
window_size_in_seconds60Any positive integerThe window size in seconds.
response_status_code429Any HTTP status codeThe HTTP status code to return when the limit is exceeded.
group_by.header_nameX-GroupAny header nameThe header name to group by.
groups.group_header_valuegroup1Any header valueThe header value to group by.
groups.allocation_percentage20Any positive integerThe percentage of the quota to allocate to the group.
defaultallowallow, block, use_default_allocationThe default action to take when a request is not part of one of the specified groups.
default_allocation_percentage10Any positive integerOnly relevant if default is set to use_default_allocation. The percentage of the quota to allocate to the default group.

Examples​

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: Strategy Based Throttling
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 100
            window_size_in_seconds: 60
            response_status_code: 429

In the above example, the plugin will enforce a limit of 100 requests per minute for all requests. If the limit is exceeded, the plugin will return a 429 HTTP status code.

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: api.com Strategy-Based Throttling
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 100
            window_size_in_seconds: 60
            response_status_code: 429

          group_quota_allocation:
            group_by:
              header_name: X-Group
            groups:
              - group_header_value: "staging"
                allocation_percentage: 20
              - group_header_value: "production"
                allocation_percentage: 80

In this example, the plugin will enforce a limit of 80 requests per minute for requests containing the X-Group: production header and 20 for X-Group: staging.

note

The percentage configuration is unique for each header, which means you have the flexibility to set one header to 100% while assigning a lower value, such as 20%, to another header.

Use Cases​

  • Avoiding “Noisy Neighbor” collisions: In situations where a company consumes APIs providers on behalf of it’s customers, it becomes essential to establish an active throttling policy. This policy ensures that consumption remains below a certain threshold, preventing any adverse impact on both the customer's performance with it’s API provider.

  • Implementing an active client-side throttling policy: By proactively defining a consumption restriction policy, the API consumer can safeguard against overloading the API provider's resources, which could otherwise lead to degraded performance for the consumer. This best practice not only prevents excessive API calls but also maintains optimal performance levels for both the API consumer and the API provider.

  • Quota Management: When multiple different services or environments are using the same API, it is often necessary to allocate a different quota to each service or environment. This can be achieved by grouping requests by a header value and allocating a different quota to each group. This allows, for example, to allocate a different quota to each environment (e.g. staging, production, etc.) or to each service (e.g. service1, service2, etc.). The header names and values are configurable, allowing for maximum flexibility.

CTRL + M