Strategy-Based Throttling

Overview

Strategy Based Throttling enforces a policy of API throttling which involves setting limits and rules on the rate and volume of API requests made by the client application or system. The purpose is to regulate the frequency and quantity of API calls to ensure optimal usage, prevent overloading the API provider's resources, and maintain a reliable and efficient integration.

Configuration

/etc/lunar-proxy/policies.yaml

global:
  remedies:
    - name: Strategy-Based Throttling
      enabled: true
      config:
        strategy_based_throttling:
          allowed_request_count: <allowed_request_count>
          window_size_in_seconds: <window_size_in_seconds>
          response_status_code: <response_status_code>

          group_quota_allocation: # Optional
            group_by:
              header_name: <header_name>
            groups:
              - group_header_value: <group_header_value1>
                allocation_percentage: <allocation_percentage1>
              - group_header_value: <group_header_value2>
                allocation_percentage: <allocation_percentage2>
            default: <allow|block|use_default_allocation> # Optional
            default_allocation_percentage: <default_allocation_percentage> # Optional

The following configuration options are available for this remediation plugin:

Parameter	Example Value	Possible Values	Description
allowed_request_count	100	Any positive integer	The number of requests allowed within the window size.
window_size_in_seconds	60	Any positive integer	The window size in seconds.
response_status_code	429	Any HTTP status code	The HTTP status code to return when the limit is exceeded.
group_by.header_name	X-Group	Any header name	The header name to group by.
groups.group_header_value	group1	Any header value	The header value to group by.
groups.allocation_percentage	20	Any positive integer	The percentage of the quota to allocate to the group.
default	allow	allow, block, use_default_allocation	The default action to take when a request is not part of one of the specified groups.
default_allocation_percentage	10	Any positive integer	Only relevant if default is set to use_default_allocation. The percentage of the quota to allocate to the default group.

Examples

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: Strategy Based Throttling
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 100
            window_size_in_seconds: 60
            response_status_code: 429

In the above example, the plugin will enforce a limit of 100 requests per minute for all requests. If the limit is exceeded, the plugin will return a 429 HTTP status code.

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/resource/{id}
    method: GET
    remedies:
      - name: api.com Strategy-Based Throttling
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 100
            window_size_in_seconds: 60
            response_status_code: 429

          group_quota_allocation:
            group_by:
              header_name: X-Group
            groups:
              - group_header_value: "staging"
                allocation_percentage: 20
              - group_header_value: "production"
                allocation_percentage: 80

In this example, the plugin will enforce a limit of 80 requests per minute for requests containing the X-Group: production header and 20 for X-Group: staging.

note

The percentage configuration is unique for each header, which means you have the flexibility to set one header to 100% while assigning a lower value, such as 20%, to another header.

Use Cases

• Avoiding “Noisy Neighbor” collisions: In situations where a company consumes APIs providers on behalf of it’s customers, it becomes essential to establish an active throttling policy. This policy ensures that consumption remains below a certain threshold, preventing any adverse impact on both the customer's performance with it’s API provider.

• Implementing an active client-side throttling policy: By proactively defining a consumption restriction policy, the API consumer can safeguard against overloading the API provider's resources, which could otherwise lead to degraded performance for the consumer. This best practice not only prevents excessive API calls but also maintains optimal performance levels for both the API consumer and the API provider.

• Quota Management: When multiple different services or environments are using the same API, it is often necessary to allocate a different quota to each service or environment. This can be achieved by grouping requests by a header value and allocating a different quota to each group. This allows, for example, to allocate a different quota to each environment (e.g. staging, production, etc.) or to each service (e.g. service1, service2, etc.). The header names and values are configurable, allowing for maximum flexibility.