Consume APIs On Behalf of Your Clients

When operating as a company that consumes APIs on behalf of your clients (CoBo company), managing API consumption becomes a critical challenge, especially with unpredictable API rate limits. These limits are dictated by how frequently users trigger API engines, creating dynamic consumption rates across different clients. This variation requires sophisticated quota management to ensure efficient resource allocation and prevent exceeding limits.

The Challenges of API Consumption on Behalf of Clients

For companies that manage API consumption for their clients, handling API rate limits, quota management, and ensuring smooth service delivery are critical. You may have several customers consuming the same API, and each customer can have different usage patterns. Managing this consumption efficiently, ensuring none of your clients’ quotas are exceeded, and prioritizing API traffic appropriately can become a complex task.

How Lunar.dev Helps

Lunar.dev offers an efficient and flexible solution for managing these issues. By routing your traffic through the Lunar.dev API Consumption Gateway, you can dynamically adjust quotas, optimize API usage, and provide full visibility into consumption patterns. Here’s how to solve these challenges with Lunar.dev:

Lunar.dev Installation

Step 1: Sign Up for Lunar.dev

Start by creating a Lunar.dev account here. After signing up, you'll be directed to the Lunar.dev Control Plane where you can manage and monitor your API consumption. Follow the installation steps to set up the Lunar.dev API Consumption Gateway.

Step 2: Install the API Consumption Gateway

Install the Lunar.dev API Consumption Gateway through Docker or Kubernetes based on your environment. Detailed installation instructions can be found on the API Consumption Gateway Installation page.

Step 3: Route Your API Traffic

Route your API traffic by modifying HTTP requests to include specific headers to capture and route outgoing traffic automatically. More details can be found on the Route Your API Traffic page.

Manage & Control

Step 1: Configure Relevant API Quotas

First, you need to define quotas for managing API consumption at both the company and individual user levels. Create a quota.yaml file to set limits based on the traffic to api.com/*.

1. Set a quota for the company:
   • Define an overall quota for a company using a custom header (x-lunar-api-company).
   • Limit API requests to api.com/search/* to 12 requests per minute.
2. Set an internal limit for individual users:
   • Group API usage by individual users via the x-lunar-api-user header.
   • Set a limit of 1000 requests per minute for all API calls to api.com/*.

/etc/lunar-proxy/quotas/quota.yaml

quotas:
  - id: LimiterMinuteCompany
    filter:
      url: api.com/search/*
    strategy:
      fixed_window:
        static:
          max: 12
          interval: 1
          interval_unit: minute
          group_by_header: x-lunar-api-company

internal_limits:
  - id: LimiterMinuteUser
    parent_id: LimiterMinuteCompany
    filter:
      url: api.com/*
      headers:
        - key: x-lunar-api-user
          value: any
    strategy:
      fixed_window:
        static:
          max: 1000
          interval: 1
          interval_unit: minute

Step 2: Apply Rate Limiting Flow

Next, define a flow that applies rate limiting for requests made to api.com/*. If the rate limit is exceeded, the system responds with a 429 Too Many Requests status.

1. Apply rate limiting based on user quotas:
   • Use the LimiterMinuteUser quota defined earlier to ensure individual users do not exceed their limit.
2. Generate a response when the limit is exceeded:
   • If the rate limit is breached, generate a 429 Too Many Requests response.

/etc/lunar-proxy/flows/flow1.yaml

name: RateLimiter

filter:
  url: api.com/*
  headers:
    - key: x-lunar-api-user
      value: any

processors:
  LimiterBoxUser:
    processor: Limiter
    parameters:
      - key: quota_id
        value: LimiterMinuteUser

  GenerateResponseTooManyRequests:
    processor: GenerateResponse
    parameters:
      - key: status
        value: 429
      - key: body
        value: "Quota Exceeded. Please try again later."
      - key: Content-Type
        value: text/plain

flow:
  request:
    - from:
        stream:
          name: globalStream
          at: start
      to:
        processor:
          name: LimiterBoxUser

    - from:
        processor:
          name: LimiterBoxUser
          condition: above_limit
      to:
        processor:
          name: GenerateResponseTooManyRequests

    - from:
        processor:
          name: LimiterBoxUser
          condition: below_limit
      to:
        stream:
          name: globalStream
          at: end

  response:
    - from:
        processor:
          name: GenerateResponseTooManyRequests
      to:
        stream:
          name: globalStream
          at: end

Step 3: Setup a Queue

For high-traffic endpoints, such as api.com/search/*, implement a queuing mechanism to manage requests and prioritize traffic based on the company making the request.

1. Queue requests:
   • Manage requests to the /search/* endpoint using a queue.
   • Limit the queue size to 100 and set the TTL (time-to-live) for queued requests to 60 seconds.
2. Prioritize requests:
   • Assign priority to requests based on the x-lunar-api-company header, giving companies like Google higher priority over Microsoft.
3. Generate a response if the queue is full:
   • Return a 429 Too Many Requests response if the queue is full and cannot process more requests.

/etc/lunar-proxy/flows/flow2.yaml

name: SearchQueueManager

filter:
  url: api.com/search/*

processors:
  QueueLimiterSearch:
    processor: Queue
    parameters:
      - key: quota_id
        value: LimiterMinuteCompany
      - key: ttl_seconds
        value: 60
      - key: queue_size
        value: 100
      - key: priority_group_by_header
        value: x-lunar-api-company
      - key: priority_groups
        value:
          Google: 1
          Microsoft: 2

  GenerateResponseTooManyRequests:
    processor: GenerateResponse
    parameters:
      - key: status
        value: 429
      - key: body
        value: "Quota Exceeded. Please try again later."
      - key: Content-Type
        value: text/plain

flow:
  request:
    - from:
        stream:
          name: globalStream
          at: start
      to:
        processor:
          name: QueueLimiterSearch

    - from:
        processor:
          name: QueueLimiterSearch
          condition: blocked
      to:
        processor:
          name: GenerateResponseTooManyRequests

    - from:
        processor:
          name: QueueLimiterSearch
          condition: allowed
      to:
        stream:
          name: globalStream
          at: end

  response:
    - from:
        processor:
          name: GenerateResponseTooManyRequests
      to:
        stream:
          name: globalStream
          at: end

By following these steps, you can efficiently manage API consumption on behalf of your clients using Lunar.dev’s powerful quota management and flow configurations. These steps allow you to implement rate limiting and queuing mechanisms that ensure fair usage of API resources, prevent rate limit violations, and provide a seamless experience for your clients.