Skip to main content
Version: 0.10.x

Smart Caching

Overview​

The Smart Caching Plugin is designed to optimize performance and reduce the load on the API provider by caching the responses and serving them when the same requests are made again within a specified period. This is particularly useful for data that does not change frequently, mostly for GET requests or other read-only operations. For example, the Google Maps API provides a list of places based on a search query. This list is unlikely to change frequently, and hence can be cached for a certain period of time.

Configuration​

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{resourceID}
    method: GET
    remedies:
      - name: Smart Caching Example
        enabled: true
        config:
          caching:
            request_payload_paths:
              - payload_type: <payload_type>
                path: <path>
            ttl_seconds: <ttl_seconds>
            max_record_size_bytes: <max_record_size_bytes>
            max_cache_size_megabytes: <max_cache_size_megabytes>
warning

The Smart Caching Plugin can't be defined globally. It can only be defined for individual endpoints. This is because the caching key is based on the request payload, which is different for each endpoint.

Configuration Options​

The following configuration options are available for this caching plugin:

ParameterExample ValuePossible ValuesDescription
request_payload_paths[{"payload_type": "path_params", "path": "userID"}]List of PayloadPath objectsPaths of the payloads to be used as keys for caching.
request_payload_paths.payload_type"path_params"Currently only "path_params" is supportedThe type of payload to be used as a key for caching.
request_payload_paths.path"userID"Any valid string path in the payloadThe specific path in the payload to be used as a key for caching.
ttl_seconds300Any positive floatTime To Live for the cache, in seconds.
max_record_size_bytes4096Any positive integerThe maximum size of a record that can be stored in the cache, in bytes.
max_cache_size_megabytes100Any positive floatThe maximum size of the cache, in megabytes.

Example​

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{resourceID}
    method: GET
    remedies:
      - name: Smart Caching Example
        enabled: true
        config:
          caching:
            request_payload_paths:
              - payload_type: path_params
                path: "resourceID"
            ttl_seconds: 300
            max_record_size_bytes: 4096
            max_cache_size_megabytes: 100

In the above example, responses from api.com/resource/{resourceID} are cached with a TTL of 300 seconds. Any request with the same resourceID within 300 seconds will be served from the cache. Responses larger than 4096 bytes will not be cached. The total size of the cache will not exceed 100 MB.

Use Cases​

  • Reduced Latency: By serving cached responses, this plugin significantly reduces the latency compared to fetching the data again from the source.
  • Reduced Load on API Provider: Caching helps in reducing the number of actual requests made to the API provider, thereby optimizing the use of resources.
  • Optimized Costs: Lowering the number of API calls can result in reduced costs, especially when dealing with third-party APIs having cost implications based on usage.
CTRL + M