Smart Caching

Overview

The Smart Caching Plugin is designed to optimize performance and reduce the load on the API provider by caching the responses and serving them when the same requests are made again within a specified period. This is particularly useful for data that does not change frequently, mostly for GET requests or other read-only operations. For example, the Google Maps API provides a list of places based on a search query. This list is unlikely to change frequently, and hence can be cached for a certain period of time.

Configuration

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/resource/{resourceID}
    method: GET
    remedies:
      - name: Smart Caching Example
        enabled: true
        config:
          caching:
            request_payload_paths:
              - payload_type: <payload_type>
                path: <path>
            ttl_seconds: <ttl_seconds>
            max_record_size_bytes: <max_record_size_bytes>
            max_cache_size_megabytes: <max_cache_size_megabytes>

warning

The Smart Caching Plugin can't be defined globally. It can only be defined for individual endpoints. This is because the caching key is based on the request payload, which is different for each endpoint.

Configuration Options

The following configuration options are available for this caching plugin:

Parameter	Example Value	Possible Values	Description
request_payload_paths	[{"payload_type": "path_params", "path": "userID"}]	List of PayloadPath objects	Paths of the payloads to be used as keys for caching.
request_payload_paths.payload_type	"path_params"	Currently only "path_params" is supported	The type of payload to be used as a key for caching.
request_payload_paths.path	"userID"	Any valid string path in the payload	The specific path in the payload to be used as a key for caching.
ttl_seconds	300	Any positive float	Time To Live for the cache, in seconds.
max_record_size_bytes	4096	Any positive integer	The maximum size of a record that can be stored in the cache, in bytes.
max_cache_size_megabytes	100	Any positive float	The maximum size of the cache, in megabytes.

Example

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/resource/{resourceID}
    method: GET
    remedies:
      - name: Smart Caching Example
        enabled: true
        config:
          caching:
            request_payload_paths:
              - payload_type: path_params
                path: "resourceID"
            ttl_seconds: 300
            max_record_size_bytes: 4096
            max_cache_size_megabytes: 100

In the above example, responses from api.com/resource/{resourceID} are cached with a TTL of 300 seconds. Any request with the same resourceID within 300 seconds will be served from the cache. Responses larger than 4096 bytes will not be cached. The total size of the cache will not exceed 100 MB.

Use Cases

• Reduced Latency: By serving cached responses, this plugin significantly reduces the latency compared to fetching the data again from the source.
• Reduced Load on API Provider: Caching helps in reducing the number of actual requests made to the API provider, thereby optimizing the use of resources.
• Optimized Costs: Lowering the number of API calls can result in reduced costs, especially when dealing with third-party APIs having cost implications based on usage.