Customize & Configure

Lunar offers a comprehensive set of configuration options through its policies.yaml file located in the /etc/lunar-proxy directory. Serving as the central configuration hub, this file acts as the backbone of Lunar's functionality.

The configuration abilities enable engineering teams to define global policies or end-point related policies, and is used to define remedy and diagnose plugins.

If no remedy or diagnose plugins are defined, Lunar Proxy will act as a regular proxy, forwarding requests to the upstream server and returning the response.

This file is loaded by Lunar Proxy on startup and can be edited and reloaded at runtime. For more information on how to update the configuration at runtime, see Update policies.yaml while Lunar Proxy is running.

tip

To effectively manage your Lunar configuration and policies, it's recommended to create a local file named policies.yaml with the described settings. Once you have your custom configuration file ready, you can run Lunar Proxy with this file by mounting it into the Docker container.

Create a local file named policies.yaml with your desired Lunar configuration settings (as instructed below). Use the -v flag in your Docker run command to mount your local policies.yaml file to the /etc/lunar-proxy directory inside the Lunar Proxy container. In the docker command below, please make sure to replace your_local_path with the actual path to your policies.yaml file on your host machine, and modify the TENANT_NAME as you desire.

Here's the modified Docker run command with the local file mount:

docker run -d --rm -p 8000:8000 -p 8081:8081 -p 8040:8040 -e TENANT_NAME="ORGANIZATION" -v "your_local_path":/etc/lunar-proxy --name lunar-proxy lunarapi/lunar-proxy:latest

By mounting your custom policies.yaml file into the container, you can easily update your Lunar Proxy configuration without the need to stop and restart the container. After making changes to your local policies.yaml file, use the apply_policies command to apply the new policies.

docker exec lunar-proxy apply_policies

Structure

The policies.yaml file is divided into four main sections, which are all optional: allowed_domains, blocked_domains, global, endpoints, exporters, and accounts. The sections are described in detail below.

Overview

The structure of policies.yaml file is as follows:

Section	Description
mTLS	A list of domains that require mutual TLS (mTLS), each with a specific path to its certificate file.
allowed_domains	A list of domains to allow through the proxy.
blocked_domains	A list of domains to block from the proxy.
global	A list of global remedy and diagnose plugins to apply to all endpoints.
endpoints	A list of endpoints to configure remedy and diagnose plugins for.
exporters	A list of exporters to configure.
accounts	A list of accounts to configure.

Template

/etc/lunar-proxy/policies.yaml

mTLS: # (optional)
  - domain: <The upstream provider's domain that requires mTLS.>
    cert: <The path to the mTLS certificate file, which must be accessible within the container.>

allowed_domains: # (optional)
  - <allowed_domain>

blocked_domains: # (optional)
  - <blocked_domain>

global: # (optional)
  remedies: # (optional) A list of remedy plugins to apply to all endpoints.
    - name: <user_defined_name>
      enabled: <true/false>
      config:
        <remedy_plugin_type>:
          # Remedy-specific configuration...

  diagnosis: # (optional) A list of diagnose plugins to apply to all endpoints.
    - name: <user_defined_name>
      enabled: <true/false>
      export: <exporter_type> # The name of the exporter to use.
      config:
        <diagnose_plugin_type>:
          # Diagnose-specific configuration...

endpoints: # (optional)
  - method: <http_method> # GET/POST/PUT/DELETE/...
    url: <url_pattern>
    remedies: # (optional) A list of remedy plugins to apply to the endpoint.
      - name: <user_defined_name>
        enabled: <true/false>
        config:
          <remedy_plugin_type>:
            # Remedy-specific configuration...
    diagnosis: # (optional) A list of diagnose plugins to apply to the endpoint.
      - name: <user_defined_name>
        enabled: <true/false>
        export: <exporter_type> # The name of the exporter to use.
        config:
          <diagnose_plugin_type>:
            # Diagnose-specific configuration...

exporters: # (optional)
  file:# (optional)
    # ...
  prometheus:# (optional)
    # ...
  s3:# (optional)
    # ...

accounts:# (optional)
  # ...

Path Parameters Configuration

Path parameters allow you to create flexible and reusable URL patterns by defining variable segments within the URL. This enables you to group similar endpoints and apply consistent policies across them.

Why Use Path Parameters?

Path parameters are particularly useful when you have multiple variations of an endpoint that follow a similar pattern, want to apply the same policies to endpoints with variable segments, and need to manage a large number of dynamic endpoints efficiently.

By using path parameters, the discover command of the proxy presents a more ordered and grouped view, making endpoint management easier. The UI control plane also reflects this organization, offering a clearer and more manageable interface.

Case Example:

Consider an API with multiple URLs, each containing different IDs:

api.com/users/123/profile
api.com/users/456/profile
api.com/users/789/profile

Without declaring a path parameter, each URL variation would be treated as a separate entry, leading to a difficult policy management. Declaring a path parameter like api.com/users/{userId}/profile consolidates these variations into a single, manageable endpoint, simplifying policy application and improving metrics views.

Defining Path Parameters in policies.yaml

To configure path parameters, you can use the following guidelines:

Basic Syntax:

Define the path parameter within curly braces {} in the URL pattern. Example: api.com/api/v1/users/{id}

Combining Wildcards and Path Parameters:

You can use wildcards (*) along with path parameters. Example: api.com/api/v1/users/{id}/*

tip

• Defining endpoints explicitly in your policies.yaml helps in grouping them appropriately. If high cardinality (50+ variations) is detected, the system will automatically suggest parameter grouping.

Example Configurations:

As part of a Strategy-Based Throttling Remedy Plugin:

/etc/lunar-proxy/policies.yaml

endpoints:
  - method: GET
    url: api.com/api/v1/users/{id}
    remedies:
      - name: Throttle 1000 requests per minute per account
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 1000
            window_size_in_seconds: 60
            response_status_code: 429
    diagnosis:
      - name: Export HAR logs to S3
        enabled: true
        config:
          har_exporter:
            transaction_max_size: 100000
            obfuscate:
              enabled: true
              exclusions:
                path_params:
                  - id
                response_headers:
                  - Retry-After
        export: s3

As void configurations:

/etc/lunar-proxy/policies.yaml

endpoints:
  - url: api.com/segmentA/{paramA}/*
    method: GET
    diagnosis:
      - name: "API /segmentA/{paramA}/ path param declaration"
        enabled: false
        config:
          void: {}
        export: file
  - url: api.com/segmentB/{paramB}/*
    method: GET
    diagnosis:
      - name: "API /segmentB/{paramB}/ path param declaration"
        enabled: false
        config:
          void: {}
        export: file
  - url: api.com/segmentC/{paramC}/*
    method: GET
    diagnosis:
      - name: "API /segmentC/{paramC}/ path param declaration"
        enabled: false
        config:
          void: {}
        export: file

Remedy and Diagnose Plugins

The main purpose of the policies.yaml file is to define remedy and diagnose plugins, which can be applied on a specific endpoint or globally to all endpoints. The structure of a remedy or diagnose plugin is as follows:

Field	Type	Description	Required?
name	String	The name of the plugin.	Yes
enabled	Boolean	Whether the plugin is enabled or not.	Yes
config	Object	The configuration of the plugin.	Yes
export (only for diagnose plugins)	String	The name of the exporter to use.	Yes, only for diagnose plugins

note

Only one export field can be specified for each diagnose plugin.

The configuration of each plugin is described in detail in the documentation of the specific plugin. For a full list of remedy and diagnose plugins see Remedy Plugins and Diagnose.

Example:

/etc/lunar-proxy/policies.yaml

global:
  remedies:
    - name: Response-Based Throttling
      enabled: true
      config:
        response_based_throttling:
          retry_after_header: Retry-After
          retry_after_type: relative_seconds
          quota_group: 1
          relevant_statuses:
            - 429
  diagnosis:
    - name: Metrics Collector
      enabled: true
      config:
        metrics_collector: {}
      export: prometheus

exporters:
  prometheus: {}

In this example, a response-based throttling remedy plugin is enabled and configured to apply to all endpoints (because it is defined under the global section). The Metrics Collector diagnose plugin is enabled and configured to apply to all endpoints. And export the collected metrics to the prometheus exporter, which is defined in the exporters section.

mTLS Configuration

Mutual Transport Layer Security (mTLS) is a protocol that provides both authentication and encryption for communication between two parties. It ensures that only authorized entities can communicate and that their data is protected from eavesdropping and tampering.

Using a Combined Certificate and Key File

If you choose to combine the certificate and key into a single .pem file, this file can be referenced under the cert field in the policies.yaml file. This approach is useful if you prefer a simplified configuration:

cat server.crt server.key > server.pem

In your policies.yaml, you would configure it as follows:

mTLS:
  - domain: api.example.com
    cert: /etc/lunar-proxy/certs/mtls/server.pem

Using Separate Certificate and Key Files

If you use separate files, ensure that the certificate and key have identical names (with different extensions), and are stored in the same directory. Only the .crt file needs to be specified in the policies.yaml file; the key will be automatically derived based on the matching name.

For example, if you have server.crt and server.crt.key:

mTLS:
  - domain: api.example.com
    cert: /etc/lunar-proxy/certs/mtls/server.crt

The key file server.crt.key will be automatically associated with server.crt.

Example Configuration In this example, the certificate and key pairs for two domains are stored in /etc/lunar-proxy/certs/mtls/. The policies.yaml only references the .crt files:

mTLS:
  - domain: api.example.com
    cert: /etc/lunar-proxy/certs/mtls/example.crt
  - domain: api.store.com
    cert: /etc/lunar-proxy/certs/mtls/store.crt

Allowed and Blocked Domains Configuration

To control access to specific domains, you can use the optional allowed_domains or blocked_domains configurations in your policies.yaml file. Each entry is treated as a regular expression (regex) to allow precise matching, offering flexibility for domain pattern definitions.

Key Considerations

Exact Match Configuration

To match a domain exactly (e.g., api.example.com), use a strict regex pattern like ^api\.example\.com$. This prevents unintended matches like api1example2com or apixexample.com, which regex might otherwise interpret as valid.

Wildcard Patterns

For more inclusive patterns, you can allow subdomains using a regex such as:

• '^([a-zA-Z0-9-]+\.)?api\.example\.com$': Allows api.example.com and any subdomains like dev.api.example.com.
• '^([a-zA-Z0-9-]+\.)?example\.com$': Allows example.com and subdomains like support.example.com.

For broader matches, you can use a wildcard pattern to allow all subdomains (e.g., ^.*\.example\.com$), which would cover all subdomains like dev.example.com and blog.example.com.

Default Behavior

• If allowed_domains is not defined: All domains are allowed by default.
• If allowed_domains is defined: Any domain not matching an entry in allowed_domains will be blocked.
• If blocked_domains is not defined: No domains are blocked by default.

Example Configuration

/etc/lunar-proxy/policies.yaml

allowed_domains:
  - "^api\\.io$"

blocked_domains:
  - "^comp\\.io$"

global:    
  remedies:
    - name: Strategy Based Throttling Quick Start
      enabled: true
      config:
        strategy_based_throttling:
          allowed_request_count: 6
          window_size_in_seconds: 60
          response_status_code: 429

In this setup, only the domains that strictly match api.io are allowed, while comp.io is blocked. The regex syntax ensures that only exact matches are accepted, providing precise control over domain access.

HTTP Proxy Route Configuration

In addition to specifying domains for general proxying, you can configure an HTTP Proxy Route under allowed_domains for secure forwarding of HTTPS requests to specific domains using the HTTP CONNECT method. This enables transparent TLS passthrough while enforcing domain-level restrictions.

note

Currently, features like discover or any other flow-based configurations are not supported in this setup. This limitation arises because the transparent proxy operates at Layer 4, bypassing the Layer 7 processing required for such features.

• Environment Variables: To enable this feature, ensure the following environment variables are set:

TLS_PASSTHROUGH_ON=true  # Enables transparent proxying
TLS_PASSTHROUGH_PORT=8880  # Specifies the port for TLS passthrough

• Testing: Use the following curl command to test proxying:

curl -p -x http://<lunar-proxy-domain>:8880 https://api.example.com/resource

Global Section

The global section contains the remedy and diagnose plugin configurations that apply to all endpoints. The section has the following structure:

Field	Type	Description	Required?
remedies	List	A list of remedy plugins to apply globally (See Remedy Plugins for more information).	No
diagnosis	List	A list of diagnose plugins to apply globally (See Diagnose for more information).	No

Example:

/etc/lunar-proxy/policies.yaml

global:
  remedies:
    - name: Response-Based Throttling
      enabled: true
      config:
        response_based_throttling:
          retry_after_header: Retry-After
          retry_after_type: relative_seconds
          quota_group: 1
          relevant_statuses:
            - 429
  diagnosis:
    - name: Metrics Collector
      enabled: true
      config:
        metrics_collector: {}
      export: prometheus

Endpoints Section

The endpoints section contains a list of defined endpoints and their associated remedy and diagnose plugin configurations. It has the following structure:

Field	Type	Description	Required?
method	String	The HTTP method (GET, POST, etc.).	Yes
url	String	The URL pattern for the endpoint (e.g. api.com/api/v1/users).*	Yes
remedies	List	A list of remedy plugins to apply to the endpoint (See Remedy Plugins for more information).	No
diagnosis	List	A list of diagnose plugins to apply to the endpoint (See Diagnose for more information).	No

Example:

/etc/lunar-proxy/policies.yaml

endpoints:
  - method: GET
    url: api.com/api/v1/users/{id}
    remedies:
      - name: Throttle 1000 requests per minute per account
        enabled: true
        config:
          strategy_based_throttling:
            allowed_request_count: 1000
            window_size_in_seconds: 60
            response_status_code: 429
    diagnosis:
      - name: Export HAR logs to S3
        enabled: true
        config:
          har_exporter:
            transaction_max_size: 100000
            obfuscate:
              enabled: true
              exclusions:
                path_params:
                  - id
                response_headers:
                  - Retry-After
        export: s3

Using YAML Anchors for Common Remedies

In scenarios where multiple endpoints under the same hostname require the same remedy configuration, you can use YAML anchors to avoid redundancy and streamline configuration management. YAML anchors allow you to define a remedy once and reference it in multiple places.

note

When using YAML anchors for remedy configurations, it's crucial to define the anchor at the top of the policies.yaml file. This ensures that the anchor is available for reference before it is used in any endpoint configurations. Defining the anchor at the bottom or after its references can lead to errors in parsing the file.

Example:

/etc/lunar-proxy/policies.yaml

# IMPORTANT: Define anchors like throttling_remedy at the top of the file
# to ensure they are available for reference before being used in endpoint configurations.

# Define an anchor for the common throttling remedy
throttling_remedy: &throttling_remedy
  enabled: true
  config:
    strategy_based_throttling:
      allowed_request_count: 1
      window_size_in_seconds: 60
      response_status_code: 429


endpoints:
  - url: api.com/*
    method: GET
    remedies:
      - <<: *throttling_remedy
        name: Strategy Based Throttling - GET


  - url: api.com/*
    method: POST
    remedies:
      - <<: *throttling_remedy
        name: Strategy Based Throttling - POST

In the above example, the throttling_remedy is defined once and then applied to all GET and POST requests to api.com/*, ensuring consistency and reducing the potential for configuration errors.

tip

This approach can be combined with other endpoint configurations, such as using wildcards or path parameters, to create flexible and powerful policy definitions.

Exporters Section

The exporters section defines configurations for exporting output data from the diagnose plugins. It has the following available configurations:

Field	Type	Description	Required?
File	Object	The configuration for the file exporter.	No
AWS S3	Object	The configuration for the S3 exporter.	No
Prometheus	Object	The configuration for the Prometheus exporter.	No

note

For more information on the available configurations for each exporter, see the documentation for the specific exporter at Data Exporters.

caution

The prometheus exporter cannot be updated while the proxy is running. To update prometheus bucket boundaries, restart Lunar Proxy.

Example:

exporters:
  file:
    file_dir: /var/log/lunar-proxy
    file_name: output.log
  s3:
    bucket_name: lunar-proxy-bucket
    region: us-east-1
  prometheus:
    bucket_boundaries:
      - 100
      - 500
      - 1000
      - 3000

Accounts Section

The accounts section defines configurations for authenticating requests to 3rd party APIs. It is used by some remedy plugins such as API Account Orchestration. It has the following available configurations:

Field	Type	Description	Required?
account_identifier	String	A unique string which acts as an identifier of the account. This is used to reference the account in the remedy plugin configuration.	Yes
tokens	List	A list of tokens to use for authenticating requests to the 3rd party API. The list can contain multiple tokens if the API requires more than one value to be set (e.g. two headers - one for username and one for password).	Yes
header	Object	The header to add to the request. The header name and value are specified in the name and value fields respectively.	Yes

Example:

accounts:
  aliceb@comp.io: # This is an account identifier
    tokens:
      - header:
          name: Authorization
          value: Bearer 123
  johnf@comp.io:
    tokens:
      - header:
          name: Authorization
          value: Bearer 456
  bobc@comp.io:
    tokens:
      - header:
          name: Authorization
          value: Bearer 789

Full Example

The following is an example of a policies.yaml file.

/etc/lunar-proxy/policies.yaml

global:
  remedies:
    - name: Response-Based Throttling
      enabled: true
      config:
        response_based_throttling:
          retry_after_header: Retry-After
          retry_after_type: relative_seconds
          quota_group: 1
          relevant_statuses:
            - 429
  diagnosis:
    - name: Metrics Collector
      enabled: true
      config:
        metrics_collector: {}
      export: prometheus

endpoints:
  - method: GET
    url: api.com/api/v1/users/{id}
    remedies:
    - name: Throttle 1000 requests per minute per account
      enabled: true
      config:
        strategy_based_throttling:
          allowed_request_count: 1000
          window_size_in_seconds: 60
          response_status_code: 429
  diagnosis:
    - name: Export HAR logs to S3
      enabled: true
      config:
        har_exporter:
          transaction_max_size: 100000
          obfuscate:
            enabled: true
            exclusions:
              path_params:
                - id
              response_headers:
                - Retry-After
      export: s3

exporters:
  file:
    file_dir: /var/log/lunar-proxy
    file_name: output.log
  s3:
    bucket_name: lunar-proxy-bucket
    region: us-east-1
  prometheus:
    bucket_boundaries:
      - 100
      - 500
      - 1000
      - 3000

In this example we have:

• A global remedy that applies response-based throttling. (See Response-Based Throttling for more information).
• A global diagnose plugin that exports metrics to Prometheus. (See Metrics Collector for more information).
• An endpoint definition for a GET request to api.com/api/v1/users/{id}. This endpoint has a remedy that applies strategy-based throttling, and a diagnose plugin that exports HAR logs to S3. (See Strategy-Based Throttling and HAR Log Collector for more information).
• Two exporters: one for exporting data to S3, and one for exporting metrics to Prometheus (See Data Exporters for a full list of available exporters).

Update policies.yaml while the Lunar Proxy is running

The policies.yaml file is the de-facto control-center of the Lunar Proxy. In some scenarios, the need to modify it might come up - whether it's adding a new policy, or removing/modifying an existing one.

The Lunar Proxy supports configuration updates while it is running for both remedy and diagnose plugin configurations. After you have altered policies.yaml according to your needs, run the apply_policies command:

Docker

docker exec lunar-proxy apply_policies

K8S

kubectl exec <lunar-proxy-pod-name> -- apply_policies

If the update is valid, you should get a success message ("successfully applied policies from file"). If the update is invalid, you will get a failure message which contains some information about the root cause of the failure. It is advised to either fix the issue or revert the changes done to policies.yaml so the system is not left in an inconsistent state.