Skip to main content
Version: 0.10.x

Customize & Configure

Lunar offers a comprehensive set of configuration options through its policies.yaml file located in the /etc/lunar-proxy directory. Serving as the central configuration hub, this file acts as the backbone of Lunar's functionality.

The configuration abilities enable engineering teams to define global policies or end-point related policies, and is used to define remedy and diagnose plugins.

If no remedy or diagnose plugins are defined, Lunar Proxy will act as a regular proxy, forwarding requests to the upstream server and returning the response.

This file is loaded by Lunar Proxy on startup and can be edited and reloaded at runtime. For more information on how to update the configuration at runtime, see Update policies.yaml while Lunar Proxy is running.

tip

To effectively manage your Lunar configuration and policies, it's recommended to create a local file named policies.yaml with the described settings. Once you have your custom configuration file ready, you can run Lunar Proxy with this file by mounting it into the Docker container.

Create a local file named policies.yaml with your desired Lunar configuration settings (as instructed below). Use the -v flag in your Docker run command to mount your local policies.yaml file to the /etc/lunar-proxy directory inside the Lunar Proxy container. In the docker command below, please make sure to replace your_local_path with the actual path to your policies.yaml file on your host machine, and modify the TENANT_NAME as you desire.

Here's the modified Docker run command with the local file mount:

docker run -d --rm -p 8000:8000 -p 8081:8081 -p 8040:8040 -e TENANT_NAME="ORGANIZATION" -v "your_local_path":/etc/lunar-proxy --name lunar-proxy lunarapi/lunar-proxy:latest

By mounting your custom policies.yaml file into the container, you can easily update your Lunar Proxy configuration without the need to stop and restart the container. After making changes to your local policies.yaml file, use the apply_policies command to apply the new policies.

docker exec lunar-proxy apply_policies

Structure

The policies.yaml file is divided into four main sections, which are all optional: allowed_domains, blocked_domains, global, endpoints, exporters, and accounts. The sections are described in detail below.

Overview

The structure of policies.yaml file is as follows:

| ------------------------------------------- | ----------------------------------------------------------------------- | --------- | | allowed_domains | A list of domains to allow through the proxy. | No | | blocked_domains | A list of domains to block from the proxy. | No | | global | A list of global remedy and diagnose plugins to apply to all endpoints. | No | | endpoints | A list of endpoints to configure remedy and diagnose plugins for. | No | | exporters | A list of exporters to configure. | No | | accounts | A list of accounts to configure. | No |

Template

/etc/lunar-proxy/policies.yaml
allowed_domains: # (optional)
- <allowed_domain>

blocked_domains: # (optional)
- <blocked_domain>

global: # (optional)
remedies: # (optional) A list of remedy plugins to apply to all endpoints.
- name: <user_defined_name>
enabled: <true/false>
config:
<remedy_plugin_type>:
# Remedy-specific configuration...

diagnosis: # (optional) A list of diagnose plugins to apply to all endpoints.
- name: <user_defined_name>
enabled: <true/false>
export: <exporter_type> # The name of the exporter to use.
config:
<diagnose_plugin_type>:
# Diagnose-specific configuration...

endpoints: # (optional)
- method: <http_method> # GET/POST/PUT/DELETE/...
url: <url_pattern>
remedies: # (optional) A list of remedy plugins to apply to the endpoint.
- name: <user_defined_name>
enabled: <true/false>
config:
<remedy_plugin_type>:
# Remedy-specific configuration...
diagnosis: # (optional) A list of diagnose plugins to apply to the endpoint.
- name: <user_defined_name>
enabled: <true/false>
export: <exporter_type> # The name of the exporter to use.
config:
<diagnose_plugin_type>:
# Diagnose-specific configuration...

exporters: # (optional)
file:# (optional)
# ...
prometheus:# (optional)
# ...
s3:# (optional)
# ...

accounts:# (optional)
# ...

Remedy and Diagnose Plugins

The main purpose of the policies.yaml file is to define remedy and diagnose plugins, which can be applied on a specific endpoint or globally to all endpoints. The structure of a remedy or diagnose plugin is as follows:

FieldTypeDescriptionRequired?
nameStringThe name of the plugin.Yes
enabledBooleanWhether the plugin is enabled or not.Yes
configObjectThe configuration of the plugin.Yes
export (only for diagnose plugins)StringThe name of the exporter to use.Yes, only for diagnose plugins
note

Only one export field can be specified for each diagnose plugin.

The configuration of each plugin is described in detail in the documentation of the specific plugin. For a full list of remedy and diagnose plugins see Remedy Plugins and Diagnose.

Example:

/etc/lunar-proxy/policies.yaml
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus

exporters:
prometheus: {}

In this example, a response-based throttling remedy plugin is enabled and configured to apply to all endpoints (because it is defined under the global section). The Metrics Collector diagnose plugin is enabled and configured to apply to all endpoints. And export the collected metrics to the prometheus exporter, which is defined in the exporters section.

Allowed and Blocked Domains Configuration

You can control which domains are allowed or blocked using the optional allowed_domains or blocked_domains configurations in your policies.yaml file. This feature also supports regex implementations, *example.com - would allow dev.example.com and help.example.com.

Usage

  • If allowed_domains is not defined: All domains are allowed.
  • If allowed_domains is defined: Any domain not matching the values in allowed_domains will be blocked.
  • If blocked_domains is not defined: No domains are blocked.

Example Configuration

allowed_domains:
- "api.io"

blocked_domains:
- "comp.io"

global:
remedies:
- name: Strategy Based Throttling Quick Start
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 6
window_size_in_seconds: 60
response_status_code: 429

### Global Section

The global section contains the remedy and diagnose plugin configurations that apply to all endpoints. The section has the following structure:

| Field | Type | Description | Required? |
| ----------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------ | --------- |
| [remedies](#remedy-and-diagnose-plugins) | List | A list of remedy plugins to apply globally (See [Remedy Plugins](/product-features/remedy-plugins) for more information). | No |
| [diagnosis](#remedy-and-diagnose-plugins) | List | A list of diagnose plugins to apply globally (See [Diagnose](/product-features/diagnose-export/diagnoseIndex) for more information). | No |

#### Example:

```yaml
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus

Endpoints Section

The endpoints section contains a list of defined endpoints and their associated remedy and diagnose plugin configurations. It has the following structure:

FieldTypeDescriptionRequired?
methodStringThe HTTP method (GET, POST, etc.).Yes
urlStringThe URL pattern for the endpoint (e.g. api.com/api/v1/users).*Yes
remediesListA list of remedy plugins to apply to the endpoint (See Remedy Plugins for more information).No
diagnosisListA list of diagnose plugins to apply to the endpoint (See Diagnose for more information).No

URL Matching

note
  • The URL pattern is treated as an exact match. If you want to create a policy for multiple variations of the endpoint try using wildcard. Include a wildcard at the end of the URL. Example: api.com/api/v1/users/*.
  • Path parameters are supported. Example: api.com/api/v1/users/{id}.
  • You can use a mixture of wildcards and path parameters in the URL pattern. Example: api.com/api/v1/users/{id}/*.
  • The URL pattern should not include the scheme or port.

Example:

/etc/lunar-proxy/policies.yaml
endpoints:
- method: GET
url: api.com/api/v1/users/{id}
remedies:
- name: Throttle 1000 requests per minute per account
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 1000
window_size_in_seconds: 60
response_status_code: 429
diagnosis:
- name: Export HAR logs to S3
enabled: true
config:
har_exporter:
transaction_max_size: 100000
obfuscate:
enabled: true
exclusions:
path_params:
- id
response_headers:
- Retry-After
export: s3

Exporters Section

The exporters section defines configurations for exporting output data from the diagnose plugins. It has the following available configurations:

FieldTypeDescriptionRequired?
FileObjectThe configuration for the file exporter.No
AWS S3ObjectThe configuration for the S3 exporter.No
PrometheusObjectThe configuration for the Prometheus exporter.No
note

For more information on the available configurations for each exporter, see the documentation for the specific exporter at Data Exporters.

caution

The prometheus exporter cannot be updated while the proxy is running. To update prometheus bucket boundaries, restart Lunar Proxy.

Example:

exporters:
file:
file_dir: /var/log/lunar-proxy
file_name: output.log
s3:
bucket_name: lunar-proxy-bucket
region: us-east-1
prometheus:
bucket_boundaries:
- 100
- 500
- 1000
- 3000

Accounts Section

The accounts section defines configurations for authenticating requests to 3rd party APIs. It is used by some remedy plugins such as API Account Orchestration. It has the following available configurations:

FieldTypeDescriptionRequired?
account_identifierStringA unique string which acts as an identifier of the account. This is used to reference the account in the remedy plugin configuration.Yes
tokensListA list of tokens to use for authenticating requests to the 3rd party API. The list can contain multiple tokens if the API requires more than one value to be set (e.g. two headers - one for username and one for password).Yes
headerObjectThe header to add to the request. The header name and value are specified in the name and value fields respectively.Yes

Example:

accounts:
aliceb@comp.io: # This is an account identifier
tokens:
- header:
name: Authorization
value: Bearer 123
johnf@comp.io:
tokens:
- header:
name: Authorization
value: Bearer 456
bobc@comp.io:
tokens:
- header:
name: Authorization
value: Bearer 789

Full Example

The following is an example of a policies.yaml file.

/etc/lunar-proxy/policies.yaml
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus

endpoints:
- method: GET
url: api.com/api/v1/users/{id}
remedies:
- name: Throttle 1000 requests per minute per account
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 1000
window_size_in_seconds: 60
response_status_code: 429
diagnosis:
- name: Export HAR logs to S3
enabled: true
config:
har_exporter:
transaction_max_size: 100000
obfuscate:
enabled: true
exclusions:
path_params:
- id
response_headers:
- Retry-After
export: s3

exporters:
file:
file_dir: /var/log/lunar-proxy
file_name: output.log
s3:
bucket_name: lunar-proxy-bucket
region: us-east-1
prometheus:
bucket_boundaries:
- 100
- 500
- 1000
- 3000

In this example we have:

  • A global remedy that applies response-based throttling. (See Response-Based Throttling for more information).
  • A global diagnose plugin that exports metrics to Prometheus. (See Metrics Collector for more information).
  • An endpoint definition for a GET request to api.com/api/v1/users/{id}. This endpoint has a remedy that applies strategy-based throttling, and a diagnose plugin that exports HAR logs to S3. (See Strategy-Based Throttling and HAR Log Collector for more information).
  • Two exporters: one for exporting data to S3, and one for exporting metrics to Prometheus (See Data Exporters for a full list of available exporters).

Update policies.yaml while the Lunar Proxy is running

The policies.yaml file is the de-facto control-center of the Lunar Proxy. In some scenarios, the need to modify it might come up - whether it's adding a new policy, or removing/modifying an existing one.

The Lunar Proxy supports configuration updates while it is running for both remedy and diagnose plugin configurations. After you have altered policies.yaml according to your needs, run the apply_policies command:

docker exec lunar-proxy apply_policies

If the update is valid, you should get a success message ("successfully applied policies from file"). If the update is invalid, you will get a failure message which contains some information about the root cause of the failure. It is advised to either fix the issue or revert the changes done to policies.yaml so the system is not left in an inconsistent state.

Click me for guidance 😀