Customize & Configure
Lunar offers a comprehensive set of configuration options through its policies.yaml
file located in the /etc/lunar-proxy
directory. Serving as the central configuration hub, this file acts as the backbone of Lunar's functionality.
The configuration abilities enable engineering teams to define global policies or end-point related policies, and is used to define remedy and diagnose plugins.
If no remedy or diagnose plugins are defined, Lunar Proxy will act as a regular proxy, forwarding requests to the upstream server and returning the response.
This file is loaded by Lunar Proxy on startup and can be edited and reloaded at runtime. For more information on how to update the configuration at runtime, see Update policies.yaml
while Lunar Proxy is running.
To effectively manage your Lunar configuration and policies, it's recommended to create a local file named policies.yaml
with the described settings. Once you have your custom configuration file ready, you can run Lunar Proxy with this file by mounting it into the Docker container.
Create a local file named policies.yaml
with your desired Lunar configuration settings (as instructed below).
Use the -v
flag in your Docker run command to mount your local policies.yaml
file to the /etc/lunar-proxy
directory inside the Lunar Proxy container. In the docker
command below, please make sure to replace your_local_path
with the actual path to your policies.yaml
file on your host machine, and modify the TENANT_NAME
as you desire.
Here's the modified Docker run command with the local file mount:
docker run -d --rm -p 8000:8000 -p 8081:8081 -p 8040:8040 -e TENANT_NAME="ORGANIZATION" -v "your_local_path":/etc/lunar-proxy --name lunar-proxy lunarapi/lunar-proxy:latest
By mounting your custom policies.yaml
file into the container, you can easily update your Lunar Proxy configuration without the need to stop and restart the container. After making changes to your local policies.yaml
file, use the apply_policies
command to apply the new policies.
docker exec lunar-proxy apply_policies
Structure
The policies.yaml
file is divided into four main sections, which are all optional: allowed_domains, blocked_domains, global, endpoints, exporters, and accounts. The sections are described in detail below.
Overview
The structure of policies.yaml
file is as follows:
| ------------------------------------------- | ----------------------------------------------------------------------- | --------- | | allowed_domains | A list of domains to allow through the proxy. | No | | blocked_domains | A list of domains to block from the proxy. | No | | global | A list of global remedy and diagnose plugins to apply to all endpoints. | No | | endpoints | A list of endpoints to configure remedy and diagnose plugins for. | No | | exporters | A list of exporters to configure. | No | | accounts | A list of accounts to configure. | No |
Template
allowed_domains: # (optional)
- <allowed_domain>
blocked_domains: # (optional)
- <blocked_domain>
global: # (optional)
remedies: # (optional) A list of remedy plugins to apply to all endpoints.
- name: <user_defined_name>
enabled: <true/false>
config:
<remedy_plugin_type>:
# Remedy-specific configuration...
diagnosis: # (optional) A list of diagnose plugins to apply to all endpoints.
- name: <user_defined_name>
enabled: <true/false>
export: <exporter_type> # The name of the exporter to use.
config:
<diagnose_plugin_type>:
# Diagnose-specific configuration...
endpoints: # (optional)
- method: <http_method> # GET/POST/PUT/DELETE/...
url: <url_pattern>
remedies: # (optional) A list of remedy plugins to apply to the endpoint.
- name: <user_defined_name>
enabled: <true/false>
config:
<remedy_plugin_type>:
# Remedy-specific configuration...
diagnosis: # (optional) A list of diagnose plugins to apply to the endpoint.
- name: <user_defined_name>
enabled: <true/false>
export: <exporter_type> # The name of the exporter to use.
config:
<diagnose_plugin_type>:
# Diagnose-specific configuration...
exporters: # (optional)
file:# (optional)
# ...
prometheus:# (optional)
# ...
s3:# (optional)
# ...
accounts:# (optional)
# ...
Remedy and Diagnose Plugins
The main purpose of the policies.yaml
file is to define remedy and diagnose plugins, which can be applied on a specific endpoint or globally to all endpoints.
The structure of a remedy or diagnose plugin is as follows:
Field | Type | Description | Required? |
---|---|---|---|
name | String | The name of the plugin. | Yes |
enabled | Boolean | Whether the plugin is enabled or not. | Yes |
config | Object | The configuration of the plugin. | Yes |
export (only for diagnose plugins) | String | The name of the exporter to use. | Yes, only for diagnose plugins |
Only one export
field can be specified for each diagnose plugin.
The configuration of each plugin is described in detail in the documentation of the specific plugin. For a full list of remedy and diagnose plugins see Remedy Plugins and Diagnose.
Example:
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus
exporters:
prometheus: {}
In this example, a response-based throttling remedy plugin is enabled and configured to apply to all endpoints (because it is defined under the global section). The Metrics Collector diagnose plugin is enabled and configured to apply to all endpoints. And export the collected metrics to the prometheus exporter, which is defined in the exporters section.
Allowed and Blocked Domains Configuration
You can control which domains are allowed or blocked using the optional allowed_domains
or blocked_domains
configurations in your policies.yaml
file. This feature also supports regex implementations, *example.com - would allow dev.example.com and help.example.com.
Usage
- If
allowed_domains
is not defined: All domains are allowed. - If
allowed_domains
is defined: Any domain not matching the values in allowed_domains will be blocked. - If
blocked_domains
is not defined: No domains are blocked.
Example Configuration
allowed_domains:
- "api.io"
blocked_domains:
- "comp.io"
global:
remedies:
- name: Strategy Based Throttling Quick Start
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 6
window_size_in_seconds: 60
response_status_code: 429
### Global Section
The global section contains the remedy and diagnose plugin configurations that apply to all endpoints. The section has the following structure:
| Field | Type | Description | Required? |
| ----------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------ | --------- |
| [remedies](#remedy-and-diagnose-plugins) | List | A list of remedy plugins to apply globally (See [Remedy Plugins](/product-features/remedy-plugins) for more information). | No |
| [diagnosis](#remedy-and-diagnose-plugins) | List | A list of diagnose plugins to apply globally (See [Diagnose](/product-features/diagnose-export/diagnoseIndex) for more information). | No |
#### Example:
```yaml
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus
Endpoints Section
The endpoints section contains a list of defined endpoints and their associated remedy and diagnose plugin configurations. It has the following structure:
Field | Type | Description | Required? |
---|---|---|---|
method | String | The HTTP method (GET , POST , etc.). | Yes |
url | String | The URL pattern for the endpoint (e.g. api.com/api/v1/users ).* | Yes |
remedies | List | A list of remedy plugins to apply to the endpoint (See Remedy Plugins for more information). | No |
diagnosis | List | A list of diagnose plugins to apply to the endpoint (See Diagnose for more information). | No |
URL Matching
- The URL pattern is treated as an exact match. If you want to create a policy for multiple variations of the endpoint try using wildcard. Include a wildcard at the end of the URL. Example:
api.com/api/v1/users/*
. - Path parameters are supported. Example:
api.com/api/v1/users/{id}
. - You can use a mixture of wildcards and path parameters in the URL pattern. Example:
api.com/api/v1/users/{id}/*
. - The URL pattern should not include the scheme or port.
Example:
endpoints:
- method: GET
url: api.com/api/v1/users/{id}
remedies:
- name: Throttle 1000 requests per minute per account
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 1000
window_size_in_seconds: 60
response_status_code: 429
diagnosis:
- name: Export HAR logs to S3
enabled: true
config:
har_exporter:
transaction_max_size: 100000
obfuscate:
enabled: true
exclusions:
path_params:
- id
response_headers:
- Retry-After
export: s3
Exporters Section
The exporters section defines configurations for exporting output data from the diagnose plugins. It has the following available configurations:
Field | Type | Description | Required? |
---|---|---|---|
File | Object | The configuration for the file exporter. | No |
AWS S3 | Object | The configuration for the S3 exporter. | No |
Prometheus | Object | The configuration for the Prometheus exporter. | No |
For more information on the available configurations for each exporter, see the documentation for the specific exporter at Data Exporters.
The prometheus
exporter cannot be updated while the proxy is running. To update prometheus bucket boundaries, restart Lunar Proxy.
Example:
exporters:
file:
file_dir: /var/log/lunar-proxy
file_name: output.log
s3:
bucket_name: lunar-proxy-bucket
region: us-east-1
prometheus:
bucket_boundaries:
- 100
- 500
- 1000
- 3000
Accounts Section
The accounts section defines configurations for authenticating requests to 3rd party APIs. It is used by some remedy plugins such as API Account Orchestration. It has the following available configurations:
Field | Type | Description | Required? |
---|---|---|---|
account_identifier | String | A unique string which acts as an identifier of the account. This is used to reference the account in the remedy plugin configuration. | Yes |
tokens | List | A list of tokens to use for authenticating requests to the 3rd party API. The list can contain multiple tokens if the API requires more than one value to be set (e.g. two headers - one for username and one for password). | Yes |
header | Object | The header to add to the request. The header name and value are specified in the name and value fields respectively. | Yes |
Example:
accounts:
aliceb@comp.io: # This is an account identifier
tokens:
- header:
name: Authorization
value: Bearer 123
johnf@comp.io:
tokens:
- header:
name: Authorization
value: Bearer 456
bobc@comp.io:
tokens:
- header:
name: Authorization
value: Bearer 789
Full Example
The following is an example of a policies.yaml
file.
global:
remedies:
- name: Response-Based Throttling
enabled: true
config:
response_based_throttling:
retry_after_header: Retry-After
retry_after_type: relative_seconds
quota_group: 1
relevant_statuses:
- 429
diagnosis:
- name: Metrics Collector
enabled: true
config:
metrics_collector: {}
export: prometheus
endpoints:
- method: GET
url: api.com/api/v1/users/{id}
remedies:
- name: Throttle 1000 requests per minute per account
enabled: true
config:
strategy_based_throttling:
allowed_request_count: 1000
window_size_in_seconds: 60
response_status_code: 429
diagnosis:
- name: Export HAR logs to S3
enabled: true
config:
har_exporter:
transaction_max_size: 100000
obfuscate:
enabled: true
exclusions:
path_params:
- id
response_headers:
- Retry-After
export: s3
exporters:
file:
file_dir: /var/log/lunar-proxy
file_name: output.log
s3:
bucket_name: lunar-proxy-bucket
region: us-east-1
prometheus:
bucket_boundaries:
- 100
- 500
- 1000
- 3000
In this example we have:
- A global remedy that applies response-based throttling. (See Response-Based Throttling for more information).
- A global diagnose plugin that exports metrics to Prometheus. (See Metrics Collector for more information).
- An endpoint definition for a GET request to
api.com/api/v1/users/{id}
. This endpoint has a remedy that applies strategy-based throttling, and a diagnose plugin that exports HAR logs to S3. (See Strategy-Based Throttling and HAR Log Collector for more information). - Two exporters: one for exporting data to S3, and one for exporting metrics to Prometheus (See Data Exporters for a full list of available exporters).
Update policies.yaml
while the Lunar Proxy is running
The policies.yaml
file is the de-facto control-center of the Lunar Proxy. In some scenarios, the need to modify it might come up - whether it's adding a new policy, or removing/modifying an existing one.
The Lunar Proxy supports configuration updates while it is running for both remedy and diagnose plugin configurations. After you have altered policies.yaml
according to your needs, run the apply_policies
command:
- Docker
- K8S
docker exec lunar-proxy apply_policies
kubectl exec <lunar-proxy-pod-name> -- apply_policies
If the update is valid, you should get a success message ("successfully applied policies from file"). If the update is invalid, you will get a failure message which contains some information about the root cause of the failure. It is advised to either fix the issue or revert the changes done to policies.yaml
so the system is not left in an inconsistent state.