Skip to main content
Version: 0.10.x

Metrics Collector

Lunar can be easily set up to collect comprehensive metrics on the traffic passing through it. These metrics can then be exported to a metric server like Prometheus for utilization in dashboards and alerts.

Specific request or response headers can be selected and they will be a label in the exported metrics by using the request_header_names and response_header_names sections. It means the metrics can be filtered and aggregated based on different values of these headers. For example, you can select the X-Group header and the metrics will look like this:

lunar_transaction_sum{method="GET",normalized_url="provider1.com/users",request_X_Group="production",status_code="200"} 2286
lunar_transaction_sum{method="GET",normalized_url="provider1.com/users",request_X_Group="staging",status_code="200"} 2189

Additionally, specific counters can be defined to extract values from the request or response payloads using the counters sections. These values will be stored as metrics and can be used for alerting or monitoring purposes. For example, you can extract the value of the Retry-After header from the response and store it as a metric, which will look like this:

lunar_response_headers_Retry_After_total{method="GET",normalized_url="provider1.com/users",status_code="429"} 20845
lunar_response_headers_Retry_After_total{method="GET",normalized_url="provider2.com/posts",status_code="429"} 24199

Set Up Metrics Collector Diagnosis Plugin​

Global Example​

You have the flexibility to configure the Metrics Collector diagnosis plugin either globally or per endpoint. No specific plugin configuration is required for this setup.

/etc/lunar-proxy/policies.yaml
global:
  diagnosis:
    - enabled: true
      name: "Global Metrics Collector"
      export: "prometheus"
      config:
        metrics_collector:
          request_header_names: # Optional
            - "x-group"
          response_header_names: # Optional
            - "content-type"
          counters: # Optional
            - name_suffix: "Retry-After"
              payload: "response_headers"
              key: "Retry-After"
note

To use the metrics collector without any specific configuration, you can simply declare it as metrics_collector: {}.

Endpoint-specific Example​

/etc/lunar-proxy/policies.yaml
endpoints:
  - url: api.com/resource/{id}
    method: PUT
    diagnosis:
      - enabled: true
        name: "api.com Metrics Collector"
        export: "prometheus"
        config:
          metrics_collector:
            request_header_names: # Optional
              - "x-lunar-consumer-tag"
            response_header_names: # Optional
              - "content-type"
            counters: # Optional
              - name_suffix: "Retry-After"
                payload: "response_headers"
                key: "Retry-After"

Supported Exporters​

note
  • The plugin can export data to File, S3, or Prometheus. However, the most common usage is to export to Prometheus, which seamlessly integrates with Lunar and can be accessed at :3000/metrics.
  • This diagnosis plugin has no fields to configure, it just needs to be declared in order to be turned on; hence the somewhat uncommon syntax on metrics_collector: {} (yes, this is perfectly valid YAML!)
  • This diagnosis plugin exports the normalized_url field.
    • If defined as a global policy, it will be translated to the host part of the URL of any incoming request.
    • If defined as an endpoint-specific policy, it will simply be the declared endpoint URL - in the example above, it would be api.com/resource/[id].

This is done in order to ensure low cardinality of label values which is crucial for good performance of metric systems such as Prometheus.