HAR Collector Processor

Overview

The HAR Collector processor captures HTTP transaction data in the HTTP Archive (HAR) format. It enables logging and exporting of API traffic for debugging, analytics, and compliance monitoring. The collected HAR data can be stored or processed using the HAR Metrics Exporter Flow.

---

Input and Output

This processor operates within the request or response streams, ensuring that complete request-response transactions are logged after a response is generated:

• Input Stream: Request/Response – The processor captures response data after the request has been processed.

• Output Stream: Request/Response – The response continues through the stream without modification while the HAR data is collected and stored.

By sitting in the response stream, the HARCollector processor ensures that all relevant transaction details, including request metadata and response payloads, are recorded for later analysis.

---

Parameters

Each parameter is defined as a key-value pair inside the parameters section.

exporter_id

Type: string
Required: True
Defines the ID of the File Exporter configured in gateway_config.yaml.

Example:

- key: exporter_id
  value: "MyFileExporter"

---

transaction_max_size_bytes

Type: number
Required: False
Default: 32768 (32KB)
Specifies the maximum size (in bytes) of each HTTP transaction to be logged. Transactions exceeding this size will be truncated.

Example:

- key: transaction_max_size_bytes
  value: 65536

---

obfuscate_enabled

Type: boolean
Required: False
Default: false
Enables or disables obfuscation of sensitive data in the logs.

Example:

- key: obfuscate_enabled
  value: false

---

obfuscate_exclusions

Type: list_of_strings
Required: False
Default: []
Defines a list of JSON paths specifying transaction components that should be excluded from obfuscation if the processor obfuscate_enabled is True.

Example:

- key: obfuscate_exclusions
  value:
    - "$.response.header['Retry-After']"
    - "$.request.body.user.name"

In this example, the request.user.name and header['Retry-After'] values will be returned.

---

Best Practices

• Ensure exporter_id is properly configured in gateway_config.yaml.
• Use obfuscate_enabled to prevent logging sensitive information.
• Specify obfuscate_exclusions to retain critical debugging details.
• Optimize transaction_max_size_bytes to balance detail retention and log size.
• Define where the logs will be exported to in the 'gateway_config.yaml.

For more details on configuring flows, visit the Lunar.dev Flows Documentation.

---

HAR Collector Processor Template

Har Collector Example

  HARCollectorResponse:
    processor: HARCollector  # Specifies the processor used for HAR data collection
    parameters:
      - key: exporter_id
        value: "file_exporter_01"  # ID of the File Exporter defined in gateway_config.yaml
      - key: transaction_max_size_bytes
        value: 5000  # Maximum size limit for each HTTP transaction log (in bytes)
      - key: obfuscate_enabled
        value: true  # Enables obfuscation of sensitive data in the logs
      - key: obfuscate_exclusions
        value:
          - '$.request.query_param.id' # Exclude specific query parameters from obfuscation
          - '$.request.body.user.name' # Exclude specific paths in the request body from obfuscation
          - '$.response.headers["Retry-after"]' 

Use Case

The HAR Collector processor can be used to capture API request/response logs while ensuring sensitive data is obfuscated:

APILogging:
  processor: HARCollector
  parameters:
    - key: exporter_id
      value: "SecureLogExporter"
    - key: transaction_max_size_bytes
      value: 32768
    - key: obfuscate_enabled
      value: true
    - key: obfuscate_exclusions
      value:
        - "$.request.headers['X-Custom-Token']"

This configuration will log API transactions while excluding specific sensitive fields from obfuscation.