Lunar.dev Logs

Logging is essential for monitoring, troubleshooting, and optimizing the Lunar.dev API Consumption Gateway. Logs are stored in /var/log/lunar-proxy, with key files providing insights into specific operations:

• aggregation-output-plugin.log: Tracks Discover feature activities, helping you monitor aggregation and proxy configuration updates.
• fluent-bit.log: Logs related to Fluent Bit, a data exporter used for exporting metrics and logs from the API Gateway.
• lunar-engine.log: Captures logs from the core Lunar.dev Engine, crucial for understanding the system's behavior, diagnosing issues, and ensuring smooth operation of the proxy.
• lunar-proxy.log: Captures the access logs passing through the gateway.

Log Rotation

To prevent log files from consuming excessive disk space, Lunar.dev now offers an automatic log rotation mechanism. This feature ensures efficient log management and retention, providing a reliable way to maintain logs without manual intervention.

The log rotation mechanism operates based on the following environment variables:

Variable	Meaning	Default Value	Possible Values
LOG_ROTATE_INTERVAL	Frequency of log rotation	hourly	"5min", "10min", "30min", "hourly", "daily", "weekly", "monthly"
LOG_ROTATE_SIZE	Maximum log file size to trigger rotation	10M	Any valid file size (e.g., 5M, 20M)
LOG_ROTATE_RETAIN	Number of rotated files to retain	3	Positive integer values

How It Works

• A cron job initiates the log rotation process at the frequency defined by LOG_ROTATE_INTERVAL.
• If a log file exceeds the size specified by LOG_ROTATE_SIZE, it is compressed and archived.
• The compressed files are stored as filename.log.[interval_count_number].gz.
• Only the number of rotated files specified by LOG_ROTATE_RETAIN will be kept. Older files are automatically removed.

Example Docker Run

docker run --rm -d -p 3000:3000 -p 8000:8000 -p 8081:8081 -p 8040:8040 \
  -e LOG_ROTATE_SIZE="20M" \
  -e LOG_ROTATE_INTERVAL="10min" \
  -e LOG_ROTATE_RETAIN="5" \
  -e LUNAR_STREAMS_ENABLED=true \
  -e TENANT_NAME="ORGANIZATION" \
  -v $(pwd):/etc/lunar-proxy \
  --name lunar-proxy lunar-proxy

Example Output

Compressed log files will appear in the following format:

/var/log/lunar-proxy/
├── lunar-engine.log
├── lunar-engine.log.1.gz
├── lunar-engine.log.2.gz
├── lunar-engine.log.3.gz

In this example, lunar-engine.log is the active log file, while .gz files are the rotated and compressed logs.

Gateway Startup Indicator

The Gateway provides a startup message to help users understand the state of key components during initialization. This message offers visibility into the status of various components, making it easier to troubleshoot startup issues.

Example Startup Message:

lunar-hub:
  APIKey: dd123456-1234-1234-1234-12345abcdefg
  Lunar Hub: Connected

Redis:
  Prefix: lunar
  URL: redis://lunar-redis:6379
  Status: Successfully initialized Redis Client

lunar-engine:
  FailSafe: Enabled
  Gateway Version: 1.0.0
  Tenant Name: CompanyName
  Log Level: TRACE
  Bind Port: 8000
  HealthCheck Port: 8040
  Engine Mode: Lunar Flows

---