Flow Configuration Template

Flow Configuration

/etc/lunar-proxy/flows/flow.yaml

name: <FlowName> # The name of the flow, e.g., ClientSideLimitingFlow

filter:
  url: <URLPattern> # Define the URL pattern for the filter, e.g., api.example.com/*
  method: ["<HTTPMethod>"] # Optional: List of HTTP methods, e.g., GET, POST
  headers:
    - key: <HeaderKey> # Optional: Header key, e.g., 'X-API-Key'
      value: <HeaderValue> # Optional: header value, e.g., '12345', '67890'

processors:
  CountLLMTokens:
    processor: CountLLMTokens # token counting processor
    parameters:
      - key: store_count_header
        value: <header-name> # Optional: override the default custom header we wish to update with the request token count, Default: x-lunar-estimated-tokens
      - key: encoding
        value: <Encoding> # Optional: override the default type of encoding using for tokenization, e.g., cl100k_base, o200k_base, p50k_base, r50k_base
      - key: model 
        value: <model-name> # Optional: override model name instead of lunar automatic detection (can be specific or use wilcard, to include range of models) - gpt-4o-*, gpt-4-*, gpt-4, gpt-4o etc.

flow:
  request:
    - from:
        stream:
          name: globalStream # The stream to start the request flow
          at: start # Start point
      to:
        processor:
          name: CountLLMTokens # Process the request through the Limiter processor

    - from:
        processor:
          name: CountLLMTokens # After the Limiter processor
      to:
        stream:
          name: globalStream # Send the request to the global stream
          at: end # End point of the request flow

  response:
    - from:
        stream:
          name: globalStream # The stream to start the request flow
          at: start # Start point
      to:
        stream:
          name: globalStream # Send response back to the global stream
          at: end # End point of the response flow

Parameter	Description	Type	Mandatory/Optional	Example
name	The name of the flow, describing its purpose or function.	String	Mandatory	ClientSideLimitingFlow
filter.url	URL pattern to match requests for this flow.	String	Mandatory	api.website.com/*
filter.method	List of HTTP methods (GET, POST, etc.) that the flow applies to.	List	Optional	['GET', 'POST']
filter.headers.key	Header key for filtering requests. Used to apply limits based on specific headers.	String	Optional	X-API-Key
filter.headers.value	header value to match the header key.	List	Optional	'67890'
processors.CountLLMTokens.processor	The processor that handles counting LLMs tokens.	String	Mandatory	CountLLMTokens
processors.CountLLMTokens.parameters.key	specify the parameter name for the processor.	String	Mandatory	store_count_header, encoding
processors.CountLLMTokens.parameters.value	Value of the Custom Quota used by the processor.	String	Mandatory	x-my-header, cl100k_base

Quota Configuration

Please refer to the Custom Quota Page.