Skip to main content
Version: Next

Count LLM Tokens Processor

Count LLM Tokens Processor​

CountLLMTokens Processor
CountLLMTokens:
    processor: CountLLMTokens # token counting processor
    parameters:
      - key: store_count_header
        value: <header-name> # Optional: override the default custom header we wish to update with the request token count, Default: x-lunar-estimated-tokens
      - key: encoding
        value: <Encoding> # Optional: override the default type of encoding using for tokenization, e.g., cl100k_base, o200k_base, p50k_base, r50k_base
      - key: model 
        value: <model-name> # Optional: override model name instead of Lunar.dev automatic detection (can be specific or use wilcard, to include range of models) - gpt-4o-*, gpt-4-*, gpt-4, gpt-4o etc.