Cost & Pricing Model

Arbiter computes LLM pricing dynamically at the end of every stream. Instead of using flat estimation averages, the backend pulls configurations from config/modelPricing.ts and evaluates cost metrics inside calculateRunCost.

Pricing Configuration Columns

Each configuration tracks pricing per 1 million tokens (USD) and includes fields for cached read/write pricing, batch discounts, and verification dates:

export interface ModelPricing {
  provider: 'openai' | 'claude' | 'gemini' | 'deepseek' | 'mistral';
  modelId: string;
  displayName: string;
  contextWindow: string;
  inputPricePer1M: number;
  cachedInputPricePer1M?: number;
  cacheWritePricePer1M?: number;
  cacheReadPricePer1M?: number;
  outputPricePer1M: number;
  batchDiscountPct: number;
  notes?: string;
  sourceUrl: string;
  lastVerifiedDate: string;
}

Cost Calculation Rules

The helper function calculateRunCost inside lib/agents/utils.ts applies vendor-specific algorithms:

1. OpenAI & DeepSeek Caching

Charged by separating normal prompt tokens from cache hits (defined by cachedInputPricePer1M). For DeepSeek, cache hits drop prompt prices to $0.0028/M tokens.

2. Anthropic Claude Cache Reads & Writes

Claude splits input tokens into standard inputs, cache writes (which incur a 25% premium, charged at cacheWritePricePer1M), and cache reads (charged at a 90% discount, cacheReadPricePer1M).

3. Google Gemini Tiered Pricing

Gemini pricing changes based on whether the context length exceeds 200,000 tokens. For gemini-2.5-pro, input tokens ≤ 200K are charged at $1.25/M, whereas tokens > 200K double to $2.50/M.

4. Batch Processing Discounts

If a model is processed in batch mode, a flat discount multiplier is applied (e.g. 50% discount for Gemini, Claude, OpenAI, and Mistral):

const discountMultiplier = isBatch ? (1 - model.batchDiscountPct / 100) : 1;
return (inputCost + outputCost) * discountMultiplier;