System Overview
Welcome to the Arbiter system documentation. Written by engineers, for engineers. No marketing fluff, no AI-generated filler.
We built Arbiter to solve a simple but frustrating problem: LLMs are highly volatile, opinionated, and prone to silent failures. Fanning queries across multiple LLM providers simultaneously and dynamically synthesizing their outputs is the only way to get reliable, balanced responses.
At its core, Arbiter is a multi-model LLM router, evaluator, and telemetry tracker. The system takes a single user prompt, fans it out to multiple models concurrently, feeds their individual streams into an evaluation agent, and delivers a unified, high-quality response.
Core Pillars
Fanned Execution
Run N models concurrently (OpenAI, Claude, Gemini, DeepSeek, Mistral) in parallel threads. We record time-to-first-token (TTFT) and latency metrics for every single worker.
Evaluator Synthesis
Feed fanned worker responses into a single evaluator agent (like Claude Sonnet 5). The evaluator acts as a referee, highlighting contradictions and outputting a comprehensive synthesis.
Dynamic Cryptography
Supports Bring Your Own Key (BYOK). Keys are encrypted locally using AES-256-GCM before writing to Postgres, decrypted only inside the session pipeline, and never stored in plain text.
Token-Level Telemetry
Granular cost tracking pulling directly from official pricing logs. We calculate costs including cache read/write rates and tiered context bounds, storing results in Postgres.
Quick Navigation
To understand the technical design and boundaries, proceed through the following sections:
- ⬡ System Architecture — View the high-level interactive node graph and component directories.
- ⬥ OAuth & Sessions — Trace the Google PKCE callback and automatic transparent refresh flows.
- ◎ Guest vs. Logged-In Mode — Understand storage scopes (sessionStorage vs. Postgres) and rate limits.
- ⟳ SSE Orchestration Pipeline — Simulate the streaming chunk sequence and done telemetry events.
- ◆ Provider Adapters — Examine the
AgentAdaptercontract and provider mappings. - ◉ Database Data Model — Walk the Drizzle table schemas and entity relation linkages.
- ◇ Cost & Pricing — Deconstruct the caching math and context tier computations.