LLM Wiki — Synaptic Lattice

Bifrost

Properties1
tagsprompt-engineering, tool-origin, token-usage-reduction

Leitura de 3 min

Bifrost

What It Is (confidence: medium)

Bifrost is the open-source AI gateway from Maxim AI. As an MCP gateway it sits between Claude Code and a team’s MCP servers: Claude Code connects to a single /mcp endpoint instead of each server, and Bifrost handles discovery, tool governance, execution, and orchestration. The article is a vendor guide walking through MCP token-cost mechanics and Bifrost’s approach; benchmark claims are vendor-published.

Code Mode (confidence: medium)

The core token lever. Instead of injecting every MCP tool definition into context, Bifrost exposes connected servers as a virtual filesystem of lightweight Python stub files. The model reads only what it needs and writes a short Python script that Bifrost executes in a sandboxed Starlark interpreter. Four meta-tools replace full schema injection regardless of how many servers are connected:

  • listToolFiles — discover available servers/tools.
  • readToolFile — load Python function signatures for a server or tool.
  • getToolDocs — fetch detailed docs before using a tool.
  • executeToolCode — run the orchestration script against live tool bindings.

The article attributes this pattern to Anthropic’s code-execution-with-mcp (a Google Drive→Salesforce workflow dropped from 150,000 to 2,000 tokens) and notes Cloudflare independently observed the same exponential-savings shape. Cost is bounded by what the model actually reads, not how many tools exist.

Governance as a Cost Lever (confidence: medium)

Every request carries a virtual key scoped at tool level (not server level), so the model only ever sees definitions for tools the key is allowed to call — unauthorized tools cost zero tokens. MCP Tool Groups attach named tool collections to any combination of virtual keys, teams, customers, or providers, resolved in-memory at request time. The centralized gateway also produces a per-call audit trail (tool, server, arguments, latency, virtual key, parent LLM request, token cost) and surfaces token cost per key/tool/server plus Prometheus and OpenTelemetry metrics.

Claims (confidence: low)

Vendor-published benchmark: input tokens drop 58% at 96 tools, 84% at 251, 92% at 508, with pass rate held at 100%; 11 microseconds of overhead at 5,000 requests/second. These are Bifrost’s own numbers from getmaxim.ai — no independent benchmark captured. The article’s cited external measurements (~7,000 tokens/message for a 4-server setup, 15–20k/turn heavier) are references to third-party blogs, not vault evidence.

Implications (confidence: medium)

Bifrost is the gateway/infrastructure-layer answer to a client-side problem the prompt wiki already documents: tool-context-management (four Claude-Platform approaches) and tool-search-tool (deferral) optimize what the client loads, while Bifrost moves governance and orchestration out of the prompt entirely at the fleet level. It is a tool-origin peer of headroom and token-optimizer-mcp in the token-reduction cluster, but positioned for platform teams (dozens to hundreds of engineers) rather than single developers.

Open Questions

  • Whether the 58–92% input-token figures hold under independent measurement (vendor self-published).
  • Billed-cost impact is not directly measured — see the token-reduction-is-not-cost-reduction caveat on token-usage-reduction.
  • Whether the Starlark-sandboxed script execution preserves full tool fidelity (params, streaming, auth) versus direct MCP calls.

code-mode-orchestration — the generalized pattern Bifrost implements tool-context-management — client-side complement to the gateway layer tool-search-tool — client-side deferral; Bifrost does the same at fleet level token-usage-reduction — the cost-lever space this gateway operates in claude-code-system-prompt — primary agent the gateway fronts

Sources

  • raw/prompts/articles/bifrost-mcp-gateway-claude-code-token-costs.md

Visão de gráfico

Backlinks