Tool Mart

Using one AI coding assistant is a configuration file. Using five vendors side by side — Claude, OpenAI Codex, Google Gemini, Mistral, and DeepSeek — turns into a systems problem of conflicting credentials, incompatible protocol assumptions, and silent permission leaks.

Most engineering teams are moving toward multi-model workflows for a simple economic reason: frontier reasoning models are worth their price on architecture and adversarial code review, while faster workhorse models handle bounded implementation and research at a fraction of the cost. Tool Mart is the control plane I built to run that fleet from a single declarative catalog.

The problem: five vendors, five silos

For most of the year I worked across two coding CLIs: Claude Code and OpenAI Codex. Keeping their tool connections, reusable skills, and permission rules aligned by hand was tedious, though manageable if I remembered to edit both configuration files whenever I added a tool.

When I expanded to a five-vendor setup — adding Google’s agent harness for orchestration alongside Mistral and DeepSeek as high-volume coding and research workhorses — manual configuration broke down immediately:

• Every vendor speaks a different config dialect. Some read JSON, others read TOML, and each expects slightly different YAML frontmatter inside reusable skill definitions.
• Rotating OAuth tokens collide. Services like Linear and Coda issue rotating OAuth 2.0 refresh tokens. When two agents hold separate copies of a credential file, the first agent to refresh its token invalidates the token the second agent is holding — breaking tool calls mid-session.
• Tool schemas silently eat context windows. An orchestrator that loads tool schemas on demand can connect to fourteen MCP servers at zero upfront cost. A headless worker that eagerly injects every connected tool definition into its system prompt burns over 100,000 tokens before reading the first line of the task.
• Permission boundaries behave differently under auto-approval. What looks like a “read-only” mode in one CLI can leave shell execution wide open in another.

Tool Mart replaces five separate configuration silos with one declarative manifest and a lightweight runtime bridge.

One declarative manifest and shared credential store compile into native configuration files, linked skill directories, and permission profiles for five vendor harnesses.

---

How it works

1. One declarative catalog, zero secrets in Git

The catalog declares four layers in one place:

1. 14 MCP servers across four transport patterns:
   • 5 Cloud HTTP services (Memory, Library, News, Calendar, and Docs), authenticated via API keys pulled from the secrets vault at sync time.
   • 2 OAuth 2.0 cloud services (Linear and Coda), routed through Tool Mart’s local stdio-to-HTTP OAuth bridge.
   • 1 local browser daemon for authenticated web automation.
   • 6 local stdio tools (GitHub, Document Search, academic paper discovery, email, job monitoring, and browser DevTools).
2. 4 skill packs (over 60 reusable workflows) linked into each agent’s discovery folder via OS directory junctions, so a skill edit is live everywhere immediately.
3. Lifecycle hooks for session-start context recall, automatic session naming, and end-of-turn memory extraction.
4. Role-specific permission policies that match each agent’s job in the fleet.

Syncing merges these declarations non-destructively into each vendor’s config file — preserving model and provider settings while updating the managed tool, hook, and permission blocks. A companion doctor command verifies skill frontmatter, checks OAuth token health, and runs live JSON-RPC handshakes against every configured server.

2. Solving split-brain OAuth across parallel agents

Linear and Coda protect their MCP endpoints with OAuth 2.0 PKCE and rotating refresh tokens. If two CLI harnesses maintain separate token stores, the first agent to refresh an expired access token invalidates the refresh token held by the other agent. Worse, several CLI harnesses either lack built-in OAuth flows for arbitrary HTTP servers or trap tokens inside proprietary keychains.

Tool Mart solves this with a shared stdio-to-HTTP bridge backed by a single credential store and an OS file lock:

When five agents run in parallel, OS file locking and atomic file replacement prevent concurrent refresh calls from invalidating rotating OAuth tokens.

When an access token is within 60 seconds of expiry — or when the remote server returns 401 Unauthorized — the bridge acquires an exclusive OS byte-range lock, re-reads the credential file from disk in case another agent process already refreshed the token while waiting for the lock, and only calls the token endpoint if the on-disk token is still stale. The new token pair is written to a temporary file and swapped into place atomically.

Live multi-agent use surfaced a subtle OAuth edge case with Coda: when a rotating refresh token is replayed inside Coda’s replay-protection window, the server can return HTTP 200 with an already-expired access token (expires_in <= 0). Without a guard, an agent enters a tight 401 -> refresh -> 200 (expired) -> 401 loop. The bridge detects expires_in <= 0, preserves the rotated refresh token, and steps cleanly past the replay cache on the next retry.

3. Context-driven session naming

When four or five terminal sessions run in parallel, generic session IDs become indistinguishable. Tool Mart hooks into turn completion to name sessions automatically from conversation context:

• Cadence: Names the session after the first turn completes, then refreshes every three user turns so the title evolves if the session pivots to a new problem.
• Format: "<Project or topic> · <Specific task or goal>" (capped at 72 characters).
• Direct HTTPS inference: Rather than spawning a child CLI process (which costs 15–30 seconds of cold startup and risks triggering global lifecycle hooks recursively), the naming helper calls a fast small model directly over HTTPS in ~1.5–3 seconds, with a recursion guard if it ever falls back to a local CLI.

---

Hard-won multi-vendor lessons

Operating five different vendor harnesses against the same local repositories and cloud backends surfaced four lessons that don’t appear in single-vendor setups.

1. Read-only profiles must use allowlists, never denylists

When delegating a read-only code review to a subagent under auto-approval, the intuitive configuration disables the file-editing tools (disabled_tools = ["edit", "write_file"]). Under auto-approval, that is a trap: any enabled shell tool (bash, powershell) remains free to mutate the repository or run destructive commands.

2. Tool-schema token budgeting for headless workers

Connecting all 14 MCP servers to an interactive orchestrator works well when the harness loads tool schemas lazily on demand. Harnesses that eagerly inject every connected MCP tool definition into the system prompt pay a steep upfront tax: all 14 servers add roughly 100,000+ tokens of JSON schema before the first user message is read.

When I first wired a headless DeepSeek worker to the full 14-server catalog, simple background tasks were starting with half their effective context window already consumed by document, issue-tracker, and browser schemas they never called. Tool Mart now scopes headless workers to just two core knowledge services (Memory and Library) by default, keeping worker prompts lean and fast.

3. Making stateless cloud MCP servers work across every client

Five of my MCP services run as stateless Python functions on AWS Lambda behind API Gateway. Getting those exact same Lambda endpoints to work reliably across five local CLI harnesses and cloud-hosted agent environments required solving four separate protocol mismatches:

Wire Failure Mode	Root Cause	Contract Enforced Across All Cloud Services
Large payload truncation	Default Server-Sent Events (text/event-stream) responses hit 64 KB single-line buffer limits in several stdio-to-HTTP bridges.	Configure json_response=True so tool responses return standard application/json.
Read prompts blocking headless runs	Without explicit MCP annotations, strict clients treat every tool as potentially mutating and pause for manual approval.	Attach readOnlyHint=True and destructiveHint=False annotations to every read/search tool, and destructiveHint=True only on deletes.
Silent list truncation (only last item seen)	When a Python MCP tool returns a bare list, the server framework serializes each element as a separate TextContent block (content[0..N-1]). Clients that inspect only content[-1] silently drop N-1 results and see only the final item.	Wrap every list response in a single dict envelope ({"results": items, "count": len(items)}) with structured_output=False so the payload always serializes as one TextContent block (content[0]).
30–60s cold-start hangs on connect	Under the MCP Streamable HTTP spec, clients probe GET on the MCP endpoint at startup to open a standalone server-push SSE stream. On stateless AWS Lambda, an open GET hangs until the Lambda times out.	Intercept GET requests at the raw Lambda handler before the ASGI adapter and return HTTP 405 Method Not Allowed (Allow: POST) — the spec-compliant signal to proceed immediately with POST.

4. Empirical fleet routing (5-model inter-rater benchmark)

To decide which agent belongs in which role, I ran a blind inter-rater reliability benchmark across all five harnesses, scoring 10 real engineering task pairs across an 8-dimension rubric. All five models reached 100% pass/fail agreement on clear-cut cases and 0.900–0.923 pairwise Spearman rank correlation on overall scores.

The divergence appeared on adversarial “mixed-signal” commits — for example, a commit whose message claimed a full retry fix while the actual git diff only changed a timeout constant. The frontier reviewers (Claude, Codex, Gemini) penalized the unverified claim based on the diff (70–72/100), whereas the faster workhorses (Mistral, DeepSeek) gave partial credit to the prompt narrative (79–81/100).

That empirical split directly shaped the fleet hierarchy that Tool Mart configures: high-volume coding and research go to the faster workhorses, while surgical diff verification and adversarial audits stay with the frontier models.

---

The stack, annotated

Layer	Technology	Why it is here
Catalog & Compiler	Python 3.12 (zero third-party dependencies)	Compiles the declarative manifest into JSON and TOML configs, sanitizes skill frontmatter, and links directory junctions
OAuth Bridge	Stdio-to-HTTP proxy + OS file locking	Shares rotating OAuth 2.0 tokens safely across 5 concurrent agent harnesses without split-brain invalidation
Secrets Management	Infisical Vault CLI	Injects API keys at sync or launch time so zero secrets live in Git
Session Naming	Direct HTTPS small-model inference	Summarizes session intent after Turn 1 and every 3 turns in ~1.5–3s without spawning recursive child CLIs
Health Verification	doctor live probe suite + unit tests	Audits skill frontmatter, file encodings, OAuth expiry, and live JSON-RPC initialize + tools/list handshakes across all 14 servers