For the last six months at Disrekt, we’ve been building Gnoxis: The AI Executive.
We started the way many teams do: pick a strong model, wire it up, ship something.
Then we added another model that was cheaper.
Then one with better reasoning.
Then a vision model.
Then a safer option for particular workflows.
Pretty soon, we weren’t “just” building features anymore, we were juggling:
- Different APIs and SDKs
- Different auth and rate limits
- Different error shapes and quirks
- Different dashboards for usage and cost
The big question quietly shifted from “will this improve the experience?” to “do we have the energy to integrate and maintain yet another provider?” Gnoxis was the mirror. The real story was our AI stack turning into a messy orchestra.
The problem: model sprawl, glue code, and inequality
If you’re doing anything serious with AI right now, this probably sounds familiar:
- Multiple LLM providers in production
- “Historical reasons” dictating which feature uses which model
- Guardrails, retries, and prompts copy‑pasted across integrations
- Cost/latency scattered across logs, dashboards, and spreadsheets
- One provider hiccups, multiple surfaces in your product ripple
On paper, you “just call a model.”
In reality, you’re maintaining a tangle of keys, configs, feature flags, and assumptions.
The hidden cost isn’t only tokens, it’s engineering time and optionality. And fragmentation quietly raises the barrier to entry: if only teams with platform budgets can responsibly run multi‑model stacks, we’re deciding who gets to build the next wave of AI products.That’s not a model problem. That’s a fragmentation problem.
The future is many models – one mental model
We don’t believe in one perfect “everything” model.
We see a healthy stack that’s multi‑model by default:
- Frontier + open + domain models, mixed by capability, price, and latency
- Privacy‑sensitive work routed to regional or private backends
- New contenders emerging every few weeks, often worth trying
The difference between “chaos” and “composable” is the mental model you use to work with the mess. Ideally you can express, centrally:
- “Use the cheaper model unless quality drops below this threshold.”
- “If provider X is down, fail over to provider Y.”
- “Anything with PII stays on these regionalized endpoints only.”
“For this feature, A/B test across these two models behind the same interface.”
And change those rules without rewriting half your app.
It’s high time we made models and compute feel more accessible – not just in price, but in friction. If every experiment costs a week of plumbing, a lot of good ideas never leave the whiteboard.
What the market is teaching us (so you don’t have to learn it the hard way)
1) Prices really do differ by orders of magnitude, and shift.
As of this writing, representative rates per 1M tokens: Claude Sonnet 4.5 is $3 in / $15 out, and Google’s Gemini 2.5 Pro is $1.25 in / $10 out (≤200k tokens), with lighter “Flash” tiers even cheaper; OpenAI’s GPT‑4o sits around $2.50 in / $10 out. That spread is the whole case for smart routing. (Claude Console)
2) Routing isn’t hand‑wavey, it’s measurable.
Academic and open‑source work shows query‑aware routing can cut cost 40–98% while matching most of a top model’s quality, by sending easy queries to cheaper models and escalating hard ones only when needed. (Stanford’s FrugalGPT; Microsoft/Berkeley’s Hybrid LLM; LMSYS’s RouteLLM.) (arXiv)
3) Leaderboards churn fast.
Crowd‑preference and eval leaderboards are updated constantly (Arena ELO, MT‑Bench, MMLU‑Pro, GPQA), which means “best for X” is a moving target. Your stack should assume churn, not fight it. (LMArena)
4) Reliability is real‑world, not theoretical.
Even first‑party status pages record incidents. Google’s Vertex Gemini had an elevated‑error incident (Jan 10–13, 2025). OpenAI reported a partial Responses API outage (Oct 10, 2025). And infra providers like Cloudflare can wobble and take half the internet (including ChatGPT) with them. Plan for failover. (Google Cloud Status)5) Data residency and PII guardrails are now table stakes.
OpenAI introduced EU data‑residency for API/Enterprise with broader regional options; Vertex AI documents in‑region processing when you select regional endpoints; AWS Bedrock offers PII detection/masking guardrails. If you want small teams to ship responsibly, these controls can’t be “choose your own adventure.” (OpenAI)
A conductor, not a platform: what we actually built
We didn’t set out to build a giant platform. We built the thin layer we wished we had:
- One interface your apps talk to, instead of five
- One policy surface for routing, failover, and safety
- One view of usage, latency, and cost by model/app/workspace
- One normalization of requests/responses across providers
Under the hood it:
- Normalizes API calls and responses
- Handles keys, retries, timeouts, backoff, and health checks
- Tracks tokens, cost, and p95 latency by model and feature
Applies routing and safety rules consistently.
So… what does “model curation” actually mean?
When we say Symphony handles model curation, we mean:
- A living model registry: price (in/out/cache), latency percentiles, rate‑limit envelopes, context limits, residency options, and safety tooling per provider. Pricing and capabilities are pulled from first‑party docs where possible. (Claude Console)
- Continuous evaluation: we benchmark candidate models using a small, rotating task‑shaped eval set plus public signals (Arena ELO, MT‑Bench, MMLU‑Pro, GPQA), with LLM‑as‑judge where appropriate and bias checks where necessary. The goal isn’t a trophy, it’s a curated shortlist per use case you can safely start from. (arXiv)
- Policy‑backed routing: you declare guardrails and intent (cost caps, quality thresholds, residency/PII constraints), and Symphony chooses the model/deployment accordingly, with automatic failover when a provider degrades or rate limits spike. (If you’ve built with LiteLLM/OpenRouter/Portkey, you know pieces of this pattern; Symphony is the opinionated “conductor” for the whole orchestra.) (GitHub)
- Observability hooks by default: every call is traced with usage and cost metadata so you can answer “who spent what, where, and why?” (We interoperate with open‑source tools like Phoenix or Langfuse if you already have them.) (Arize AI)
- Residency & PII controls: policy tags can force regionalized endpoints and PII redaction, with provider‑specific knobs (e.g., Vertex regional processing, Bedrock PII filters, OpenAI regional projects). (Google Cloud Documentation)
What changes in practice
For product teams
You think in user outcomes, not vendor quirks:
- “Draft the email and escalate if quality < 0.85.”
- “Keep anything with PII on EU endpoints.”
- “A/B these two models behind one interface.”
For infra/platform
You get one control plane for governance, spend, reliability:
- Set per‑workspace budgets and rate limits
- Toggle providers or regions centrally
- Watch p50/p95/p99 latency and cost per feature
For small teams
This is the accessibility piece we care about most: adding a new model or spinning up an experiment should feel like changing a config, not kicking off a new integration project. External work suggests the payoff is real, routing/cascades routinely save 40–98% without noticeable quality loss. Our design goal is to make those wins trivial to try, not a month of glue code. (arXiv)
Why this matters for democratization
If responsible multi‑model stacks require dedicated platform teams, we’re narrowing who gets to compete. A thin, policy‑driven conductor lowers the friction tax, not just the price tag, so solo builders and lean startups can experiment like they had an infra team. That’s the point.
What comes next (and how to kick the tires)
I’m Rithesh, founder of Disrekt.
The small internal layer we built for Gnoxis became the most stable piece of our stack. Internally we called it the conductor, the thing that lets models play together while the rest of the system doesn’t have to care. Now we’re opening it up:
Disrekt Symphony is a Universal LLM interface that handles model curation for you, while you orchestrate your AI through a single, seamless experience.
A few “batteries‑included” promises:
- Policy‑first: cost caps, failover, residency, and safety as config, not code
- Eval‑informed: curated shortlists per task, refreshed as the landscape shifts
- Vendor‑agnostic: swap providers without rewriting product surfaces
We’re early, but the core has been battle‑tested in our own stack. If your AI already feels like an unruly orchestra, try Symphony, push it hard, and tell us what feels rough or magical.
To thank early adopters, we’ve set up a limited‑time, limited‑seats offer of up to 20% off LLM usage credits: https://symphony.disrekt.com/pricing