model routing

advancedspreading

A solution that automatically routes every question to the AI model expected to answer it well enough, yet as quickly and cheaply as possible.

In plain terms

Imagine the general practitioner's office: the assistant does not send every patient with a cold to the professor. Simple cases are handled on-site; complex ones are referred to a specialist. Model routing does the same with artificial intelligence: it decides which model is needed for which question.

This is necessary because AI models differ greatly. The largest are mostly smarter but slower and more expensive; the smaller ones are fast and cheap but often fail on difficult tasks. If every query goes to the strongest model, it is like taking a taxi to the corner store.

The router works in the background, and you often do not even notice it. A well-configured system handles most questions with a cheap model and sends only the truly difficult ones to the expensive one. Thus, in favorable cases, the cost can drop to a fraction while quality hardly declines.

How it works

Your question first reaches not a model but a router. This is a small, fast program or itself a tiny AI model.
The router assesses the task: how difficult it is, what topic it covers (for example, coding, translation, calculation), how long it is, and whether deep thinking is required.
Based on pre-set rules or patterns learned from past experience, it selects the appropriate model. It weighs the price, speed, and expected quality to do so.
The selected model answers the question. Some systems also check the result, and if it is weak, they forward the question to a stronger model.
If a model is unavailable or overloaded, the router switches to a backup, so the service does not go down.

An everyday example

You ask the chatbot how many dekagrams are in a pound, and the answer comes immediately. Then you ask about the pitfalls of a complex contract, and the system noticeably "thinks" longer. In such cases, the router likely decided in the background: it assigned the first question to a fast, cheap model, and the second to a stronger, more thorough model.

Who is affected

entrepreneur, executiveThe cost of AI usage largely depends on whether a company uses the most expensive model for every task or intelligently distributes the work.

developerYou must design which subtask the application or agent performs with which model, and what happens if one fails. → Code

everyoneWhen using a chatbot, you often do not know which model is responding, yet this influences the quality of the answer.

A common misconception

MythWhen using an AI service, the same model always responds, specifically the best one.

In realityMany services switch between multiple models in the background, assigning simple-looking questions to smaller models. That is why the same service may appear brilliant at times and superficial at others.

Limits and risks

The router can make mistakes: if it treats a difficult question as easy, you receive a confident-sounding but worse answer from a weaker model.
Often you cannot see which model responded, making it hard to judge how much to trust the result and difficult to reproduce the same outcome.
If your query can reach models from multiple companies, your data may circulate in multiple places; for sensitive information, it is worth investigating this.
The provider's interest is saving costs, yours is quality. These two do not always align.

Similar, but not the same

load balancing – Load balancing distributes traffic among identical machines so none is overloaded, whereas model routing selects among models with different capabilities based on the task's nature.
AI gateway – An AI gateway is the broader entry point that also manages access, logging, and the cost budget; routing is just one of its functions.
agent orchestration – Orchestration coordinates the collaboration of multiple agents and the sequence of tasks, while routing only decides which model receives a specific request.

Related concepts

Broader

AI infrastructure, AI gateway, cost optimization

Narrower

model cascade, switching to a backup model, semantic routing

On the edge

large language model, small language model, inference, token, agent orchestration

A brief history

The idea gained significant attention in 2023 with a Stanford University study titled FrugalGPT. It demonstrated that by routing queries first to a cheaper model and only escalating to a more expensive one when necessary, costs could be reduced to a fraction for the tasks examined while maintaining similar quality. In 2024, the RouteLLM open framework, developed primarily by researchers at the University of California, Berkeley, made trainable routers widely testable. In summer 2025, OpenAI introduced a router embedded in ChatGPT alongside the launch of GPT-5, which chooses between a fast model and a more deliberative one.

Where things stand

By the end of 2026, it will be a built-in feature in a significant portion of major chat services and corporate AI gateways, and a common element in designing multi-model agents. However, transparency of decisions and consistency of quality remain weak points.

Status: 2026 November

Also known as

model routing · model routing · model governance · LLM routing · LLM router · model router · modellrouter · intelligent routing

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.