Building a Model-Router Agent

Part of a series on running AI workflows across model providers — this one is the actual build.

With three million models on the Hub and sixteen-plus backing infrastructure providers behind Inference Providers, "which model should answer this" stops being a one-time setup decision and starts being something worth deciding per request. This post builds that: an agent that looks at an incoming task, decides whether it's cheap-and-simple or genuinely hard, and routes to a different Hugging Face model accordingly — using DNotifier's real defineAgent and Workflow primitives.

The shape of it

The routing decision is itself a small, cheap model call — not a guess encoded in application logic.

A request comes in. A lightweight classifier agent — running on a small, fast model — decides whether the task is routine or complex. Routine tasks go to a smaller, cheaper model. Complex tasks route to a larger, more capable one. The routing decision itself is cheap, because it's a narrow classification, not a full attempt at the task.

Step 1: Define the routing classifier

import { DNotifier } from "@dnotifier-realtime/dnotifier";

const routerAgent = DNotifier.defineAgent({
  name: "complexity-router",
  model: "huggingface/meta-llama/Llama-3.1-8B-Instruct:fastest",
  async run(ctx) {
    const classification = await ctx.sendAI({
      message: {
        text: `Classify this request as "simple" or "complex".
        Simple: factual lookups, short rewrites, basic classification.
        Complex: multi-step reasoning, long-document analysis, nuanced judgment calls.
        Respond with exactly one word.

        Request: "${ctx.input.text}"`,
      },
    });
    ctx.state.complexity = classification.text.trim().toLowerCase();
    return ctx.state.complexity;
  },
});

An 8-billion-parameter model, routed to the fastest available backing provider, is plenty for a one-word classification task — there's no reason to spend more here.

Step 2: Define the two answer paths

const simpleAnswerAgent = DNotifier.defineAgent({
  name: "simple-answer-agent",
  model: "huggingface/meta-llama/Llama-3.1-8B-Instruct:fastest",
  async run(ctx) {
    const answer = await ctx.sendAI({ message: { text: ctx.input.text } });
    ctx.state.answer = answer.text;
    return answer.text;
  },
});

const complexAnswerAgent = DNotifier.defineAgent({
  name: "complex-answer-agent",
  model: "huggingface/openai/gpt-oss-120b:cheapest",
  async run(ctx) {
    const answer = await ctx.sendAI({ message: { text: ctx.input.text } });
    ctx.state.answer = answer.text;
    return answer.text;
  },
});

Note the different routing hints — :fastest for the small model where latency matters more than shaving pennies, :cheapest for the larger model where the cost difference between backing providers is actually worth optimizing for on a more expensive call.

Step 3: Wire it into a workflow

const routingWorkflow = new DNotifier.Workflow({
  name: "complexity-based-model-router",
  description: "Classifies request complexity and routes to an appropriately sized model",
  observability: true,
  async entry(ctx) {
    await ctx.agents.run(routerAgent, { text: ctx.input.text });

    if (ctx.state.complexity === "complex") {
      await ctx.agents.run(complexAnswerAgent, { text: ctx.input.text });
    } else {
      await ctx.agents.run(simpleAnswerAgent, { text: ctx.input.text });
    }

    return { complexity: ctx.state.complexity, answer: ctx.state.answer };
  },
});

routingWorkflow.registerAgents([routerAgent, simpleAnswerAgent, complexAnswerAgent]);

Step 4: Run it

const notifier = new DNotifier({
  appId: process.env.DNOTIFIER_APP_ID,
  secret: process.env.DNOTIFIER_SECRET,
  userId: "routing-system",
  transport: "ws",
  WebSocketImpl: WebSocket,
});

await notifier.connect();

const outcome = await notifier.runWorkflow(routingWorkflow, {
  text: "What year did the Berlin Wall fall?",
});

console.log(outcome.complexity, "→", outcome.answer);

A simple factual question routes to the small model and comes back fast and cheap. Feed the same workflow a genuinely multi-step analytical question, and the router sends it to the larger model instead — automatically, without a human deciding case by case.

Why this is worth the extra step

Most real traffic is simpler than teams assume. A huge share of everyday requests — short factual lookups, basic rewrites, simple classification — don't need a large model's reasoning depth. Routing them to a smaller model isn't a compromise; it's the correctly sized tool for the job.

The routing cost is small relative to what it saves. An 8-billion-parameter classification call is cheap and fast enough that running it on every request, even ones that end up routed to the expensive model anyway, is easily worth it once volume is real.

It's provider-flexible by construction. Because both the router and the two answer agents are just defineAgent calls with a model field, nothing here is locked to Hugging Face specifically — the same pattern works routing between Claude Sonnet and Claude Opus, or between a Hugging Face model and a completely different provider, without changing the workflow's shape.

A real use case: a customer-facing FAQ bot with occasional hard questions

Most support-bot traffic really is routine — "what are your hours," "how do I reset my password," things with a short, factual answer. A small minority of questions are genuinely nuanced — comparing plan tiers with unusual account history, or a multi-part question mixing billing and technical topics. Running everything through the larger model wastes money on the ninety-plus percent that didn't need it; running everything through the small model produces visibly worse answers on the genuinely hard minority. The router pattern above handles both cases correctly without a human pre-sorting tickets, and the classification itself — logged through observability: true — gives you a real, ongoing signal for whether your complexity threshold is actually calibrated right.

What to refine before this is production-ready

The two-bucket "simple/complex" split here is a starting point, not a finished system. A real deployment would want to validate the router's classifications against actual outcomes over time — are "simple"-routed answers actually holding up, or is the router being too aggressive about sending things downmarket — and adjust the classifier's prompt or add a third tier accordingly. None of that changes the core pattern; it's tuning on top of it.

Frequently asked questions

Does the router agent add noticeable latency?

A small amount — one extra fast model call before the real answer — but for anything beyond the most latency-sensitive interactions, it's a worthwhile tradeoff for the cost savings on high-volume routine traffic.

Can I route based on something other than complexity — like language or topic?

Yes — the router agent's classification prompt defines what it's sorting by; complexity is one useful axis, but language, topic, or urgency are equally valid things to route on with the same pattern.

What happens if the router misclassifies a request?

It's a soft failure, not a hard one — a complex question routed to the simple model gets a weaker answer, not an error. Monitoring the classification and answer quality together over time, per the note above, is how you catch and correct systematic misclassification.

Could I add a third tier for extremely hard requests?

Yes — the pattern generalizes to any number of tiers; you'd add more classification categories and more corresponding answer agents, each pointed at an appropriately sized model.

Is this pattern specific to Hugging Face models?

No — the routing logic is provider-agnostic. This example uses two different Hugging Face model sizes because that's a natural fit for Hugging Face's broad size range, but the same workflow shape works routing between any two models on any connected provider.

The Bottom Line

Next in this series: with three million models to choose from, a practical framework for actually picking one.

Read part one: Hugging Face on DNotifier: Setup Guide. Read part seven: Picking an Open Model: A Guide. Explore dnotifier.com.