Syllabus Lesson 221 of 239 · Project: Production AI Gateway
Project: Production AI Gateway

Route to a Handler + Fallback

The classifier told you what the request wants. Now the gateway has to do something with it: hand it to the right piece of code. That is routing, and in production it is never as simple as one intent maps to one function. Services go down. A weather provider times out. Your job is to make the gateway resilient, so a single flaky dependency does not take the whole request with it.

The pattern is a fallback chain: each intent maps to an ordered list of handlers to try. The first one is the preferred handler; if it fails, you slide down to the next, and the last link is always a generic fallback that cannot fail. Unknown intents skip straight to that fallback. A hiring manager reads this as "understands graceful degradation," which is most of what reliability work actually is.

You will build two functions over a routing table like this:

ROUTES = {
    "weather": ["weather_api", "weather_cache", "fallback"],
    "account": ["account_service", "fallback"],
    "billing": ["billing_service", "fallback"],
}
  • route(intent) -> the name of the primary handler for an intent (the first link in its chain). An intent that is not in the table routes to "fallback".
  • handle(request, handlers) -> the request is a dict like {"intent": ..., "text": ...} and handlers maps a name to a callable. Walk the intent's chain in order, try each handler, and return the result of the first one that does not raise. Report which handler won, e.g. {"handler": name, "result": ...}.

The heart of it is a loop with a try/except so a crashing handler is caught and you keep going:

for name in chain_for(intent):
    fn = handlers.get(name)
    if fn is None:
        continue
    try:
        result = fn(request)
        return {"handler": name, "result": result}
    except Exception:
        continue   # this handler is down, slide to the next link

Because the chain always ends in fallback, a request is only unservable if even the fallback is missing or broken. Press Run to route a few requests, then watch one with a deliberately broken primary handler fall through to the next link.

Your turn

Write route(intent) returning the primary handler name for an intent (the first entry of its fallback chain; an unknown intent -> "fallback"). Write handle(request, handlers) where request is {"intent", "text"} and handlers maps a name to a callable: walk the intent's chain, try each handler in order, and return {"handler": name, "result": ...} for the first that does not raise. Unknown intents and failing primaries must fall back down the chain (which always ends in fallback).

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output