Output Guards & Trace Aggregation

A model's reply is untrusted until you check it. Two output guards cover most needs. An allow-list guard insists the output is exactly one of a known set, which is how you make a classifier reliable: if the model is supposed to answer positive, negative, or neutral, anything else is rejected. A deny-list guard flags any output containing a banned term (a leaked secret, a slur, a competitor name) regardless of surrounding text.

Production systems also need traceability: every call recorded with its input, output, token count, latency, and cost, so you can answer "what did this cost and how slow was it?" after the fact. You aggregate those records into a summary: total cost, and latency percentiles. The p95 latency (the value 95% of calls came in under) is the number on-call engineers actually watch, because averages hide the slow tail.

import numpy as np
p95 = np.percentile(latencies, 95)   # the slow-tail number, not the mean

Build four functions. (1) validate_output(text, allowed_labels) -> True only if text.strip() is in allowed_labels. (2) contains_banned(text, banned) -> True if any banned term appears in text case-insensitively. (3) build_trace(input_text, output_text, tokens, latency_ms, cost) -> a dict with exactly those five keys. (4) aggregate_traces(traces) -> build a pandas.DataFrame from the trace dicts and return a dict with n, total_cost (rounded to 6 dp), p50_latency, and p95_latency computed with numpy.percentile.