What Is an LLM?

A large language model (LLM) like the ones behind chat assistants is, at heart, a giant next-token predictor. A few terms make the whole thing click.

• Tokens. Text is chopped into pieces called tokens, roughly words or word-fragments. The model reads and writes tokens, not raw characters. A rough rule: one token is about four characters of English.
• Prediction. Given the tokens so far, the model outputs a probability for every possible next token. It picks one, appends it, and repeats. That loop is how it writes whole paragraphs.
• Prompt. The text you feed in. Everything the model knows about your request lives in the prompt, so how you phrase it matters a lot (next lessons).
• Temperature. A knob on the randomness of that pick. Near 0 the model almost always takes the most likely token (focused, repetitive). Higher values let it sample less likely tokens (more creative, less reliable).
• Hallucination. Because it predicts plausible text rather than looking facts up, an LLM can state wrong things with total confidence. Always verify facts it gives you.

Real tokenizers are subword-based and learned from data. We will not build that here, but we can build the simplest possible tokenizer (split on whitespace) to feel what counting tokens is like. Token counts are how usage and limits are measured.

def tokenize(text):
    return text.split()   # whitespace tokens

tokenize("the cat sat")   # ['the', 'cat', 'sat']

Splitting on whitespace is only a feel-for-it stand-in. Real tokenizers split into subword pieces, so a common budgeting rule of thumb is tokens ≈ len(text) / 4 characters, and production code reads the exact count from the usage field the API returns (or a dedicated count-tokens endpoint) rather than guessing.

Reasoning models

Everything above describes a model that emits its answer directly. A newer family, reasoning models, first spends a hidden budget of "thinking" tokens working through the problem step by step, then writes the final answer. That extra deliberation buys real accuracy on hard, multi-step tasks (maths, code, careful planning), and you often get a dial: a higher effort or thinking budget means more thinking tokens, more latency, and more cost. The engineering judgment is simply when it is worth it: reach for a reasoning model on the genuinely hard step, and use a fast, cheap model for the routine ones. You will put exactly that tradeoff to work later when you build a cost-aware cascade that routes easy requests to the cheap model and only escalates the hard ones.