Building with models: the parts that are not the model

The first thing to internalise: the model is the smallest part of a working AI feature. The interesting engineering is what surrounds it — what context you give it, how you check the output, and what happens when it is wrong.

Context is the product. The same model produces a useless answer and an excellent one depending entirely on what you put in the prompt. For anything beyond a toy, the context is assembled from your data — retrieved, ranked, trimmed to fit — and that assembly logic is where most of the value lives.

A MINIMUM VIABLE AI FEATURE — note what is NOT the model call

  1. retrieve      pull the few documents/snippets that could matter
                   (vector search, keyword search, or plain SQL — start simple)
  2. assemble      order them, trim to a budget, label them clearly
  3. call          model(prompt, context)        <- this line is the small part
  4. validate      is the output the right SHAPE? parseable? in range?
  5. fall back     what does the user see if it is wrong or the call fails?
  6. measure       log inputs/outputs so you can tell if quality moved

If steps 1,2,4,5,6 are missing, you have a demo, not a feature.

Evaluation is what separates a demo from a product. Without it, every change is a guess and every regression is discovered by a user. You do not need a research-grade harness to start: a fixed set of 20–50 real inputs with known-good outputs, run automatically on every change, catches most regressions.