Note
Evaluation is system design
Evaluation becomes more useful when it is designed with the workflow, not added after the model already appears to work.
A model demo answers a narrow question: can the system produce a plausible result on this example?
Production engineering needs different questions. What failure matters? What evidence would reveal it? Can the same behaviour be reproduced? Which change is allowed to make one metric better by making another worse?
Those questions shape architecture.
If a result needs independent verification, the workflow may need separate producer and reviewer roles. If a decision needs to survive a restart, the relevant evidence cannot live only in conversation history. If a release depends on retaining quality while reducing cost, the evaluator needs to capture both rather than reporting a single aggregate score.
That is why I find it more useful to treat evaluation as part of the system contract. The evaluator is not just a scoreboard at the end; it influences what state is recorded, what boundaries exist, and what claims the implementation is allowed to make.