Guide · Dubai · MENA

LLM evals, explained for founders: know it works before you ship

Every founder building with AI reaches the same moment. The demo works. The model answers well in testing. Then real users arrive with real, messy inputs, and the feature starts getting things wrong in ways nobody saw coming. An eval is how you catch that before launch instead of after.

Across engagements in Dubai and MENA I keep meeting founders who can describe their product in detail but cannot answer one basic question: how do you know the model is actually right? Evals answer it. They are not a nice-to-have for teams with spare engineering time. They are the difference between shipping something you can defend and shipping something you are hoping works.

What an eval is

An eval, short for evaluation, is a test set for your AI feature. It is a list of real or realistic inputs, each paired with the answer you would consider correct or acceptable, run against your model before anyone outside the team sees the output.

Think of it as the same discipline as unit tests, applied to a system that does not give the same answer twice. You cannot assert exact string equality on a model's output the way you would on a function's return value. But you can score outputs against criteria: did it extract the right fields, did it follow the required format, did it avoid the specific mistakes you already know matter.

Three things make an eval useful rather than decorative:

• It uses inputs your users will actually send, not the clean examples from your own head.
• It has a scoring method, even a simple pass or fail rubric, so results are comparable across runs.
• It runs before every ship, not once at the start and never again.

Why vibes fail in production

"I tried it a few times and it looked good" is the most common quality process I see in early-stage AI features, and it is the one that fails first. It fails for a specific, predictable reason: the person testing writes clean, well-formed inputs, because that is how humans phrase things when they know what a system expects. Real users do not know that. They send typos, half-finished thoughts, a different language mixed into an English form, sarcasm, and requests the product was never scoped to handle.

I have watched this exact pattern sink a launch. A team demos an AI feature internally, everyone nods, it ships, and within a week the support queue fills with the exact failure modes an eval set would have surfaced in an afternoon. The model was not the problem. The absence of a test was.

There is a second reason vibes fail: models drift. A provider updates a model version, a prompt gets a small tweak for an unrelated reason, and quality shifts in ways nobody notices until a user complains. Without a number you are tracking over time, you have no way to tell a genuine regression from a bad day.

Building a first eval set

You do not need hundreds of examples or a data science background to start. You need somewhere between 30 and 50 well-chosen cases and an hour to write them down.

Pull from three sources:

1. Real historical data, if you have it. Past support tickets, past documents, past transactions. This is the closest thing to production truth you can get before launch.
2. Deliberately awkward inputs, the ones a demo never includes. Typos, missing fields, an unexpected language, an angry or sarcastic tone, an input that is technically off-topic.
3. Known edge cases your domain already has. Ask whoever knows the domain best what breaks people, and write those cases down explicitly.

For each input, write the answer you would accept, or the criteria an answer must meet if there is no single correct string. Run your model against the full set, score every result, and read the failures yourself rather than trusting a summary percentage. The failures are where you learn what to fix.

A regional marketplace I worked with built its first eval set from exactly this mix before shipping an AI-assisted loan reconciliation workflow. The eval set surfaced a handful of formatting failures on messy historical records that never showed up in the clean demo data. Fixing those before launch, rather than after, is what let the feature run reconciliation end to end without a human catching errors downstream.

Guardrails, for what the eval set does not catch

An eval tells you how the model performs on the inputs you thought to test. It cannot cover every input a real user will eventually send. Guardrails are what hold in the gap:

• Input validation before the model runs, to reject the obviously malformed rather than letting the model guess.
• Output checks after the model responds, confirming the answer matches the expected structure or falls inside a sane range.
• A confidence or uncertainty signal, even a rough one, so weak answers route to a human instead of straight to the user.
• Logging on every call, so when something does go wrong, you can see exactly what the model saw and said, not just that it failed.

None of this needs to be sophisticated on day one. A basic rule that catches the worst quarter of failures is worth more than a perfect eval score sitting on top of no safety net at all.

A simple starting workflow

Put together, a workable eval loop looks like this:

1. Write 30 to 50 real and awkward test cases with the answer or criteria you expect.
2. Run the model against the set, score every result, and read the failures directly.
3. Fix what you can, and add guardrails for what you cannot fully fix.
4. Re-run the same eval set before every meaningful prompt or model change.
5. Grow the set as production reveals new failure patterns, so it keeps pace with real usage instead of staying frozen at launch.

This is not a one-off exercise. Treat the eval set as a living document that grows every time production surfaces a case you had not thought of.

FAQ

What are LLM evals, in one sentence? A test set of real or realistic inputs paired with expected answers, run against your model to check quality before real users see the output.

How many examples do I need for a first eval set? Enough to cover the common case and the edge cases you already know about, often 30 to 50 for a narrow first version. Grow it as usage teaches you more.

Do I need a data science team to run evals? No. Writing an eval set and scoring outputs is a product and domain-knowledge task, not a machine learning task. Most early-stage teams can build a first version in an afternoon.

How often should I re-run the eval set? Before every ship that touches the prompt or the model, and on a schedule after launch, since providers update models and quality can drift without any change on your side.

What is the difference between an eval and a guardrail? An eval measures quality before you ship. A guardrail catches problems live in production, for the inputs no eval set fully anticipated. You need both.

Book a call

If you are scoping an AI feature and want a second opinion on how to test it before real users do, book an intro call. For weekly writing on AI and product from zero, read the Scalable newsletter.