InfoPlatform.ai BlogWhat Three Deliberately Hard Production Runs Found, Including In Our Own Tests
We ran three deliberately hard production runs against our own platform. Zero clean passes. Here is what was found, including in our own test instruments, and why publishing it is the point.
Run 62: the first honestly scored model failed
Until this run, a model earned TRAINED_AND_SERVED for answering at all. Any HTTP 200 counted. We rewrote the evaluation so a trained model has to be right, scored against held-out ground truth from its own training corpus. The first model scored under the honest standard failed. Nothing about the model changed; the measurement did. That gap had been invisible for every prior run.
Run 63: twenty-three file formats, one ever tested
The platform advertises 23 accepted file types. Every production run since the matrix was written had uploaded CSV. Pushing the same 120 pairs through eleven formats found csv/tsv/jsonl/parquet/xlsx landing ~88% extraction while docx extracted 17 of 120, txt 15, html 15, md 12, all four reporting status "ready" identically to the healthy formats. Silent degradation wearing a success status. The instrument needed three of its own bugs fixed first, the last inside the silent-failure detector itself.
Runs 64-65: adversarial data survives, and our instruments lied four times
Adversarially poisoned training data survived cleaning intact through the full pipeline. And separately: four measurement bugs were in our tooling, not the product, including a guard classifier scoring honest absence ("no check exists yet") the same as pretence. A wrong number we publish is our error regardless of which side of the fence produced it.
Why this is the marketing asset
The recursive self-improvement literature is blunt: without guardrails, an agent evaluating its own output can talk itself into believing a bad change is good. The countermeasure is instrumentation, rewards logged in real time, adaptations within guardrails, memories auditable. Our loop converts every finding into a machine-checked guard in the same run, refuses to relax thresholds to make red turn green, and publishes the whole log.
For the people deploying AI into enterprises, forward deployed engineers, and the teams evaluating them, this is the difference between a demo story and a delivery substrate. You can read every run, including the ugly ones, in our deploy report.
If you need models that answer for your client's controlled data, start here.
Build Your Custom AI Model
Upload your data and get a production-ready API endpoint. No ML expertise required.
Start Training FreeRelated Articles
Your Delivery Loop Is Already a Self-Improving Harness. The Research Says Make It Honest or It Collapses.
The 2026 recursive self-improvement literature names the exact mechanisms that keep an improving loop safe: checks with teeth, held-out keep-better gates, and auditable memory. Forward deployed engineers are being asked to build these for clients. Here is the pattern, from a loop that runs against its own production weekly.
Industry InsightsThe Forward Deployed Engineer's Problem: The Client's Data Cannot Leave
Palantir invented the FDE in 2005 because its customers could not disclose their work. Twenty years later the constraint is identical: the client cannot put proprietary data into multi-tenant SaaS, and the generic model keeps missing because it was never tuned on their corpus.
Industry InsightsAgents That Stop for a Human: Approval Gates for Determinative Work
An agent that can act on its own is the wrong shape for a determination that carries legal weight. Draw the workflow, put a human gate in it, and the run pauses in an approval inbox until a person decides. Verified in production, not on a roadmap.