You are two sips into your Monday-morning coffee when a message lands: “Why are the health plan’s completeness numbers lower than ours?”
You open the file. The validation checks ran, the file passed, Nothing in the dashboard was red.
Then you realize: the vendor changed its export format last week.
The validation gate you created ran the old feed. The feed changed but the checks still ran on the old one.The health plan trusted the numbers you sent. That trust was misplaced.
The missing data does not stay inside the company. It becomes a compliance question, a quality-measure question, sometimes a contractual one, the moment it reaches a health plan or a regulator.
A 2025 study in Health Services Research measured this exact failure pattern in Medicaid claims data from 2001 to 2019. Completeness rose from 19.2% to 92.3% over that period. It dropped during the years states were migrating to a new reporting system. It recovered once the migration settled.
The study points to a specific vulnerability: an upstream change can reduce completeness without stopping the file from arriving at all. A changed export format can drop records or fields while the file itself still looks intact, and checks built for the old feed often keep passing the new one. The file arrived intact. The data inside it did not.
Teams usually know when these changes are coming. The validation thresholds stay fixed anyway.
Picture a validation gate as an admitting desk checking insurance eligibility before a patient sees a provider. A team builds that desk once, trains it on one set of checks, and leaves it running the same way for years. The checks underneath are not exotic: identity confirmed, coverage confirmed active, the plan and provider confirmed linked.
How a data validation gate works: staging, four checks, and the pass/fail decision.
The hard part is remembering to tighten the desk’s checks right before a known disruption instead of leaving them on autopilot. A tolerance band tuned for a stable feed is exactly wrong during the month a vendor changes its export format — the moment a shortfall is most likely and a static gate is least likely to catch it. Nobody revisited the thresholds going into the change.
A validation gate is not a project with an end date. It needs to work like a calendar: tightening before every known migration, vendor swap, or reporting-format change, then relaxing once the new feed proves stable.
A short file can clear every check and still land short. It can reach downstream reporting untouched and sit there for weeks before anyone notices the numbers look off. The same short file can also get caught the same day, flagged before it reaches production because someone tightened the checks at exactly the right moment.
The same short file, with and without a validation gate catching it.
Before a migration or vendor cutover, tighten the gate on purpose. Compare the new feed against the old one directly. Narrow the tolerance for missing records for the first few cycles. Review exceptions by hand before the file reaches production, at least until the new feed proves itself.
When the feed changes, the validation gate has to change with it. A file that goes out short does not usually announce itself. The report was clean, the checks passed, and the number the health plan is now relying on was never complete.
Source cited: Samples H, Lloyd K, Ryali R, et al. Completeness and quality of comprehensive managed care data compared with fee-for-service data in national Medicaid claims from 2001 to 2019. Health Serv Res. 2025;60(3):e14429. doi:10.1111/1475-6773.14429.
