Dataset and gate
A 100-case blind run with two separate release requirements
The independent historical confirmation manifest contained 100 blind cases. Ninety-nine completed technically and one returned an error, giving 99% technical coverage. The pre-existing numerical gate required at least 80% precision and at least 80% recall.
The integrity reconstruction also required explicit proof that each evaluated capture was complete under its capture contract. That second requirement is not cosmetic: a model can appear to meet a threshold while relying on partial or unverifiable inputs.
Decision matrix
The 99 technically successful cases produced this confusion matrix
Counts are reproduced from the frozen aggregate artifact. They describe agreement with the blind benchmark labels, not website quality or generated-code share.
| Outcome | Count | Plain-language meaning | Contribution |
|---|---|---|---|
| True positive | 42 | Positive benchmark case predicted positive | Supports precision and recall |
| False positive | 9 | Negative benchmark case predicted positive | Reduces precision and specificity |
| True negative | 41 | Negative benchmark case predicted negative | Supports specificity and accuracy |
| False negative | 7 | Positive benchmark case predicted negative | Reduces recall |
Decision matrix
The numerical metric gate passed
Both primary thresholds exceeded their predefined minimum. The overall release gate still failed because numeric performance was only one required condition.
| Metric | Observed | Required threshold | Threshold result |
|---|---|---|---|
| Precision | 82.4% | At least 80% | Passed |
| Recall | 85.7% | At least 80% | Passed |
| Specificity | 82.0% | Not a primary gate | Reported |
| Accuracy | 83.8% | Not a primary gate | Reported |
| F1 | 84.0% | Not a primary gate | Reported |
Decision matrix
The evidence-integrity gate failed
The legacy result rows did not persist explicit capture-completeness state, so completeness could not be reconstructed after the fact.
| Integrity check | Observed | Required | Decision |
|---|---|---|---|
| Explicitly complete rows | 0 | Completeness evidence for evaluated captures | Failed |
| Unverifiable legacy rows | 99 | No unverifiable evaluated rows | Failed |
| Metric thresholds | Both passed | Precision and recall at least 80% | Passed |
| Overall release gate | Not passed | Metrics and capture integrity | Rejected |
Repeatable process
What the failed gate changed in the research process
The practical lesson is to design integrity evidence into capture and evaluation—not to reconstruct it after a promising number appears.
- 01
Persist capture completeness
Record whether every required surface and artifact completed under the declared contract.
- 02
Separate technical, metric and integrity gates
Require each gate explicitly and prevent one successful metric from overriding another failed condition.
- 03
Hash immutable inputs and outputs
Bind the manifest, raw results and evaluation artifact to stable hashes before interpretation.
- 04
Publish the negative decision
Report the observed metrics while preserving the fact that the confirmation was not accepted.
Interpretation
Good-looking metrics cannot repair missing provenance
Rejecting the run does not mean its 99 predictions never happened. It means the evidence package was insufficient for the stronger claim that the model had passed confirmation under the complete protocol. The appropriate status preserves both facts: the recorded metrics crossed 80%, and the release gate did not pass.
This is the standard VibeFootprint applies to public claims as well. A score should not imply more than the retained evidence can support, and missing provenance should lower the claim—not be filled with a confident assumption.
Plain answers
Questions about the failed confirmation gate
Did the model pass the 80/80 metric threshold?+
Yes. The reconstruction recorded 82.4% precision and 85.7% recall. The overall release gate still failed because verified capture completeness was also mandatory.
Why not accept the metrics and add the missing evidence later?+
Completeness evidence describes the inputs used for those exact predictions. It cannot be reliably recreated after the historical captures when the required state was not persisted.
Are these current VibeFootprint production metrics?+
No. They belong to a frozen historical v0.4 confirmation reconstruction evaluated on 15 August 2026 and should not be used as a current production-performance claim.