Created: 2026-05-22 UTC
PASS 45 ADDENDUM
v8 corrected-cohort training and significance check
Continued research by training a new v8 model on corrected pooled cohorts, adding a boosted-tree
challenger in candidate selection, and running paired significance checks against v7, v6, and v4 on set11.
Artifacts Produced
scripts/python/train_ndd_model_v8_all10_corrected_holdout_set11.pyartifacts/models/ndd_panel_classifier_v8_all10_corrected.joblibartifacts/downloads/training/ndd_model_training_report_v8_all10_corrected_holdout_set11.jsonartifacts/downloads/training/set11_v8_vs_v7_significance_v1.jsonartifacts/downloads/training/set11_v8_vs_v6_significance_v1.jsonartifacts/downloads/training/set11_v8_vs_v4_significance_v1.jsonscripts/python/update_research_metrics_v2_pass45.py(metrics integration)
Outcome Snapshot
Paired Significance Summary (set11 @ 0.38)
| Comparison | Delta F1 (v8 - ref) | Delta Recall | Delta Accuracy | McNemar p | Interpretation |
|---|---|---|---|---|---|
| v8 vs v7 | 0.0000 | 0.0000 | 0.0000 | 1.0000 | No measurable difference on current holdout. |
| v8 vs v6 | 0.0000 | 0.0000 | 0.0000 | 1.0000 | No measurable difference on current holdout. |
| v8 vs v4 | +0.0762 | +0.1250 | +0.0625 | 1.0000 | Directional improvement vs v4; evidence remains low-power due small n. |
Research Interpretation
- The corrected-cohort v8 run did not outperform v7/v6 on set11 under current feature set and threshold policy.
- Model family expansion was executed through a boosted-tree challenger, but random forest remained best by CV ROC AUC.
- Result supports keeping v7/v8 as parallel research candidates while prioritizing feature expansion and broader external cohorts.
Immediate Next Steps
- Add structure-derived features (provider diversity, model-confidence, coverage flags) into the training feature row pipeline.
- Re-run v8 with same holdout protocol and paired significance to quantify marginal value of structure-derived features.
- Ship export-bundle frontend workflow so executive and researcher review uses identical evidence snapshots.
Generated 2026-05-22 06:05 UTC