What happens when a retinal grading model meets a different population and grading standard? Explore the recorded evidence, including the result that did not transfer.
Frozen baseline baseline-6e66147526. These are Kaggle mirrors of external datasets. No external threshold fitting is included below.
Two decisions, two sensitivities
Grade ≥ 2 uses the original ordinal boundary. The separate operating point was selected on APTOS out-of-fold data for 90% sensitivity. They must not be mixed.
Bars use a common 0–100% scale. Wilson 95% intervals assume independent eyes; patient clustering is unavailable.
Where the ordinal scale breaks
Recall by true severity grade at the original grade thresholds
Grade
Eyes
Recall
Recall differs from predicted class frequency. Grade 2 is the referral boundary; low recall here can conceal moderate disease.
02 / CONTROLLED COMPARISONS
Preprocessing did not solve transfer.
Fold models were each scored on the shared APTOS test set. The SD below describes model variation on that same holdout, not an independent outer-CV generalization interval. The ensemble result is separate.
Variant
Sweep
Folds
Mean test QWK ± SD
Ensemble QWK
What the evidence supports
Metadata predicts labels; external performance varies markedly; a controlled fine-tuning study supports a role for grading labels. These are distinct findings, with distinct limits.
What remains unknown
Prospective performance, patient-cluster uncertainty, robustness across additional sites and consistent grading standards. Duplicate-pair disagreement does not establish a formal accuracy ceiling.
03 / EXAMINATION-DISJOINT FOLLOW-UP · 26 OCTOBER 2026
Better on Messidor. A cost on APTOS.
A separate paired run starts from baseline-c8232af68337, a different baseline from the historical tables above. Five models per arm were fine-tuned on identical images. Only training labels differed; adjudicated validation targets were common to both arms. No supplied examination crosses partitions.
New Messidor test split · same original APTOS cuts for all arms
Arm
Referable AUC
QWK
Sensitivity
Specificity
False negatives
Report the gain and the loss.
This is a previously inspected cohort and a third-party examination pairing transcription. Longitudinal patient identity is unverified. Grade-4 recall remains low; 24 test eyes support that estimate.
A separate completed GPU sweep has been audited from all five saved prediction files. A retrieved examination-pair table reveals seven examinations crossing the historical fine-tuning split, including six test/development overlaps. The completed follow-up repairs examination separation and quantifies both a Messidor gain and an APTOS retention loss. Both studies remain exploratory; longitudinal patient identity is unverified. The report documents repeated-holdout limitations, label-noise assumptions and the remaining data requirements.
This page is generated offline from six committed reports and audits. Source hashes let reviewers detect changed inputs. They do not independently verify the original training run.