NEXUSOPEN RESEARCH / 01Source repository ↗

COMPUTER VISION · DISTRIBUTION SHIFT

A high score.
An external failure.

What happens when a retinal grading model meets a different population and grading standard? Explore the recorded evidence, including the result that did not transfer.

EfficientNet-B0Five-fold ensembleTwo external cohorts

01 / EXTERNAL VALIDITY

Same model. Different outcomes.

Frozen baseline baseline-6e66147526. These are Kaggle mirrors of external datasets. No external threshold fitting is included below.

Two decisions, two sensitivities

Grade ≥ 2 uses the original ordinal boundary. The separate operating point was selected on APTOS out-of-fold data for 90% sensitivity. They must not be mixed.

Bars use a common 0–100% scale. Wilson 95% intervals assume independent eyes; patient clustering is unavailable.

Where the ordinal scale breaks

Recall by true severity grade at the original grade thresholds
GradeEyesRecall

Recall differs from predicted class frequency. Grade 2 is the referral boundary; low recall here can conceal moderate disease.

02 / CONTROLLED COMPARISONS

Preprocessing did not solve transfer.

Fold models were each scored on the shared APTOS test set. The SD below describes model variation on that same holdout, not an independent outer-CV generalization interval. The ensemble result is separate.

VariantSweepFoldsMean test QWK ± SDEnsemble QWK

What the evidence supports

Metadata predicts labels; external performance varies markedly; a controlled fine-tuning study supports a role for grading labels. These are distinct findings, with distinct limits.

What remains unknown

Prospective performance, patient-cluster uncertainty, robustness across additional sites and consistent grading standards. Duplicate-pair disagreement does not establish a formal accuracy ceiling.

03 / EXAMINATION-DISJOINT FOLLOW-UP · 26 OCTOBER 2026

Better on Messidor. A cost on APTOS.

A separate paired run starts from baseline-c8232af68337, a different baseline from the historical tables above. Five models per arm were fine-tuned on identical images. Only training labels differed; adjudicated validation targets were common to both arms. No supplied examination crosses partitions.

New Messidor test split · same original APTOS cuts for all arms
ArmReferable AUCQWKSensitivitySpecificityFalse negatives

Report the gain and the loss.

This is a previously inspected cohort and a third-party examination pairing transcription. Longitudinal patient identity is unverified. Grade-4 recall remains low; 24 test eyes support that estimate.

Full audited results, rare-grade recall and validation thresholds ↗

SCIENTIFIC ASSESSMENT / 26 OCTOBER 2026

Evidence quality matters as much as the score.

A separate completed GPU sweep has been audited from all five saved prediction files. A retrieved examination-pair table reveals seven examinations crossing the historical fine-tuning split, including six test/development overlaps. The completed follow-up repairs examination separation and quantifies both a Messidor gain and an APTOS retention loss. Both studies remain exploratory; longitudinal patient identity is unverified. The report documents repeated-holdout limitations, label-noise assumptions and the remaining data requirements.

Read the scientific assessment and GPU audit ↗

04 / REPRODUCIBILITY

Every displayed result has a source.

This page is generated offline from six committed reports and audits. Source hashes let reviewers detect changed inputs. They do not independently verify the original training run.

Methods, reproduction and limitations ↗ · Kaggle research narrative ↗