Medical-image machine learning

Image-based biomarker prediction

Testing what histology can tell us about colorectal cancer.


I built and evaluated pathology-image models for CMS, microsatellite instability (MSI), and RAS/BRAF mutations. The central question was whether each prediction could be useful at an operating point chosen for its intended clinical task.

MSI sensitivity
94.4%
Negative predictive value
98.3%
Specificity
50.1%

Held-out test results for the MSI encoder ensemble at the frozen screening threshold.

Cohorts, the histopathology pipeline and the four evaluated tasks: CMS, MSI, mutations and survival.
Whole-slide images become patch embeddings, which are pooled to make slide-level predictions.

Whole-slide images become patch embeddings, which are pooled to make slide-level predictions.

From tissue patches to predictions

The pipeline combines pathology foundation-model embeddings with attention-based multiple-instance learning. For MSI, I evaluated a high-sensitivity approach that could direct patients toward confirmatory testing. RAS and BRAF required different threshold choices because the useful prediction, and the consequences of an error, differed by target. MSI pre-screening was the most promising application. CMS evaluation remained limited by uncertainty in the reference labels, while clinically tuned RAS/BRAF thresholds left too little coverage to support implementation.

  • PyTorch
  • Multiple-Instance Learning
  • Foundation Models
Next projectImage-based direct survival prediction