Comparing AI anatomy segmentation models when ground truth is missing

A study published in the Journal of Medical Imaging introduces a framework for comparing AI-based anatomy segmentation tools in the absence of expert reference annotations.
The study uses chest CT scans from the National Lung Screening Trial (NLST) to evaluate six open-source segmentation models by focusing on consistency across the models rather than accuracy.
To make models comparable, the researchers standardized the output formats, harmonized labels using common medical terminology, and integrated results into existing visualization tools like OHIF Viewer and 3D Slicer.
The lung segmentations showed strong agreement among models, whereas heart, rib, and vertebrae segmentations revealed more discrepancies, highlighting systematic failures in certain models.
The study underscores the importance of a non-ground truth evaluation framework using standardization, quantitative measures, and visual review for assessing AI segmentation tools.
The methods and tools developed by the team can potentially be applied to other medical imaging datasets and segmentation tasks, offering essential guidance in AI-assisted research.

You may have missed