Evaluation of computer-aided detection systems in the detection of small invasive breast carcinoma

Purpose: To retrospectively compare two CAD systems for detecting invasive breast cancers manifesting as noncalcified masses smaller than 16 mm. Materials and Metbods: Waiver of informed consent was granted by the Institutional Review Board that approved this HIPAA-compliant study. Mammograms obtained from two institutions providing consecutive invasive carcinomas manifesting as noncalcified masses smaller than 16 mm were evaluated by using two commercially available CAD systems (R2 ImageChecker M1000, version 5.0A and iCAD Second Look, version 6.0 mid operating point). To provide statistical power to test for a possible 10% difference in the sensitivity performance between the systems, 192 consecutive mammographic studies (182 unifocal, six multifocal, and four bilateral cancers) were collected. Masses were characterized using the Breast Imaging Reporting and Data System (BI-RADS). Per study specificity and mass false marker rate were determined by using 51 normal four-view studies, while scoring only the mass false-positive marks for noncalcified masses. Associations between mass characteristics and supplying institution were compared by using χ² tests. A P value of .05 was considered to indicate a significant difference. Results: The respective per study sensitivity, per image (ie, per view) sensitivity, per study specificity, and mass false-positive marker rates were 81.8%, 64.7%, 39.2%, and 1.08 for the R2 ImageChecker M1000 system, and 60.9%, 42.6%, 31.4%, and 1.41 for the iCAD Second Look system. The overall per study and per image sensitivities were significantly better for R2 than for iCAD (McNemar test, all P < .001), with a nonsignificant higher per study specificity and lower mass false marker rate on normal studies. CAD results demonstrated at least a 20% variation between BI-RADS categories 4a and 5 for per study and per image sensitivity. Conclusion: A statistically significant difference was observed in per study and per image sensitivity in our mammography data set with small (<16 mm), noncalcified invasive breast malignancies between two CAD systems. Differences in per study specificity and mass false marker rate were noted but were not statistically significant.