Syllabus for Refresher Course RSNA 2003
Date, Time: Friday, December 5, 2003 beginning at 8:30AM
Title: Categorical Course in Diagnostic Radiology Physics: Advances in Digital Radiography --- Digital Radiographic Computer-aided Diagnosis
Computer-Aided Diagnosis in Digital Chest Radiography
Kunio Doi, Ph.D. (Department of Radiology, The University of Chicago)
About two decades ago, serious investigations were attempted toward development of computer-aided diagnostic (CAD) schemes for detection and classification of lesions in radiographic images. However, at the early phase of research and development, some computer scientists criticized it by saying, “CAD simply will not work,” which has been proved to be completely wrong. The reason for this strong criticism at that time might have been related to an unsuccessful attempt at previous research efforts toward the
development of automated diagnosis. It may be useful, therefore, to clarify the distinction between CAD and automated diagnosis. The concept of automated diagnosis is based on the assumption or expectation that machines and computers are better in certain tasks than humans, and thus abnormalities would be found by use of a computer, and eventually radiologists would be replaced by computers. Therefore, research efforts on automated diagnosis were focused only on the development of computer algorithms, and the effect of computer output on radiologists’ image interpretation was not taken into account at that time.
The goal of CAD is to improve the quality and the productivity of radiologists’ tasks by improving the accuracy and consistency in radiologic diagnosis and also by reducing the image reading time. The general approach for CAD is to find the location of a lesion, and also to determine an estimate of the probability of a disease; these correspond to CAD for the detection of the lesion and CAD for differential diagnosis. Basic technologies involved in CAD schemes are (1) image processing for detection and extraction of abnormalities, (2) quantitation of image features for candidates of abnormalities, (3) data processing for classification of image features between normals and abnormals (or benign and malignant), (4) quantitative evaluation and retrieval of images similar to unknown lesions, and (5) observer performance studies by use of ROC analysis.
Because the concept of CAD is broad and general, CAD can be applied to all imaging modalities such as projection radiography, CT, MRI, ultrasonography, and nuclear-medicine imaging, all of the body parts such as skull, thorax, heart, abdomen, and extremities, and all kinds of examinations including skeletal imaging, soft-tissue imaging, and angiography. However, the majority of CAD schemes developed in the past include the detection of breast lesions on mammograms (4-9), the detection of lung nodules in
chest radiographs (10-12) and thoracic CT (13-15), and the detection of polyps in CT colonography (16, 17). Therefore, the current results obtained from basic research and clinical applications of CAD may be considered as the tip of an iceberg, and thus a major impact of CAD on medical imaging and diagnostic radiology may be expected in the future.
It has been well documented that radiologists may miss about 30% of lung nodules in chest radiographs, some of which are clearly visible in retrospect. Therefore, the purpose of CAD for detection of nodules in chest radiographs is to indicate the potential locations of nodules as a prompt to radiologists. The computerized scheme for automated detection of nodules was based on a difference-image technique (10, 11) with which nodules were enhanced and the majority of background normal structures were suppressed. The candidates of nodules were then identified by thresholding of pixel values in the difference image derived from the chest radiograph. A number of image features on nodule candidates were quantified, and some false positives caused by normal anatomic structures were removed by a rule-based method together with the use of an artificial neural network (ANN). Finally, the locations of potential sites for nodules were indicated by markers such as arrows on chest images displayed on a monitor, as illustrated in Fig. 1-(a), with the corresponding difference image shown in Fig. 1-(b).
It is apparent in Fig. 1 that a relatively obvious nodule overlapped with a rib in the right lung was correctly detected by the computer, but one false positive was included in the computer output. It is important to reduce the number of such false positives as much as possible in all CAD schemes, including that for detection of lung nodules. The usefulness of computer output such as that shown in Fig. 1-(a) has been investigated by observer performance studies by use of ROC analysis (12). Two sets of digital chest radiographs, i.e., one without and another with computer output, were presented to radiologists for detection of nodules. ROC curves obtained from this study are shown in Fig. 2, where the performance level of the computer output used in this study was a sensitivity of 80% with one false positive per chest image. It is apparent that radiologists’ performance in detecting nodules in chest radiographs was improved when computer output was available. The difference in Az values obtained from the two ROC curves has been confirmed to be statistically significant (12).
Figure 3 shows Az values for 16 radiologists obtained without and with computer output. A number of important observations can be made regarding the results in Fig. 3. First, all of the radiologists were able to improve their detection performance by use of the computer output. Without CAD, the average Az value for eight residents was lower than that for attendings. However, the average gain in Az value due to CAD was greater for the resident group than for the attending group. With CAD, therefore, the
average Az value for the eight residents became comparable to that for the eight attendings. These results indicate that CAD can assist many radiologists in improving the accuracy in detecting lung nodules, and also reducing the variation in the detection accuracy due to the variation in radiologists’ experience (residents vs. attendings). Therefore, these results appear to indicate the potential that the purpose of CAD as described above can be realized in improving the accuracy and consistency of radiologic diagnoses.
Once a lung nodule is found in a chest radiograph, the subsequent task for a radiologist is to assess the nature of the lesion if the nodule is malignant or benign. This task for classification of lung nodules is considered difficult for radiologists. The purpose of CAD for classification of nodules in chest radiographs is to provide the likelihood of malignancy as a second opinion in assisting radiologists’ decisions (18, 19). The computerized scheme for determination of the likelihood of malignancy was based on the analysis of many image features obtained from the nodule in a chest radiograph and also the corresponding difference image. The image features included features obtained from the outline of the nodule such as the shape and size, the distribution of pixel values inside and outside the nodule, and the distribution of edge components. Figure 4 illustrates the comparison of the outlines of three malignant and three benign nodules obtained by an automated computerized method and by four radiologists. It is apparent that the variation in manual outlines by radiologists was included; also, the correspondence of outlines between the manual and computerized methods was varied to some extent. Therefore, the outlines obtained from the computer were used only as estimates of the approximate size and shape of the nodule for subsequent detailed analysis of image features.
The likelihood (%) of malignancy was determined by use of linear discriminant analysis (LDA) or ANN on multi-dimensional distribution of image features. Figure 5 shows the likelihood of malignancy for three malignant and three benign nodules. The usefulness of these results for classification of nodules has been investigated by observer performance studies by use of ROC analysis (20, 21). Figure 6 shows the comparison of ROC curves obtained without and with computer output in distinguishing between benign
and malignant nodules in chest radiographs. It is apparent that radiologists’ performance in the distinction between benign and malignant nodules was improved significantly by use of the computer output. However, it is important to note that the ROC curve for radiologists with CAD was still lower than that obtained with the computer alone. This result seems to indicate that, although radiologists were able to utilize “some of the computer output” in improving their performance, they could not take full advantage of the computer output effectively. The reasons for this may be related to the lack of experience with such computer output.
Therefore, when radiologists become familiar with CAD for classification of nodules, the benefits obtained with CAD might be increased further in the future.
Detection and diagnosis on abnormalities related to interstitial diseases in chest radiographs are considered difficult for radiologists, partly because the variation in radiographic patterns due to interstitial infiltrates is large and very complex. Therefore, the purpose of CAD for detection of interstitial infiltrates is to quantify texture patterns of interstitial opacities by use of the Fourier transform (22, 23), and to indicate the extent and the distribution of localized abnormalities on chest images. Figure 7 shows a chest image with a number of markers superimposed on the lungs, namely, normal areas were marked by pluses, whereas abnormal areas with reticular, nodular or mixed patterns were marked by squares, circles, or hexagons, respectively. The larger the size of the markers for abnormal areas, the more severe the abnormalities of interstitial opacities would be.
Observer performance studies (24, 25) were carried out for examining the effect of computer output, such as those illustrated in Fig. 7, on the detection of subtle abnormalities due to interstitial infiltrates. Figure 8 shows a comparison of ROC curves obtained by radiologists without and with computer output; the ROC curve for the computer result is also shown. It is apparent that radiologists’ performance in detecting subtle interstitial infiltrates was improved by use of the computer output. One may ask
why radiologists were able to improve their performance despite the fact that the ROC curve for the computer output was slightly lower than that for radiologists without CAD. This improvement was possible because the radiologists’ performance was different from the computer results, i.e., for some cases radiologists were correct but computer results were incorrect, and vice versa for other cases; therefore, the radiologists were able to change their initial incorrect judgments when they recognized the correct computer output. This is a clear indication that the computer output can be helpful as a second opinion to assist radiologists’ image
Once interstitial infiltrates are detected on a chest radiograph, the next task for a radiologist is to provide the differential diagnosis on potential interstitial lung diseases. The ANN is a powerful computational tool for learning the relationship between the input, such as clinical parameters and radiologic findings, and the output, such as interstitial diseases; thus, the ANN may be able to provide a useful second opinion on the differential diagnosis of interstitial diseases. Figure 9 is a schematic diagram of the ANN (26, 27) which was designed to distinguish between 11 interstitial lung diseases on the basis of 10 clinical parameters and 16 radiologic findings. Thus, the ANN consisted of 26 input units and 11 output units. The ANN was trained by entering repeatedly and randomly a large number of input data and output data, where all of the input data were converted to numbers ranging from 0 to 1.0, and the output values for a specific disease and for non-disease cases were 1.0 and 0, respectively.
Once the training of the ANN is completed, the ANN can be applied to an unknown case with a set of new input data, and thus provide an output value corresponding to the likelihood of each disease at each output unit, as illustrated in Fig. 10. The ANN output in this case indicated correctly the largest value at the output unit for idiopathic pulmonary fibrosis. The usefulness of the ANN output for differential diagnosis was investigated by observer performance studies (27). Figure 11 shows ROC curves for
the differential diagnosis on interstitial lung diseases by radiologists without and with the ANN output. It is apparent in Fig. 11 that radiologists’ diagnostic accuracy was improved substantially by use of the ANN output; however, the ROC curve for the ANN alone was superior to that obtained by radiologists with the ANN output. It may be possible in the future for radiologists’ performance to be improved further when radiologists become familiar with the ANN output and/or when an additional method for assisting radiologists such as the use of similar images is developed.
It has been well demonstrated in the literature (28-33) that temporal subtraction of sequential chest radiographs is useful for detection of subtle interval changes. The temporal subtraction technique for chest radiographs was based on the use of a non-linear image warping technique (28-30) , as illustrated in Fig. 12. The co-ordinate system for a previous chest image was warped so that major landmarks in chest images including ribcage edges and ribs in a current chest image were matched to those in the
previous image. Therefore, the subtraction of the warped previous image from the current image provided the temporal subtraction image which can indicate the appearance and /or disappearance of abnormalities as well as changes in the existing lesions. Figure 13 shows a comparison of the current image, previous image, and temporal subtraction image. It is apparent in Fig. 13 that a large mass lesion in the lower right lung was clearly visualized as a dark shadow, whereas it may not be so obvious in the current image alone. The effect of temporal subtraction images on radiologists’ diagnostic accuracy was investigated by observer performance
studies without and with the temporal subtraction images. Figure 14 shows ROC curves for detection of focal new abnormalities in paired current and previous chest images without and with the corresponding temporal subtraction image. The Az value for the ROC curve was improved significantly when the temporal subtraction images were available. In addition, the average reading time was reduced by about 20% by use of the temporal subtraction technique (31). The reduction in radiologists’ reading time was possible because radiologists were able to reach decisions more quickly with the use of temporal subtraction images, particularly in cases of
normal chest radiographs.
A number of CAD schemes for detection and classification of lesions in digital chest radiographs have been developed to assist radiologists’ image interpretation. Observer performance studies indicated clearly that radiologists were able to improve their diagnostic accuracy when the computer output was available. It is likely that CAD will have a great impact on medical physics and diagnostic radiology.
The author is grateful to all of the members of the Kurt Rossmann Laboratories for Radiologic Image Research who have contributed to research and development of computer-aided diagnostic schemes over the last 20 years; and Elisabeth Lanzl for improving the manuscript. This work has been supported by USPHS Grant CA 24806. K. Doi is a share-holder of R2 Technology, Inc., Los Altos, CA and Deus Technology, Inc., Rockville, MD.
(1) Doi K, Giger ML, Nishikawa RM, Hoffmann KR, MacMahon H, Schmidt RA, Chua KG. Digital radiography: A useful clinical tool for computer-aided diagnosis by quantitative analysis of radiographic images. Acta Radiologica 1993; 34:426-439.
(2) Doi K, MacMahon H, Katsuragawa S, Nishikawa RM, Jiang Y. Computer-aided diagnosis in radiology: potential and pitfalls. European J Radiology 1999; 31:97-109.
(3) Doi K, MacMahon H, Giger ML, Hoffmann KR (editors). Computer Aided Diagnosis in Medical Imaging (Elsevier, Amsterdam), 1999.
(4) Chan HP, Doi K, Galhotra S, Vyborny CJ, MacMahon H, Jokich PM. Image feature analysis and computer-aided diagnosis in digital radiography. 1. Automated detection of microcalcifications in mammography. Med Phys 1987; 14:538-548.
(5) Chan HP, Doi K, Vyborny CJ, Schmidt RA, Metz CE, Lam KL, Ogura T, Wu Y, MacMahon H. Improvement in radiologists' detection of clustered microcalcifications on mammograms: The potential of computer-aided diagnosis. Invest Radiol 1990; 25:1102-1110.
(6) Yin FF, Giger ML, Doi K, Metz CE, Vyborny CJ, Schmidt RA. Computerized detection of masses in digital mammograms: Analysis of bilateral subtraction images. Med Phys 1991; 18: 955-963.
(7) Huo Z, Giger ML, Vyborny CJ, Wolverton DE, Schmidt RA, Doi K. Automated computerized classification of malignant and benign mass lesions on digitized mammograms. Acad Radiol 1998; l 5: 155-168.
(8) Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K. Improving breast cancer diagnosis with computer-aided diagnosis. Acad Radiol 1999; 6:22-33.
(9) Jiang Y, Nishikawa RM, Schmidt RA, Toledano AY, Doi K. The potential of computer-aided diagnosis (CAD) to reduce variability in radiologists’ interpretation of mammograms containing microcalcifications. Radiology 2001; 220:787-794.
(10) Giger ML, Doi K, MacMahon H. Image feature analysis and computer-aided diagnosis in digital radiography. 3. Automated detection of nodules in peripheral lung fields. Med Phys 1988; 15:158-166.
(11) Xu XW, Doi K, Kobayashi T, MacMahon H, Giger ML. Development of an improved CAD scheme for automated detection of lung nodules in digital chest images. Med Phys 1997; 24: 1395-1403.
(12) Kobayashi T, Xu XW, MacMahon H, Metz CE, Doi K. Effect of a computer-aided diagnosis scheme on radiologists’ performance in detection of lung nodules on radiographs. Radiology 1996; 199: 843-848.
(13) Giger ML, Bae KT, MacMahon H. Computerized detection of pulmonary nodules in CT images. Invest Radiol 1994; 29: 459-465.
(14) Armato SG III, Giger ML, MacMahon H. Automated detection of lung nodules in CT scans: Preliminary results. Med Phys 2001; 28: 1552-1561.
(15) Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K. Performance of automated CT nodule detection on missed cancers from a lung cancer screening program. Radiology 2002; 225: 685-692.
(16) Yoshida H, Nappi J, MacEneaney P, Rubin D, Dachman AH. Computer-aided diagnosis scheme for the detection of polyps with CT colonography. RadioGraphics, 2002; 22: 963-979.
(17) Yoshida H, Masutani J, MacEneaney P, Rubin D, Dachman AH. Computerized detection of colonic polyps at CT colonography on the basis of volumetric features: pilot study. Radiology 2002; 222: 327-336.
(18) Nakamura K, Yoshida H, Engelmann R, MacMahon H, Katsuragawa S, Ishida T, Ashizawa K, Doi K. Computerized analysis of the likelihood of malignancy in solitary pulmonary nodules by use of artificial neural networks. Radiology 2000; 214: 823-830.
(19) Aoyama M, Li Q, Katsuragawa S, MacMahon H, Doi K. Automated computerized scheme for distinction between benign and malignant solitary pulmonary nodules on chest images. Med Phys, 2002; 29: 701-708.
(20) Shiraishi J, Abe H, Englemann R, Aoyama M, MacMahon H, Doi K. Effect of the computer output on radiologists’ decision-making for classification of solitary pulmonary nodules in chest radiographs. In: Lemke HU, Vannier MW, Inamura K, Farman AG, Doi K, Reiber JHC (eds.) Proceedings of Computer Assisted Radiology and Surgery (CARS 2002), Springer, Berlin, 2002; 712-716.
(21) Shiraishi J, Abe H, Engelmann R, Aoyama M, MacMahon H, Doi K. Computer-aided diagnosis for distinction between benign and malignant solitary pulmonary nodules in chest radiographs: ROC analysis of radiologists’ performance. Radiology 2003 (in press).
(22) Katsuragawa S, Doi K, MacMahon H. Image feature analysis and computer-aided diagnosis in digital radiography: Detection and characterization of interstitial lung disease in digital chest radiographs. Med Phys 1988; 15: 311-319.
(23) Ishida T, Katsuragawa S, Nakamura K, Ashizawa K, MacMahon H, Doi K. Computerized analysis of interstitial lung diseases on chest radiographs based on lung texture, geometric-pattern features and artificial neural networks. Proc SPIE, 2002; 4684: 1331-1338.
(24) Monnier-Cholley L, MacMahon H, Katsuragawa S, Morishita J, Doi K. Computerized analysis of interstitial infiltrates on chest radiographs: A new scheme based on geometric pattern features and fourier analysis. Acad Radiol 1995; 2: 455-462.
(25) Monnier-Cholley L, MacMahon H, Katsuragawa S, Morishita J, Ishida T, Doi K. Computer aided diagnosis for detection of interstitial infiltrates in chest radiographs: Evaluation by means of ROC analysis. AJR 1998; 171: 1651-1656.
(26) Ashizawa K, Ishida T, MacMahon H, Vyborny CJ, Katsuragawa S, Doi K. Artificial neural networks in chest radiographs: Application to differential diagnosis of interstitial lung disease. Acad Radiol 1999; 6: 2-9.
(27) Ashizawa K, MacMahon H, Ishida T, Nakamura K, Vyborny CJ, Katsuragawa S,
(28) Kano A, Doi K, MacMahon H, Hassell DD, Giger ML. Digital image subtraction of temporally sequential chest images for detection of interval change. Med Phys 1994; 21: 453-461.
(29) Ishida T, Katsuragawa S, Nakamura K, MacMahon H, Doi K. Iterative image warping technique for temporal subtraction of sequential chest radiographs to detect interval change. Med Phys 1999; 26: 1320-1329.
(30) Li Q, Katsuragawa S, Doi K. Improved contralateral subtraction images by use of elastic matching technique. Med Phys 2000; 27: 1934-1942.
(31) Difazio MC, MacMahon H, Xu XW, Tsai P, Shiraishi J, Armato SG, Doi K: Effect of time interval difference images on detection accuracy in digital chest radiography. Radiology 1997; 202: 447-452.
(32) Uozumi T, Nakamura K, Watanabe H, Nakata H, Katsuragawa S, Doi K: ROC analysis on detection of metastatic pulmonary nodules on digital chest radiographs by use of temporal subtraction. Acad Radiol 2001; 8: 871-878.
(33) Kakeda S, Nakamura K, Kamada K, Watanabe H, Nakata H, Katsuragawa S, Doi K: Observer performance study on the usefulness of temporal subtraction in detection of lung nodules on digital chest images. Radiology 2002; 224: 145-151.
Figure 1 (a) Chest image with computer output, and (b) difference image.
Figure 2 ROC curves for detection of lung nodules by radiologists without and with computer output.
Figure 3 Comparison of Az values for detection of lung nodules by 16 radiologists without and with CAD.
Figure 4 Comparison of nodule outlines by automated method (white lines) and manual drawings by four radiologists (dark lines).
Figure 5 Likelihood of malignancy for six nodules by use of LDA or ANN.
Figure 6 Comparison of ROC curves for distinction between benign and malignant nodules without and with computer output. ROC curve by computer alone is also shown.
Figure 7 Chest image with four different markers indicating normal areas (pluses) and abnormal areas with reticular (squares), nodular (circles), and mixed (hexagons) patterns which were provided by computerized analysis of texture patterns.
Figure 8 Comparison of ROC curves for detection of interstitial infiltrates on chest images without and with computer output. ROC curve by computer alone is also shown.
Figure 9 Schematic diagram of the ANN for differential diagnosis of interstitial lung diseases.
Figure 10 Illustration of the ANN output indicating the likelihood of each disease together with chest image.
Figure 11 Comparison of ROC curves for differential diagnosis of interstitial lung diseases without and with ANN output. ROC curve by computer alone is also shown.
Figure 12 Illustration of non-linear image-warping technique for automated image registration for producing temporal subtraction of chest images.
Figure 13 Comparison of previous, current, and temporal subtraction images.
Figure 14 Comparison of ROC curves by radiologists for detection of lung cancer on chest images without and with temporal subtraction images.
This site was last updated 05/17/04