| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
|
by the American Institute of Ultrasound in Medicine J Ultrasound Med 29:25-36 • 0278-4297 Effect of Observer Experience in the Differentiation Between Benign and Malignant Liver Tumors After Ultrasound Contrast Agent InjectionDepartment of Radiology, Cattinara Hospital, University of Trieste, Trieste, Italy (E.Q., E.B., R.P., G.C., A.M., M.A.C.); and Department of Radiology, University of Palermo, Palermo, Italy (V.A.). Address correspondence to Emilio Quaia, MD, Department of Radiology, Cattinara Hospital, University of Trieste, Strada di Fiume 447, 34149 Trieste, Italy. E-mail: equaia{at}yahoo.com
Objective. The purpose of this study was to assess the impact of the observer level of experience on the diagnostic performance of contrast-enhanced ultrasound imaging (CEUS) for differentiation between benign and malignant liver tumors. Methods. From a computerized search, we retrospectively identified 286 biopsy-proven liver tumors (105 hepatocellular carcinomas, 48 metastases, 7 intra-hepatic cholangiocarcinomas, 33 liver hemangiomas, and 93 nonhemangiomatous benign lesions) in 235 patients (140 male and 95 female; mean age ± SD, 56 ± 11 years) who underwent CEUS after sulfur hexafluoride-filled microbubble injection. The digital cine clips recorded during the arterial (10–35 seconds from injection), portal (50–120 seconds), and late (130–300 seconds) phases were analyzed by 6 independent observers without experience (group 1, observers 1–3) or with 2 to 10 years of experience in CEUS (group 2, observers 4–6). Specific training in the diagnostic and interpretative criteria was provided to the inexperienced observers. Each observer used a 5-point scale to grade diagnostic confidence: 1, definitely benign; 2, probably benign; 3, indeterminate; 4, probably malignant; or 5, definitely malignant on the basis of the enhancement pattern during the arterial phase and enhancement degree during the portal and late phases compared with the liver (hypoenhancement indicating malignant and isoenhancement to hyperenhancement indicating benign). Results. The analysis of observer diagnostic confidence revealed higher intragroup (  = 0.63–0.83) than intergroup ( = 0.47–0.63) observer agreement. The experienced observers showed higher diagnostic performance in malignancy diagnosis than did inexperienced observers (overall accuracy: group 1, 63.3%–72.8%; group 2, 75.9%–93.1%; P < .05,  2 test). Conclusions. The diagnostic performance of CEUS in liver tumor characterization was dependant on the observer’s level of experience.
Key Words: contrast-enhanced ultrasound imaging • liver • microbubbles • tumor Abbreviations: CEUS, contrast-enhanced ultrasound imaging • CT, computed tomography • MRI, magnetic resonance imaging
Gray scale and color Doppler ultrasound imaging are limited in characterizing liver tumors because of the similar appearance and vascular architecture of malignant and benign lesions.1,2 Contrast-enhanced ultrasound imaging (CEUS)3–5 has been shown to overcome the limitations of unenhanced ultrasound imaging and allows a definite improvement in liver tumor characterization.6–12 Ultrasound contrast agents are microbubbles presenting a pure intravascular distribution, even though some agents, including SH U 508A (Levovist; Schering AG, Berlin, Germany) and pefluorobutane (Sonazoid; GE Healthcare, Chalfont St Giles, England), present a postvascular hepato-specific phase from 2 to 5 minutes after intravenous injection. This phenomenon is probably determined by the adherence of the microbubbles to the hepatic sinusoids or by selective uptake by the phagocytic cells of the reticuloendothelial system. Because ultrasound contrast agents do not leak in the interstitial space but persist in the sinusoids and portal vessels,4,5 no equilibrium phase exists, and enhancement of the liver parenchyma shows progressive decay until the baseline appearance is again observed. The enhancement resulting exclusively from the hepatic arterial supply is timed from 10 to 20 seconds after intravenous contrast agent injection and lasts for 10 to 15 seconds.13,14 The portal phase lasts until 2 minutes after contrast agent injection, whereas the late phase lasts up to 4 to 6 minutes after injection until microbubble clearance from the liver parenchyma.13,14 The tumor enhancement pattern during the arterial phase, namely, the distribution of the increased echo signal intensity after ultrasound contrast agent injection within the lesion, provides a valuable clue to the benign or malignant nature of the lesion. Many different enhancement patterns have been described,6–12 some more common in malignancies (peripheral rimlike enhancement in metastases15), some typically identified in benign lesions (peripheral nodular enhancement in hemangiomas16 and a central spoked wheel-shaped enhancement in focal nodular hyperplasia17), and others common both to malignant and benign liver tumors (diffuse enhancement in hepatocellular carcinomas,15 adenomas, and focal nodular hyperplasia17 and absence of enhancement in metastases and avascular thrombotic hemangiomas16). However, the enhancement patterns observed during the arterial phase are not sufficient for characterizing liver tumors.18 The evaluation of the degree of tumor enhancement during the portal and late phases, namely, the grade of tumoral echo signal intensity compared with the liver parenchyma, is essential for the final diagnosis because malignant tumors prevalently show a hypoenhancing appearance, whereas benign tumors tend to show an isoenhancing or a hyperenhancing appearance due to persistent ultrasound contrast agent uptake.18 Frequently, the degree of tumor enhancement changes from the portal to late phase, and this could potentially influence the final diagnosis according to the observer’s visual evaluation and level of experience. To our knowledge, no previous study has analyzed how the observer experience in the interpretation of the tumor enhancement pattern and degree after microbubble injection could influence the diagnostic performance in liver tumor characterization. The objective of this study was to assess the impact of the observer level of experience on the diagnostic performance of CEUS for differentiation between benign and malignant liver tumors.
Patients Approval for this retrospective study was granted by the Ethics Review Board of our hospital, and informed consent was obtained from all patients at the time of scanning after the nature of the procedure had been fully explained. From a computerized search of our hospital’s database of radiologic records from January 2004 to January 2008, a reference radiologist responsible for the study identified 390 consecutive patients (235 male and 155 female; mean age ± SD, 57 ± 13 years) with 440 liver tumors studied by CEUS after sulfur hexafluoride-filled micro-bubble (SonoVue, Bracco SpA, Milan, Italy) injection. Liver tumors had been detected by gray scale ultrasound imaging or cross-sectional imaging (multiphase contrast-enhanced computed tomography [CT] or magnetic resonance imaging [MRI]) and corresponded to incidental liver tumors discovered during a routine clinical diagnostic workup (n = 225) or suspected malignancies identified during staging before surgery or postsurgical follow-up (n = 105) or surveillance for chronic liver disease (n = 110). To complete the diagnostic workup, those tumors detected only by gray scale ultrasound imaging were scanned by multiphase cross-sectional imaging techniques (CT, MRI, or both) 2 to 15 days after detection. All liver tumors were then studied by CEUS 1 to 28 days after detection because they were considered indeterminate at the time of diagnosis by the on-site radiologists, who had 10 to 20 years of experience in abdominal radiology. Indeterminate liver tumors were those that did not reveal any definite features with regard to the nature (benign or malignant) or histotype and those in which unenhanced gray scale ultrasound imaging with color or power Doppler or cross- sectional imaging were inconclusive. Up to 2 lesions per patient were selected for CEUS on the basis of largest diameter and best visualization through the acoustic window according to the on-site radiologist’s evaluation. Ultrasound-guided core needle biopsy was performed with 18- to 20-gauge modified Menghini needles 2 to 15 days after CEUS, and the histologic findings were considered the reference standards for the final diagnoses.
The reference radiologist excluded 154 liver tumors because of technical inadequacy of the CEUS due to failure in data storage (n = 5), incomplete tumor visibility (n = 25), or lack of a histologic diagnosis due to the typical malignant pattern depicted by CEUS, CT, or MRI (n = 110)—evidence of multiple additional enhancing lesions with a hypoenhancing appearance during the portal and late phases or tumors greater than 5 cm in diameter with heterogeneous contrast enhancement during the arterial phase and a hypoenhancing appearance during the portal and late phases; and a high probability of liver hemangiomas (n = 14)—evidence of obvious peripheral nodular enhancement (discontinuous or continuous peripheral nodular appearance) with centripetal fill-in. Therefore, 286 liver tumors (Table 1
Contrast-Enhanced Ultrasound Examinations All examinations were performed on the same system (Acuson Sequoia; Siemens Medical Solutions, Mountain View, CA; with a convex array 2- to 4-MHz 4C1 transducer) by on-site radiologists. Before microbubble injection, the liver tumors were assigned a liver segment location19,20; the tumor diameter was measured in the transverse and longitudinal planes with calipers, and the largest diameter in centimeters was recorded. Microbubbles were then manually injected as a 2.4-mL bolus through an 18- to 20-gauge intravenous cannula followed by a 10-mL normal saline flush, and each tumor was scanned by real-time continuous insonation during normal breathing or breath holding, depending on which yielded the best visualization of the tumor. Technical parameters were as follows: Cadence contrast pulse sequencing as a contrast-specific technique; tissue equalization applied before contrast agent injection; echo signal gain below noise visibility; low transmit power (mechanical index, 0.14–0.16), dynamic range, 75 dB; temporal resolution between frames, 75 to 100 milliseconds (10–13 frames per second); signal persistence turned off; and 2 focal zones below the level of the tumor. The arterial phase was timed for 10 to 35 seconds after microbubble injection, the portal venous phase for 50 to 120 seconds, and the late phase for 130 to 300 seconds.12,13 Distinct digital cine clips for the arterial, portal venous, and late phases of the CEUS study were stored on a personal computer (Pentium 4; Intel Corporation, Santa Clara, CA) connected to the ultrasound equipment by means of frame- grabber software (Mediacruise; Canopus Corporation, San Jose, CA) with an integrated high-performance hardware-based real-time Moving Picture Experts Group 2 encoder (Mediacruise MVR1000). Digital cine clips had a duration of 10 to 20 seconds during the arterial phase, 40 to 50 seconds during the portal phase, and 100 to 120 seconds during the late phase.
Cine Clip Analysis All observers worked independently and were blinded to the identification, clinical history, biopsy results, and other imaging findings of the patients. Digital cine clips of each lesion were randomly assigned to each observer, and all readings were performed on the same computer (Intel Pentium 4 with a 19-in [48-cm] thin-film transistor display) at a central location with Power-DVD software (CyberLink Corporation, Fremont, CA). Each observer was presented with digital cine clips recorded during the arterial, portal, and late phases and was asked to assess the tumor enhancement pattern during the arterial phase6–12 and the degree of tumor enhancement during the different dynamic phases.11,13,14,18 The enhancement pattern referred to the distribution of the increased echo signal intensity within the lesion, whereas the degree of tumor enhancement referred to the tumoral echo signal intensity compared with the adjacent liver parenchyma after microbubble injection. Enhancement patterns during the arterial phase were classified as absent (no difference before and after microbubble injection), dotted (tiny separate spots of enhancement distributed throughout the lesion), peripheral nodular (discontinuous or continuous peripheral enhancement with a nodular appearance), peripheral rimlike (a peripheral continuous enhancing rim that demarcated the tumor from the surrounding liver parenchyma), central (involving the central portion of the tumor, defined as spoked wheel shaped if the central vessel appeared to branch toward the tumor periphery), and diffuse (homogeneous or heterogeneous, involving the whole tumor).
Subsequently, each observer assessed the degree of tumor enhancement and visually compared the echo signal intensity of the enhancing intratumoral region(s) with a homogeneous region of the surrounding liver during the different dynamic phases. Figures 1
Afterward, the 3 scan sequences (arterial, portal, and late phases) for any given lesion were reviewed in one sitting, and the observers were free to review the digital cine clips as many times as they wished. The cases were interpreted in a set order, and 70 to 80 liver tumors were analyzed in each review session. Six reading sessions separated by a mean of 3 days were necessary to complete the review of all patient scans. The observers used a 5-point scale to grade diagnostic confidence: 1, definitely benign (evidence of an enhancement pattern typical of benignancy, including central enhancement with a spoked wheel appearance and isoenhancement or hyperenhancement during the portal and late phases); 2, probably benign (diffuse enhancement and isoenhancement or hyperenhancement during the portal and late phases); 3, indeterminate (absent or dotted enhancement and persistent hypoenhancement with or without evidence of peripheral rimlike enhancement); 4, probably malignant; or 5, definitely malignant (absent or dotted enhancement or diffuse homogeneous or heterogeneous enhancement and hypoenhancement during the portal and late phases). Readers had to complete a form with specific multiple-choice questions regarding the tumor enhancement pattern and degree and the diagnostic confidence score. At the conclusion of the review analysis, the reference radiologist transferred all data to Excel spreadsheets (Microsoft Corporation, Redmond, WA) for analysis.
Statistical Analyses
The difference in the observers’ performance in correctly diagnosing malignancy was calculated by using the
Table 2  shows the different enhancement patterns according to the tumor histotype as reported by the observers involved in the retrospective analysis. The observers did not differ in their classifications of lesion contrast enhancement patterns but exclusively in grading the degree of tumor enhancement.
Figures 6 and 7 show the distribution of the visual grading for liver tumor enhancement during the arterial and portal and late phases, respectively. Malignant (n = 160) versus benign (n = 126) liver tumors did not differ (P > .05) in the degree of contrast enhancement during the arterial phase (Figure 6), with a median grading of +1 versus +1 for all observers. On the other hand, malignant versus benign tumors differed significantly (P = .001) in the degree of contrast enhancement during the portal and late phases (Figure 7), with a median grading of –2 or –1 versus 0 or 2 according to the different observers. Malignant tumors exhibiting an atypical isoenhancing appearance in the portal and late phases (n = 38 according to observer 1; n = 27, observer 2; n = 30, observer 3; n = 25, observer 4; n = 8, observer 5; and n = 7, observer 6) corresponded to well-differentiated hepatocellular carcinomas, whereas benign tumors exhibiting an atypical hypoenhancing appearance in the portal and late phases (n = 67 according to observer 1; n = 69, observer 2; n = 48, observer 3; n = 44, observer 4; n = 32, observer 5; and n = 13, observer 6) corresponded to focal nodular hyperplasia, hepatocellular adenomas, and other benign lesions, including thrombotic avascular liver hemangiomas, necrotic nodules, and liver abscesses.
Table 3 shows the level of observer agreement during the different dynamic phases. Observer interpretations were unanimous regarding the degree of enhancement of most tumors (Figure 8). Interobserver variability occurred for interpretations of tumor enhancement during both the arterial and portal and late phases (Figure 9). Intragroup observer agreement was higher than intergroup agreement in grading tumor enhancement (Table 3). Intragroup observer agreement ( = 0.63–0.83) was also higher than intergroup agreement ( = 0.47–0.63) in grading diagnostic confidence for liver tumor characterization (Table 4). Greater observer experience provided increased diagnostic performance and confidence (Tables 5 and 6) in malignancy diagnosis. The observers’ diagnostic performance and confidence (Tables 5 and 6) in the characterization of liver tumors differed significantly (P = .01).
Contrast-enhanced ultrasound imaging is a reliable imaging tool in the characterization of liver tumors9–12 and allows real-time scanning with the possibility of prolonged liver insonation. This has been achieved as a result of the safe profile and stability of microbubbles persisting in the bloodstream for several minutes.3–5 The operator has to identify the most suitable acoustic window to evaluate the lesion and simply observe tumor enhancement after ultrasound contrast agent injection without moving the transducer from the initial position. Correct image interpretation is less straightforward and depends on the reader’s experience, as shown in this study. In this study, we excluded those liver hemangiomas with a typical enhancement pattern, namely, peripheral nodular enhancement with centripetal fill-in, and those tumors with an overt malignant pattern because their inclusion would have artificially increased the interobserver agreement and would have overestimated the diagnostic capabilities of CEUS. This study included those tumors that could have had variable interpretations of the enhancement degree compared with the adjacent liver for different observers, and we analyzed how this variability could influence the diagnostic performance and confidence of liver tumor characterization on CEUS. Visual analysis is the simplest method for analyzing liver tumor enhancement. However, visual analysis is often penalized in comparison with quantitative analysis by the fact that the observer’s eyes tend to focus on a specific portion of the tumor instead of comparing the echogenicity of the whole tumor and adjacent liver in a more global and reproducible manner. For the purposes of statistical analysis, in this study the different observers were grouped according to the absence (group 1, observers 1–3) or presence (group 2, observers 4–6) of experience in CEUS. Both inexperienced and experienced observers showed good or very good intragroup agreement in assessing the liver tumor degree of enhancement, whereas intergroup agreement was lower. This indicates a clear overlap in the visual analysis provided by the observers with similar levels of experience. The different gradings of liver tumor enhancement proposed by the observers determined different gradings in diagnostic confidence and, consequently, different values of diagnostic performance according to the observer’s experience. Although definite enhancement patterns and degrees of liver tumor enhancement were identified in most of the examined tumors, there was some interobserver variability in the assessment of the liver tumor enhancement degree during both the arterial and portal and late phases, which influenced the correct characterization of liver tumors. Because the diagnostic criteria for liver tumor characterization were based on the enhancement pattern and degree, the better diagnostic performance and confidence shown by the observers with greater experience was related to a more accurate visual assessment of tumor vascularity after microbubble injection and a more effective integration between the tumor enhancement pattern and degree observed during the different dynamic phases. We identified some overlap between benign and malignant tumors in terms of the degree of tumor enhancement during the portal and late phases due to some benign tumors having a hypoenhancing appearance similar to that of malignancies, as described previously,10,11,24 and to some hepatocellular carcinomas having an isoenhancing appearance, as do benign tumors. It has already been shown that 50% of hepatocellular carcinomas show typical wash-out by 90 seconds after ultrasound contrast agent injection, whereas in the remaining 50% of cases, contrast wash-out appears after 91 to 180 seconds or even later.25 The percentages of isoenhancing and hyperenhancing hepatocellular carcinomas in the portal and late phases were also reported to be 33% to 38%.26,27 This variability may have accounted for some of the interob-server variability found in this study. Contrast-enhanced ultrasound imaging is technically a simple imaging method and allows real-time acquisition after contrast agent administration without any drawbacks due to an incorrect scanning time in comparison with CT and MRI, which are limited by image acquisition time issues. One limitation of CEUS in comparison with multiphase CT and MRI is the fact that only a single liver tumor can be scanned at a time because the transducer has to be kept still during the examination; furthermore, ultrasound contrast agent injections are necessary to characterize additional liver tumors. Moreover, ultrasound contrast agents are generally safe agents28 in comparison with iodinated and gadolinium-based contrast agents, which can cause renal or systemic toxicity. On the other hand, CT and MRI allow simultaneous characterization of multiple liver tumors of similar or different natures in the same patient. This is rarely possible with CEUS because the image planes and technical parameters must be individually optimized for each tumor examined. Computed tomography and MRI may, however, be limited in scanning hypervascular tumors, owing to difficulties in starting the acquisition at a suitable time point,29 and are penalized by costs and patient irradiation. The main limitation of this study was that we did not evaluate intraobserver variability in differentiating between benign and malignant tumors. In this retrospective study, the ultrasound images were visually interpreted by 6 blinded readers to provide an unbiased analysis of the diagnostic capabilities of CEUS. This does not reflect real routine clinical practice, in which the ultrasound operator usually corresponds to the reader evaluating the ultrasound images. Immediate interpretation of the ultrasound images by the same operator during the routine work flow could further improve the general diagnostic performance of CEUS because the operator is frequently not blinded to the patient’s clinical history and other imaging findings. In conclusion, the diagnostic performance of CEUS in liver tumor characterization was dependant on the observer’s level of experience.
We thank Lucio Torelli, associate professor of statistics, Department of Mathematics, University of Trieste, who gave important suggestions for the statistical analysis and data presentation. Received April 23, 2009, from the Department of Radiology, Cattinara Hospital, University of Trieste, Trieste, Italy (E.Q., E.B., R.P., G.C., A.M., M.A.C.); and Department of Radiology, University of Palermo, Palermo, Italy (V.A.). Revision requested June 4, 2009. Revised manuscript accepted for publication August 13, 2009.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Top
Abstract
Introduction
0.20 and < 0.40), moderate (
