U.S. flag An official website of the United States government

U.S. Food and Drug Administration

LCD-CT: Low-contrast Detectability (LCD) Test for Assessing Advanced Nonlinear CT Image Reconstruction and Denoising Methods

Catalog of Regulatory Science Tools to Help Assess New Medical Devices

 

This regulatory science tool presents a lab method to quantitatively assess the low-contrast detectability (LCD) of advanced nonlinear computed tomography (CT) image reconstruction and denoising solutions, utilizing MITA-LCD phantom images.

 

Technical Description

The Low-contrast detectability (LCD) computed tomography (CT) is a toolkit for assessing image quality of advanced nonlinear CT image reconstruction and denoising products, including but not limited to statistical iterative, model-based iterative, and deep learning-based methods. The toolkit uses model observer (MO) to evaluate the low-contrast detectability (LCD) of targets with known locations in test images. The MO is based on a channelized Hotelling model (CHO) with Laguerre-Gauss (LG) channels [1]. Another two optional models are implemented with Difference-of-Gaussian (DOG) channels [2] and Gabor channels [3] respectively. Detectability is evaluated on images of the MITA-LCD phantom, developed previously and jointly between FDA and Medical Imaging and Technology Alliance (MITA) [4]. The MITA-LCD phantom is a cylindrical phantom filled with water-equivalent attenuation material and contains four low-contrast disk inserts of different size and contrast combinations: 3mm-14HU, 5mm-7HU, 7mm-5HU, 10mm-3HU. The specifications of the MITA-LCD phantom can be found here.

The input to the LCD-CT toolkit is a set of CT images obtained with the MITA-LCD phantom. The output consists of estimates of detection accuracy corresponding to the four low-contrast inserts. The reader model detection accuracy is measured by the area under the receiver operating characteristic curve (AUC) and the detectability signal-to-noise ratio (d’snr).

The LCD-CT toolkit also provides a function for creating a digital replica of the MITA-LCD phantom and for virtually scanning the digital phantom modules in a simulated CT scanner to generate fan-beam CT scans. This tool feature can be used to generate test data (noisy sinograms and reconstructed CT images) that may be further processed by a reconstruction and denoising algorithm to determine the LCD performance of such an algorithm. The CT simulation method implemented is based on the Michigan Image Reconstruction Toolkit (MIRT), a freely available software package developed at the University of Michigan and widely used by researchers in the CT imaging field.

Intended Purpose

The LCD-CT software toolkit is intended for quantitatively evaluating the LCD of advanced nonlinear CT image reconstruction and denoising products using MITA-LCD phantom images. Advanced nonlinear CT image reconstruction and denoising methods (products code JAK, QIH, LLZ among others) include statistical iterative, model-based iterative and deep learning-based image reconstruction and denoising methods.

The LCD-CT tool can help assess the image quality improvements of an advanced nonlinear CT image reconstruction and denoising method with respect to a reference device (image reconstruction method such as filtered back projection (FBP) method) by comparing LCDs on CT data acquired at the same radiation dose level. The tool can also help assess quantitative dose reduction potential by comparing the LCD corresponding to data acquired at a lower radiation dose level and processed by the advanced nonlinear image reconstruction and denoising method to the LCD corresponding to data acquired at a higher radiation dose level and processed by the reference reconstruction method.

Intended users are CT device developers, CT image reconstruction developers, and image denoising and processing software developers.

Testing

Code validation was performed to ensure that the LCD-CT tool functions as designed for each of the three major components, as described below:

  • The phantom objects created by the simulated digital phantom code were verified to have the same HU contrast and diameters as described in the specifications of the MITA-LCD phantom.
  • The CT simulation code was confirmed to provide parameter settings from which the simulated CT images have image resolution and noise texture of CT images in the Low-dose CT grand challenge dataset. The image resolution and noise texture were assessed by modulation transfer function and noise power spectrum [5].
  • The CHO implementation was validated in an inter-laboratory comparison study and shown to yield detectability AUC/SNR within the expected range on a dataset containing simulated signal-present and signal-absent noisy images [6].
  • The same CHO model was also integrated in the VICTRE trial pipeline to evaluate the lesion detectability in simulated digital mammography and breast tomosynthesis images. The findings of the VICTRE study were demonstrated to be consistent with the results of a clinical imaging trial involving human readers [7,8].

The LCD-CT tool implementation follows the task-based assessment framework for iterative CT reconstruction depicted in Vaishnav et al. [9]. The NEMA/MITA white paper [4] also provides an overview of the technical and phantom considerations for assessing LCD of nonlinear reconstruction methods.

Limitations

  1. The LCD-CT tool includes a channelized Hotelling reader model based on LG channels (and 2 other optional channel models: Gabor and DOG). Other types of model observers have been described in the literature including the optimal linear MO (that is, the Hotelling MO), non-prewhitening MO (NPW) and non-prewhitening MO with eye filter (NPWE). Different model observers applied to the same set of images can yield different detectability AUC values due to the different features extracted by a model observer to perform a detection task. The LCD-CT tools employs channelized model observers based on the following considerations:
    1. Channelization of image data into a few spatial-frequency bands is considered to be similar to the mechanism of the human visual system.
    2. Channelization reduces the data dimensionality to ease the data demand for covariance estimation and can yield an AUC estimate with a smaller uncertainty from a relatively small set of images (e.g., 200 pairs of signal-present and signal-absent images for the LCD test).
    3. LCD estimated by CHOs are very close to each other for this task involving round signals in a uniform CT background.
  2. Simulated CT scanners can be used to generate phantom CT images conveniently over a large range of dose levels to help developers perform initial evaluation of the dose reduction capability of their reconstruction and denoising algorithms. However, a simulated CT scanner might not model all the physical aspects of a real CT scanner. The CT simulation example in the tool is built on the publicly available MIRT package. It should be noted that the parameter settings provided in the tool correspond to an idealized fan-beam CT scanner with a point x-ray focal spot, monoenergetic x-ray source, Poisson noise model and without x-ray beam filter and scatter effects. Users can resort to more sophisticated CT simulation software for creating realistic CT images and obtaining LCD estimates better reflecting what would be achieved in physical MITA-LCD phantom scans. It is recommended that product developers always validate the conclusions made from the simulated data with physical phantom CT scans.
  3. The LCD test is designed to assess a specific task-based performance of advanced nonlinear CT reconstruction and denoising methods for establishing a quantitative dose reduction potential relative to a reference method in a practical and reproducible manner. The task is to detect low-contrast disk signals in MITA-LCD phantom scans. The MITA-LCD phantom has a uniform background and therefore, does not represent the details of anatomical backgrounds. It also does not represent different patient shapes and sizes. Detecting round signals in a uniform background does not cover all diagnostic clinical tasks. Users should be aware that a radiation dose reduction level established with the LCD test method is derived from a highly simplified task intended to objectively indicate performance and does not transfer directly to clinical tasks on specific patients.

Supporting Documentation

The matlab code of the LCD-CT tool is released in a GitHub repository [10] along with the documention and instructions to setup the enviroment and run it. Demo matlab files are also provided to help users understand how to use the LCD-CT code to assess LCD in the following senarios.

  • Demo_one_recon_LCD.m: demonstrate how to obtain mean and standard deviation (STD) of the detectability (AUC and d’) estimates of the four inserts from a set of signal-absent (SA) and signal-present (SP) noisy MITA-LCD phantom CT slices associated with one reconstrcution method.
  • Demo_two recons_diffLCD.m: demonstrate how to obtain mean and STD of the difference detectabilty (ΔAUC and Δd’) estimates of the four inserts from two sets of SA and SP MITA-LCD phantom slices corresponding to two reconstruction methods.

To run the LCD phantom code, we suggest that users download the MIRT Github version https://github.com/JeffFessler/mirt. The MIRT Github version is compatible with Windows, Linux and Mac systems. Instructions are provided in the LCDT-CT tool user manual [10] on how to set up the MIRT file directories in the MATLAB enviroment.

The CHOs use training data to estimate the mean and the inverse of the corresponding channelized data covariance to construct their observer templates. To obtain a nonsingular covariance matrix estimate for data of size N, at least N(N+1)/2 independent data points are required. For example, for a LG-CHO of five channels, then at least 15 images should be included in the training set. However, training with 100 pairs of images is suggested to achieve a more accurate estimation. A use case of the LCD-CT tool can be found in the study by Zeng et al. [5] in which the LCD performance of a deep learning-based CT image denoiser was evaluted and compared to FBP.

References

  1. Gallas, B. D., & Barrett, H. H. (2003). Validating the use of channels to estimate the ideal linear observer. Journal of the Optical Society of America. A, Optics, image science, and vision, 20(9), 1725–1738. https://doi.org/10.1364/josaa.20.001725
  2. Abbey, C. K., & Barrett, H. H. (2001). Human- and model-observer performance in ramp-spectrum noise: effects of regularization and object variability. Journal of the Optical Society of America. A, Optics, image science, and vision, 18(3), 473–488. https://doi.org/10.1364/josaa.18.000473
  3. Wunderlich, A., Noo, F., Gallas, B. D., & Heilbrun, M. E. (2015). Exact confidence intervals for channelized Hotelling observer performance in image quality studies. IEEE transactions on medical imaging, 34(2), 453–464. https://doi.org/10.1109/TMI.2014.2360496
  4. NEMA/MITA WP 1-2017: Computed Tomography Image Quality (CTIQ): Low-Contrast Detectability (LCD) Assessment When Using Dose Reduction Technology
  5. Zeng, R, Lin, CY, Li, Q, et al. Performance of a deep learning-based CT image denoising method: Generalizability over dose, reconstruction kernel and slice thickness. Med Phys. 2022; 49: 836– 853. https://doi.org/10.1002/mp.15430
  6. A Ba, CK Abbey, J Baek, M Han, RW Bouwman, C Balta, J Brankov, F Massanes, HC Gifford, I Hernandez-Giron, WJH Veldkamp, D Petrov, N Marshall, FW Samuelson, R Zeng, JB Solomon, E Samei, P Timberg, H Förnvik, I Reiser, L Yu, H Gong, FO Bochud. Inter-laboratory comparison of channelized hotelling observer computation. Med Phys. 2018 Jul;45(7):3019-3030. https://doi.org/10.1002/mp.12940
  7. Badano, A., Graff, C. G., Badal, A., Sharma, D., Zeng, R., Samuelson, F. W., Glick, S. J., & Myers, K. J. (2018). Evaluation of Digital Breast Tomosynthesis as Replacement of Full-Field Digital Mammography Using an In Silico Imaging Trial. JAMA network open, 1(7), e185474. https://doi.org/10.1001/jamanetworkopen.2018.5474
  8. Zeng, R., Samuelson, F. W., Sharma, D., Badal, A., Christian, G. G., Glick, S. J., Myers, K. J., & Badano, A. (2020). Computational reader design and statistical performance evaluation of an in-silico imaging clinical trial comparing digital breast tomosynthesis with full-field digital mammography. Journal of medical imaging (Bellingham, Wash.), 7(4), 042802. https://doi.org/10.1117/1.JMI.7.4.042802
  9. Vaishnav, J.Y., Jung, W.C., Popescu, L.M., Zeng, R., and Myers, K.J. (2014), Objective assessment of image quality and dose reduction in CT iterative reconstruction. Med. Phys., 41: 071904. https://doi.org/10.1118/1.4881148
  10. Link to the LCD_CT Github website: https://github.com/DIDSR/LCD_CT

Contact

Tool Reference

  • In addition to citing relevant publications please reference the use of this tool using RST24MD08.01