U.S. flag An official website of the United States government

U.S. Food and Drug Administration

Pediatric IQ Phantoms: Digital Pediatric Image Quality Phantoms and Simulations for Evaluating CT Denoising Methods

The Pediatric IQ Phantoms tool is a dataset of virtual image quality phantoms in a range of diameters and their computed tomography (CT) images for assessing pediatric generalizability of CT denoising devices.

Technical Description

Pediatric IQ Phantoms features:

  1. Dataset Repository consisting of three image quality (IQ) phantoms based on the CTP404 Multi-Contrast and CTP486 Uniform Water modules of Catphan 600 and the CCT189 MITA-Low Contrast module of  Catphan 189/191. Pediatric IQ Phantoms extends these phantoms with virtual implementations available in a range of sizes and simulated at routine and low x-ray dose levels.
  2. Code Repository with examples of a) how to reproduce the dataset under different imaging conditions and b) how to use the dataset to evaluate pediatric generalizability of CT denoising devices.
  3. User Manual with installation, usage instructions, and examples.

The phantoms in the dataset (summarized in Figure 1) enable assessment of standard measures of image quality [1] – CT number accuracy, noise magnitude, uniformity, contrast dependent spatial resolution, and low contrast detectability. The phantoms are designed to span the range of pediatric effective waist diameters (Figure 2) and thus can be used to assess pediatric image quality performance [2]. 

Pediatric IQ phantoms
Figure 1. Pediatric IQ phantoms consist of three virtual phantoms for assessing CT number accuracy and spatial resolution (CTP404), low contrast detectability (MITA-LCD), and noise properties and uniformity (Uniform Water). Tools are provided to simulate these phantoms in a range of diameters representing the sizes of small newborns and infants through large adults.
waist diameters
Figure 2. Representative waist diameters of pediatric subgroups. a) Pediatric subgroups and age ranges as recommended in Premarket Assessment of Pediatric Medical Devices3 and associated mean effective waist diameter ranges for those age groups.4b) plot comparison of upper bin range of mean effective waist diameter by subgroup.
Phantom Description

Each phantom is described below with  its reference physical phantom including material properties as well as phantom and inset lesion diameters. The Pediatric IQ Phantoms implementation enables simulating these phantoms with diameters between 80 to 400 mm in diameter and variable lesion size and contrast.

  • CTP404 multi-contrast phantom: for assessing CT number accuracy and contrast-dependent spatial resolution. 
    • CTP404 is a modified version of the sensitometry module CTP404 from the Catphan 600 phantom (The Phantom Laboratory, Salem, NY). 
    • This cylindrical phantom has eight unique contrast inserts ranging from -1000 to +900 HU in a uniform background of 0 HU for assessing CT number accuracy. Due to the sharp intersection between the phantom background and multi-contrast inserts, this module can be used to evaluate contrast-dependent image sharpness by measuring the contrast-dependent modulation transfer function [5]. In its standard size, CTP404 has a diameter of 150 mm with 12 mm diameter inserts. 
    • In the Pediatric IQ Phantoms CTP404, inset lesions are by default set at constant scale (e.g. 12 mm/150 mm = 0.08) relative to phantom diameter but can also be set at constant diameter (e.g. 12 mm).
  • MITA-LCD phantom: for assessing low contrast detectability.
    • MITA-LCD is a modified version of the Catphan CCT189/CCT191 MITA  Low Contrast phantom. 
    • The cylindrical phantom consists of a water-equivalent attenuation material and contains four low-contrast disk inserts of different size and contrast combinations. In its standard size the MITA-LCD phantom has a diameter of 200 mm with 3 mm at 14 HU, 5 mm at 7HU, 7 mm at 5 HU, and 10 mm at 3 HU diameter lesion inserts [6,7].
    • In the Pediatric IQ Phantoms CCT189 MITA-LCD, inset lesions are by default set at a constant scale but can be configured at absolute diameters.
  • Uniform water phantom: for assessing noise and noise texture.
    • The uniform water phantom is a modified version of the uniform water module CTP486 from Catphan 600 phantom.
    • The cylindrical phantom consists of a water-equivalent attenuation material. In its standard size, CTP486 has a diameter of 150 mm.

The acquisition parameters (summarized in Table 1) are included in the Digital Imaging and Communications in Medicine (DICOM) image header files, and in a comma separated variable (csv) file included in the dataset. In addition to the simulated CT images, the dataset also includes ground truth phantom images free of all noise and artifacts by simply discretizing the geometrically defined objects to pixelized images, as well as noise free phantom CT images which have blurring and aliasing artifacts from the simulated CT scans but no quantum or electronic noise.

Summary of Datasets
Table 1. Summary of datasets used for denoising model training and testing. This study focused on the Siemens scanners used in the low dose CT grand challenge dataset (LDGC) [9]. *The reconstruction kernel used in simulations was a D45 equivalent kernel with the same 50% and 10% MTF cutoff values as the Siemens D45 kernel. WED: water equivalent diameter [10].
Repository

The repository associated with the dataset contains examples of how to regenerate the dataset under different CT imaging conditions and scanner models as well as have access to projection data for image reconstruction evaluation. The CT simulation method implemented is based on the Michigan Image Reconstruction Toolkit (MIRT). MIRT is a freely available software package developed at the University of Michigan and widely used by researchers in the CT imaging field.

User Manual

A user manual is available in both web-based format as well as an offline pdf and provides detailed installation, usage instructions, and function reference. 

Inputs

If using the pre-generated dataset no inputs are required. The dataset is intended to be consumed by an image-based denoising model that can process and denoise the DICOM image files for subsequent size-based performance assessment. Example computational notebooks are provided to demonstrate how image quality assessments can be made by comparing the denoised outputs to the corresponding full dose and noise free ground truth images. 

If the user wishes to regenerate the dataset under different imaging or patient conditions, the configuration file used to produce the pre-generated dataset is available in the repository along with examples of other scenarios such as changing the reconstruction kernel. These configuration files can be modified as needed by the user to simulate different reconstruction kernels, x-ray source current and potential. See Documentation for further details.

Rationale

Size is one of the most important patient factors influencing CT performance as it determines the overall x-ray attenuation and noise properties. Pediatric patients have a large range of sizes that varies with development that are not often adequately represented in device evaluations due to the scarcity of pediatric radiology exams and their protected status [11]. Thus to ensure that pediatric patients are not adversely biased due to poor performing devices it is important to have least burdensome evaluations representative of pediatric patients. Pediatric IQ phantoms are designed to reproduce the size of pediatric abdomens for pediatric generalizability evaluations of image quality.

Version control, planned development information, and procedures for updates:

Version control of examples is managed using Git, dataset versioning is managed on Zenodo.

Intended Purpose 

Pediatric IQ Phantoms are intended for quantitatively evaluating image quality of nonlinear CT image reconstruction and denoising products in pediatric patients using phantom images. Pediatric evaluation is achieved by varying phantom size, where size is one of the most important factors influencing x-ray interactions and image quality in CT imaging. Nonlinear CT image reconstruction and denoising methods (products code JAK, QIH, LLZ among others) include statistical iterative, model-based iterative and deep learning-based image reconstruction and denoising methods.

Pediatric IQ Phantoms can help assess image quality performance in different patient sizes and pediatric subgroups of a nonlinear CT image reconstruction and denoising method with respect to a reference device (image reconstruction method such as filtered back projection (FBP) method) by comparing performance measures at the same radiation dose level and patient size. The tool can also help assess the generalizability of deep learning CT denoising and image reconstruction methods by comparing measures across phantoms of different sizes representing different pediatric subgroups.

Intended users are CT device developers, CT image reconstruction developers, and image denoising and processing software developers.

Testing

Code validation was performed to ensure that the Pediatric IQ phantoms were function as designed.  Specifically, for the three included phantoms MITA-LCD, CTP404, and Uniform, 

  • The digital phantom objects created by the simulated digital phantom code were verified to have the same HU contrast and diameters as described in the specifications of the MITA-LCD, CATphan600 phantom and uniform water phantoms. 
  • The CT simulation code was confirmed to provide parameter settings from which the simulated CT images have image resolution and noise texture of CT images in the Low-dose CT grand challenge dataset. The image resolution and noise texture were assessed by modulation transfer function and noise power spectrum [12].

Pediatric IQ Phantoms were validated to yield consistent noise reduction performance trends as those evaluated using anthropomorphic XCAT pediatric phantoms of similar size ranges as discussed in the peer reviewed manuscript:

  • Nelson BJ, Kc P, Badal A, Jiang L, Masters SC, Zeng R. Pediatric evaluations for deep learning CT denoising. Medical Physics. 2024;51(2):978-990. doi:10.1002/mp.16901

Limitations

  • Size is just one of several factors in which pediatric patients differ from adults in the context of CT imaging [14]. Depending on the application other differences in developmental anatomy, pathology, and imaging protocol may need to be considered. The Pediatric IQ phantoms were developed for abdominal imaging where patient size and image characteristics vary widely. 
  • Phantom-based performance assessments may bias data-driven methods like deep learning CT image denoisers trained only on patient data. However, the intended purpose of Pediatric IQ phantoms is not to estimate absolute performance in pediatric subgroups, but rather to identify outstanding performance difference trends that may exist between different subgroups due to limited training dataset representation.
  • Simulated CT scanners can be used to generate phantom CT images conveniently over a large range of dose levels and other scan settings to help developers perform initial evaluation of their reconstruction and denoising algorithms. However, a simulated CT scanner might not model all the physical aspects of a real CT scanner. The CT simulation example in the tool is built on the publicly available MIRT package. It should be noted that the parameter settings provided in the tool correspond to an idealized fan-beam CT scanner with a point x-ray focal spot, monoenergetic x-ray source, Poisson noise model and without x-ray beam filter and scatter effects. Users can resort to more sophisticated CT simulation software for creating realistic CT images and obtaining image quality measures better reflecting what would be achieved in physical IQ phantom scans.  It is recommended that product developers always validate the conclusions made from the simulated data with physical phantom CT scans.

Supporting Documentation

  • Dataset hosted by Zenodo to preview and download the pre-generated dataset
  • Repository containing all source code used to generate the dataset including examples of
    • how to reproduce the dataset under different imaging conditions
    • how to use the dataset to evaluate pediatric generalizability of CT denoising devices.
  • User Manual with installation, usage instructions, and examples.
References
  1. IEC61223-3-5 ed2 Imaging performance of computed tomography X-ray equipment. https://webstore.iec.ch/publication/59789
  2. Nelson BJ, Kc P, Badal A, Jiang L, Masters SC, Zeng R. Pediatric evaluations for deep learning CT denoising. Medical Physics. 2024;51(2):978-990. doi:10.1002/mp.16901
  3. Health C for D and R. Premarket Assessment of Pediatric Medical Devices. U.S. Food and Drug Administration. Published March 24, 2020. Accessed March 1, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/premarket-assessment-pediatric-medical-devices
  4. Boone J, Strauss K, Cody D, et al. Size-Specific Dose Estimates (SSDE) in Pediatric and Adult Body CT Examinations. AAPM; 2011. doi:10.37206/143
  5.  Richard S, Husarik DB, Yadava G, Murphy SN, Samei E. Towards task-based assessment of CT performance: system and object MTF across different reconstruction algorithms. Med Phys. 2012;39(7):4115-4122. doi:10.1118/1.4725171
  6. Computed Tomography Image Quality (CTIQ): Low-Contrast Detectability (LCD) Assessment When Using Dose Reduction Technology. NEMA. Published September 1, 2017. Accessed July 5, 2023. https://www.nema.org/standards/view/Computed-Tomography-Image-Quality-CTIQ-Low-Contrast-Detectability-LCD-Assessment-When-Using-Dose-Reduction-Technology
  7. Catphan 189 & 191 MITA. The Phantom Laboratory. Accessed May 17, 2024. https://www.phantomlab.com/catphan-mita
  8. Moen TR, Chen B, Holmes III DR, et al. Low-dose CT image and projection dataset. Medical Physics. 2021;48(2):902-911. doi:10.1002/mp.14594
  9. McCollough CH, Bartley AC, Carter RE, et al. Low-dose CT for the detection and classification of metastatic liver lesions: Results of the 2016 Low Dose CT Grand Challenge. Med Phys. 2017;44(10):e339-e352. doi:10.1002/mp.12345
  10. McCollough C, Bakalyar D, Bostani M, et al. Use of Water Equivalent Diameter for Calculating Patient Size and Size-Specific Dose Estimates (SSDE) in CT. AAPM; 2014. doi:10.37206/146
  11. Sammer MBK, Akbari YS, Barth RA, et al. Use of Artificial Intelligence in Radiology: Impact on Pediatric Patients, a White Paper From the ACR Pediatric AI Workgroup. Journal of the American College of Radiology. Published online July 25, 2023. doi:10.1016/j.jacr.2023.06.003
  12. Zeng R, Lin CY, Li Q, et al. Performance of a deep learning-based CT image denoising method: Generalizability over dose, reconstruction kernel, and slice thickness. Med Phys. 2022;49(2):836-853. doi:10.1002/mp.15430
  13. Vaishnav JY, Jung WC, Popescu LM, Zeng R, Myers KJ. Objective assessment of image quality and dose reduction in CT iterative reconstruction. Med Phys. 2014;41(7):071904. doi:10.1118/1.4881148
  14. Figaji AA. Anatomical and Physiological Differences between Children and Adults Relevant to Traumatic Brain Injury and the Implications for Clinical Assessment and Care. Front Neurol. 2017;8:685. doi:10.3389/fneur.2017.00685

Contact

Tool Reference