Catalog of Regulatory Science Tools to Help Assess New Medical Devices
This regulatory science tool is a set of computer models that allow for the generation of in silico breast radiographic images for the evaluation of digital mammography (DM) and digital breast tomosynthesis (DBT) devices.
Technical Description
The Virtual Imaging Clinical Trials for Regulatory Evaluation (VICTRE) computer modeling pipeline, as originally described in publications [1, 2], is a set of tools that allow for the generation of in silico breast radiographic images for the evaluation of digital mammography (DM) and digital breast tomosynthesis (DBT) devices.
This pipeline offers similar functionality as the original VICTRE release, but with a simplified, user-friendly interface with the possibility to run the full pipeline in a single execution.
In the original VICTRE package, to generate images for the VICTRE virtual clinical trial described, it was required to know how to execute each one of the components and generate the corresponding input files from the output of previous steps. Due to the different nature of every step, as well as the different programming languages involved in previous releases, running the pipeline was not possible in a single execution script. Instead, this Python class integrates all the steps from the original VICTRE package.
Intended Purpose
The VICTRE pipeline allows for the replication of clinical trials evaluating the performance of digital mammography and digital breast tomosynthesis. The components of the pipeline are:
- Breast generation [3]: a 3D breast phantom model is generated including several tissues. The model can be modified to generate different breast sizes and densities, among other parameters.
- Lesion generation: the VICTRE pipeline includes a breast mass generation software [6] and a calcification cluster generation algorithm that can be inserted in the breast model at any time.
- Breast compression [3]: the breast model is compressed between two plates to simulate a breast imaging device using a finite element biomechanics solver.
- X-ray projection [4]: the compressed breast model is used as an input to a Monte Carlo x-ray simulation software (VICTRE_MCGPU 1.5).
- Tomosynthesis reconstruction [5]: a filtered-back projection (FBP) reconstruction software is used to generate a digital breast tomosynthesis from the corresponding projections.
- Export results: the resulting digital mammography and the digital breast tomosynthesis can be saved as standard DICOM files to be processed or viewed in an external software.
- Reader models: the pipeline also includes reader models that can be used to assess detectability on detection and search tasks.
Testing
VICTRE tools were validated separately on each of the references below.
A validation of the full pipeline was done in the pivotal study presented on [1] which shows the comparison of a full in silico pipeline trial against a real clinical trial. The designed in silico trial involved imaging approximately 3,000 digital breast models in DM and DBT modalities. The comparison showed agreement with a real clinical trial in which DBT images were deemed to show a higher detectability for lesions, in general.
The different VICTRE modules have been previously published in the following references. Please check them to see more information about each one of them:
- VICTRE tools overview Ref. [2]
- Breast generation and compression Ref. [3]
- X-Ray projection (VICTRE-MCGPU 1.5) Ref. [4]
- Tomosynthesis reconstruction Ref. [5]
- Mass generation Ref. [6]
The FDA plans to support and continue the development of this pipeline to add new functionality.
Limitations
In silico tools may generate non-realistic models and simulations. The design of these tools allows for the generation of a wide range of breast models. The choices on parameters like size, density, or shape parameters are open and can lead to non-realistic outcomes or software execution failure. The user must be aware of the possibility of generating a dataset that might not be representative of the desired population. Examples are given to generate four common breast fibroglandular densities, namely: dense, heterogeneously dense, scattered, and fatty. The VICTRE pipeline does not have mechanisms to detect or warn the user when the results are not realistic or cannot be trusted.
Errors can happen when the input parameters are not valid. Modules like breast generation and breast mass generation have a large amount of input parameters that, if not set properly, can cause the software to error out. In most cases, the log files and errors caused by the different modules of the software will be saved to an output file inside the results folder. These files show any output generated by the breast generation, breast compression, breast mass generation, Monte Carlo projection and filtered-back projection reconstruction. These output files (extension .out) can be checked for more information.
In case of unknown errors or need of additional assistance, the GitHub repository [7] provides an Issue forum in which questions can be posted. The FDA will monitor this site to address bugs and respond to raised issues.
Supporting Documentation
The code is made public on a GitHub repository [7] along with the documentation and instructions to set up the environment and run it. Examples are provided to run different steps of the pipeline to facilitate the understanding of the code.
The VICTRE pipeline requires a GPU-enabled machine to run the projection code and it is targeted to a developer and/or scientific audience.
The main repository of the pipeline code is located on GitHub.
Documentation and details on how to setup and run the pipeline are also on GitHub. This repository is organized in the following folders:
- VICTRE: Python class that integrates all steps of the pipeline. Each subfolder corresponds to each independent step or module.
- Examples: python examples to use the VICTRE pipeline at different simulation steps.
- Lesions and phantoms: samples of generated lesions and breast models respectively. These are used in the examples.
The repository includes an installation script install.sh that compiles the necessary tools and a requirements.txt file that installs the required Python modules.
References
[1] A. Badano, C. G. Graff, A. Badal, D. Sharma, R. Zeng, F. W. Samuelson, S. Glick, and K. J. Myers, "Evaluation of Digital Breast Tomosynthesis as Replacement of Full-Field Digital Mammography Using an In Silico Imaging Trial", JAMA Network Open. 2018; 1(7).
[2] D. Sharma, C. G. Graff, A. Badal, R. Zeng, P. Sawant, A. Sengupta, E. Dahal, and A. Badano, “In silico imaging tools from the Victre clinical trial”, Med. physics 46, 3924–3928 (2019).
[3] C. Graff, “A new, open-source, multi-modality digital breast phantom”, Proc. SPIE 9783, Medical Imaging 2016: Physics of Medical Imaging, 978309.
[4] A. Badal, D. Sharma, C. G. Graff, R. Zeng, and A. Badano, “Mammography and breast tomosynthesis simulator for virtual clinical trials”, Computer Physics Communications, Volume 261, 2021, 107779.
[5] A. Sengupta, R. Zeng, D. Sharma, and A. Badano, “The first freely available open source software package for performing 3D image reconstruction for digital breast tomosynthesis”, Proc. SPIE 10573, Medical Imaging 2018: Physics of Medical Imaging, 105735M.
[6] L. de Sisternes, J. G. Brankov, A. M. Zysk, R. A. Schmidt, R. M. Nishikawa, and M. N. “A computational model to generate simulated three-dimensional breast masses”. Med. Phys., 42: 1098-1118 (2015).
[7] https://github.com/DIDSR/VICTRE_PIPELINE