Catalog of Regulatory Science Tools to Help Assess New Medical Devices
This regulatory science tool presents a computer model for optimal experimental design and estimability analysis for mechanistic models of cardiac electrophysiology to determine their ‘identifiability’.
Technical Description
Cardiac electrophysiological computational models are comprised of numerous complex non-linear equations that predict the action potential in the heart at one or multiple sites. A model is identifiable if there are a unique set of parameters for a given model output. Many of these models have been shown to be non-identifiable such that multiple parameter sets give rise to the same model output. For the purpose of evaluating the ‘robustness’ of a model, both parameter sensitivity analysis and uncertainty quantification are important methods which fail when a model is non-identifiable.
Intended Purpose
The tool is relevant to cardiac electrophysiological models such as those implemented within device software (whether software as a medical device, SaMD, or software in a medical device), and also to cardiac modeling software that will be used for in silico testing of a cardiac device (e.g., in silico evaluation of fracture risk of implantable pacemaker leads). The tool is relevant to single-cell and tissue-level cardiac solvers. The tool is intended to facilitate evaluation of model identifiability, which is one step when demonstrating the robustness of a computational model (see Guidances and Standard listed below). The tool is intended to be used by cardiac model developers and users.
Despite the importance of rigorous code evaluation, there are few tools available to model developers (cardiac or other) for performing model identifiability. This tool presents a standalone example (no user input required) for simulating a cardiac model including experimental voltage clamp protocols and evaluating model identifiability.
Testing
The tool has been validated using the parsimonious rabbit action potential model and is an exact copy of the Supplementary Material from the following manuscript:
Shotwell M, Gray RA. Practical Estimability Analysis and Optimal Design in Dynamic Multi-scale Models of Cardiac Electrophysiology, Journal of Agricultural, Biological, and Environmental Statistics, Journal of Agricultural, Biological, and Environmental Statistics, 2016:1-16. DOI: 10.1007/s13253-016-0244-7.
This manuscript presents an applied approach to optimal experimental design and estimability analysis for mechanistic multi-scale models of cardiac electrophysiology. An improved ‘sensitivity plot’—a graphical assessment of parameter estimability—that overcomes a well-known limitation in this context is also presented. The results of the original simulations were reproduced using independent code in R and the conditioning of the estimation problem was assessed using the matrix condition number and the determinant of the information matrix.
These techniques are demonstrated using a cardiac cell model and simulations in:
Gray RA, Pathmanathan P. A Parsimonious Model of the Rabbit Action Potential Elucidates the Minimal Physiological Requirements for Alternans and Spiral Wave BreakupExternal Link Disclaimer. PLOS Computational Biology, 2016, 12(10): e1005087. doi:10.1371/journal.pcbi.1005087.
Limitations
- The method is a black box method that assumes that intra- and extracellular concentrations of ions are constant.
- Forward Euler integration is implemented but may be modified by the user.
- The tissue implementation is one-dimensional, therefore they are not appropriate to study all aspects of two- and three-dimensional behavior including realistic heart geometries.
Supporting Documentation
This tool provides R code for the implementation of an approach for optimal experimental design and estimability analysis for models of cardiac electrophysiology to determine their ‘identifiability’. This code supplement is organized into five sections: 1) Cardiac cell model functions; 2) Functions for single-cell stimulation solutions; 3) Functions for action potential propagation solutions; 4) Functions for inactivation voltage clamp solutions; and 5) Functions for optimal experimental design. Each section begins with function definitions, followed by example R code that demonstrates the preceding functionality. The example at the end of section 5 demonstrates the optimal design process and before-and-after augmented sensitivity plots. The examples must be executed in sequence, and some examples may require a few minutes to compute.
The tool is entirely self-contained with examples from a cardiac electrophysiological model. The user can modify the equations representing the cell kinetics, stimulation protocol, and numerical solvers to generate the corresponding output for any cardiac model.
A brief overview of tool and instructions for implementing the R code are provided in the file:
Shotwell_Gray_Optimal-Design-Dynamic-Multi-scale-Models-20151229-web-supplement.pdf.
The code is provided in the file:
Shotwell_Gray_Optimal-Design-Dynamic-Multi-scale-Models-20151229-code-supplement.R
A comprehensive explanation and justification of the approach as well as validation are provided in the original publication by Shotwell and Gray referenced in the testing section.
Relevant FDA guidance documents and FDA-recognized standards include:
- FDA guidance “Reporting of Computational Modeling Studies in Medical Device Submissions”
- FDA draft guidance “Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions”
ASME V&V40-2018, “Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical DevicesExternal Link Disclaimer"