Catalog of Regulatory Science Tools to Help Assess New Medical Devices
This regulatory science tool presents a computer model comprising a set of test problems that can be used for verification of cardiac modeling software, ensuring that the model has been accurately implemented in the software.
Technical Description
Cardiac electrophysiological or electro-mechanical simulation software typically solve well-established equations that govern the propagation of electrical waves through the heart. One activity required for demonstrating the credibility of a computational model is code verification. Code verification is the process of determining if a mathematical model, and algorithms for solving the model, have been correctly implemented in the software. Code verification is challenging with complex models. The tool provides a set of test problems with known analytic solutions, which cardiac model developers can solve using their software and thereby test if the electrophysiology modeling software has been implemented correctly.
The tool provides test problems for the monodomain, bidomain, and bidomain-with-bath equations. The monodomain and bidomain equations are sets of partial differential equations coupled to ordinary differential equations that have been used for many decades to model electrical activity in the heart. The bidomain-with-bath equations are a related set of equations which govern electrical fields generated in the heart and surrounding torso.
Nine test problems are provided for testing the following computational models:
- monodomain in 1D, 2D and 3D;
- bidomain in 1D, 2D and 3D;
- bidomain-with-bath in 1D, 2D, and 3D.
Values of each of the following is specified in each test problem:
- The geometrical domain
- Tissue conductivities, surface-area-to-volume ratio, capacitance
- Sub-model of cellular dynamics
- Initial conditions
- Boundary conditions
- Stimulus current (zero)
The exact analytic solution of test problem is also provided for each test problem.
The user should specify each of the above inputs in their software, solve the model, and compare their solution with the exact solution provided. They can then confirm the correct implementation of their software by verifying that the error converges to zero at the expected convergence rate as the spatial and temporal discretization parameters are reduced.
Intended Purpose
The tool is intended to be used by cardiac model developers. The tool is intended to facilitate code verification of the solver, which is one of the required steps when demonstrating the credibility of a computational model (see Guidances and Standard listed below). The tool is relevant to:
- cardiac electrophysiological models, or models that have an electrophysiological component, such as electro-mechanical models
- models implemented within device software, whether software as a medical device (SaMD) or software in a medical device
- cardiac modeling software that will be used for in silico testing of a cardiac device (for example, in silico evaluation of fracture risk of implantable pacemaker leads)
- organ- or tissue-level cardiac solvers, not single-cell solvers.
Despite the importance of rigorous code verification, there are few tools available to model developers (cardiac or other) for performing code verification. One approach for code verification is to compare results with independently developed software, for example, another cardiac electrophysiological modeling software; however, differences between solvers can be difficult to interpret. Test problems with known solutions, as provided by this tool, allow exact errors to be computed, and if these errors converge to zero at the expected theoretical rates of the convergence, this provides very high confidence that the model has been implemented correctly. This tool is the only set of test problems with analytic solutions that the tool developers are aware of, for cardiac models.
Relevant FDA guidance documents and FDA-recognized standards include:
- Reporting of Computational Modeling Studies in Medical Device Submissions: Guidance for Industry and Food and Drug Administration Staff
- Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions: Draft Guidance for Industry and Food and Drug Administration Staff
- ASME V&V40, “Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices”
Testing
In Pathmanathan and Gray, IJNMBE, 2014, all nine test problems were used to evaluate the cardiac electrophysiology model in Chaste, a general purpose simulation package for computationally demanding problems in biology and physiology. It was confirmed that the numerical error using Chaste converged to zero at the expected rates (specifically: the observed rate of convergence matched the theoretical convergence rate given the linear finite elements used in Chaste). As well as verifying the Chaste solver, these results demonstrate that all nine test problems are formulated correctly, and that they do indeed represent cardiac EP test problems with analytic solutions.
In addition, a major simulation software manufacturer has evaluated their cardiac modeling software using these test problems and observed the theoretical rates of convergence (see report). This confirms there are no typographical errors in the specification of the test problems in Pathmanathan and Gray, IJNMBE, 2014.
Limitations
- The method is not a black box method: it requires specifying quantities that may not be accessible through the user interface of some cardiac modeling software, in which case source code edits will be required.
- The 2D and 3D bidomain-with-bath test problems are essentially 1D problems, although posed in 2D/3D. They therefore do not fully test all aspects of a 2D or 3D implementation of the bidomain-with-bath equations.
Supporting Documentation
Step-by-step instructions for setting up and solving the test problems are available.
Background information regarding the test problems, and application of the test problems to verify the cardiac solver Chaste, are provided in the original publication Pathmanathan and Gray, IJNMBE, 2014.
Contact
For more information:
Note: This tool was previously listed in the catalog as “Benchmark problems for verifying cardiac electrophysiological models.”