Catalog of Regulatory Science Tools to Help Assess New Medical Devices
This regulatory science tool presents a computer model that demonstrates the application of gold-standard method of manufactured solutions (MMS) code verification to verify a commercial finite element code for elastostatic solid mechanics analyses relevant to medical devices.
Technical Description
This tool provides a Jupyter/Python notebook template for generating source terms required for rigorous method of manufactured solutions (MMS) code verification. The tool uses Python and the open-source computer algebra library SymPy for symbolic mathematics.
- User inputs:
- governing equations in a closed-form representation
- a prescribed solution for the field variable(s) of interest
- governing equations in a closed-form representation
- Tool outputs:
- analytical source term functions for MMS code verification
- analytical source term functions for MMS code verification
The template includes example governing equations and source terms for verifying elastostatic solid mechanics analyses involving linear-elastic, hyperelastic (neo-Hookean), and quasi-hyperelastic (Hencky) constitutive models in a commercial finite element code.
Intended Purpose
This tool is intended to facilitate MMS code verification of physics-based computational models. Code verification is the first step in demonstrating the credibility of computational models and is an important consideration for any model intended to influence decision making during medical device development or regulatory review.
The tool is applicable to any medical device development or performance assessment activities that may benefit from physics-based computational modeling, especially solid mechanics analyses of implantable devices. Intended users include simulation code developers and simulation analysts.
Testing
This tool has been developed following practices for numerical code verification recommended in ASME V&V10-2019 Section 5.1.1.2 and ASME V&V20-2009(R2021) Section 2-3.3.
Use of the tool has been demonstrated for a commercial code in the following peer-reviewed publication:
- Aycock, K. I., Rebelo, N., & Craven, B. A. (2020). Method of manufactured solutions code verification of elastostatic solid mechanics problems in a commercial finite element solver. Computers & Structures, 229, 106175. https://doi.org/10.1016/j.compstruc.2019.106175.
Limitations
This tool requires user-provided governing equations for a computational model of interest in a closed form representation. In some circumstances, the precise form of the governing equations for a given computational model may be unknown, especially when using a closed-source simulation software that lacks comprehensive documentation. Moreover, some computational models include terms that lack a closed-form representation, for example, use of an empirical submodel, or reliance on a rate-based formulation with history dependence. For such applications, alternative code verification approaches may be necessary.
Supporting Documentation
Documentation is provided as a Jupyter notebook in Supplemental Material to Aycock et al. 2020, Comput Struct.
A static PDF version of the notebook is available in the following link for users unfamiliar with the Jupyter environment.
Tool users are encouraged to download the full package and extract the .ipynb notebook from the zip archive (https://doi.org/10.6084/m9.figshare.7726352.v2).
Use of the tool requires installation of free and open-source software including Python 3, NumPy, pandas, SymPy, Matplotlib, and Jupyter, which are available through installation of the Anaconda Distribution.
FDA guidance documents include:
- FDA guidance “Reporting of Computational Modeling Studies in Medical Device Submissions”: Section III of the guidance recommends sponsors provide a brief description of the numerical code verification performed and specifically mentions the MMS.
- FDA guidance “Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions”: The guidance describes ten categories of credibility evidence for supporting the credibility of computational modeling used in regulatory submissions. The first evidence category introduced is “Code verification results” (see Table 1 and Section VI.B(1) therein).
FDA-recognized standards include:
- ASME V&V40-2018, “Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices”: The ASME V&V40 standard introduces the concept of “credibility factors” that are used to semi-quantitatively assess the quality of V&V activities used to support the credibility of computational modeling and simulation results. Section 5.1.1.2 of the standard addresses numerical code verification and presents “order of accuracy assessment” (generally performed using MMS) as the highest credibility level in the example gradation of code verification activities. See FDA’s Recognized Consensus Standards database.
Additional standards include:
- ASME V&V10-2019, “Standard for Verification and Validation in Computational Solid Mechanics”: The ASME V&V standards provide recommendations for performing rigorous V&V to support computational modeling results. Section 5.1 of the V&V10 standard addresses code verification, and Section 5.1.1.2 addresses MMS specifically.
- ASME V&V20-2009(R2021), “Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer”: Section 2-3 of the V&V20 standard addresses code verification, and Section 2-3.3 addresses MMS specifically. Code verifications examples are also provided in Section 7-2 and Nonmandatory Appendices A and C-2.
Further recommended reading:
- Oberkampf, W. L., & Roy, C. J. (2010). Verification and validation in scientific computing. Cambridge University Press. https://doi.org/10.1017/CBO9780511760396
- Section 5: Code verification
- Section 6.3: Method of manufactured solutions (MMS)
- Section 5: Code verification
- Roache, P. J. (2009). Fundamentals of verification and validation. Hermosa Publishing.
- Chapter 3: Method of manufactured solutions: A methodology for verification of codes
- Chapter 3: Method of manufactured solutions: A methodology for verification of codes
- Roache, P. J. (2002). Code verification by the method of manufactured solutions. J. Fluids Eng., 124(1), 4-10. https://doi.org/10.1115/1.1436090
- Salari, K., & Knupp, P. (2000). Code verification by the method of manufactured solutions (No. SAND2000-1444). Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Sandia National Lab. (SNL-CA), Livermore, CA (United States). https://doi.org/10.2172/759450