U.S. flag An official website of the United States government

U.S. Food and Drug Administration

TorchSurv: Deep Learning Tools for Survival Analysis

Catalog of Regulatory Science Tools to Help Assess New Medical Devices

This regulatory science tool is a tool used for developing and evaluating deep learning-based survival models.

Technical Description

TorchSurv [1] is a Python library for training and testing artificial intelligence (AI)-based survival models in the PyTorch framework that implements loss functions and evaluation metrics commonly used in survival analysis. The loss functions are differentiable and can be easily incorporated into the training loops of arbitrary PyTorch models. This could be used to train new survival models, or to retrain existing models based on time-to-event data. Currently, the package implements loss functions for the Cox proportional hazard [2] and Weibull accelerated failure time (AFT) [3] models. Survival models could also be evaluated using metrics implemented in the package. This includes the concordance index (C-index) [4], time-dependent AUC [5], and time-dependent Brier score [6]. All metrics implemented in the package are accompanied by confidence intervals. The package repository is hosted on the GitHub platform and contains testing code and documentation. Minor updates and bug fixes will be addressed through issue requests and pull requests on GitHub.

Intended Purpose 

Survival analysis, or time-to-event analysis, focuses on estimating the time from a starting point to the occurrence of an event. In medical device applications, the event of interest could be sepsis onset, cancer progression, or bone fracture. TorchSurv provides well-documented and tested reference implementations of common loss functions and evaluation metrics for developing deep learning-based survival models. This tool enables medical device developers to train and assess new AI models for survival analysis.

  • Relevant Product Areas: Artificial Intelligence Algorithms, Survival Analysis, Risk Predictors

Testing

Each core method in TorchSurv was tested for correctness on synthetic data, where model outputs were compared with analytically derived value and/or the outputs of other widely used survival analysis packages, such as Scikit-Learn. Additionally, an example use case is included in which the package was applied to develop and evaluate a model trained from the German Breast Cancer Study Group 2 dataset [7].

Limitations

  • (User Expertise) This tool is intended for users with experience in developing AI models in PyTorch. It does not contain graphical user interfaces or other support for standalone use. It is also not intended to be used with other deep learning frameworks, such as TensorFlow or Flax.
  • (Dependency Versions) TorchSurv has minimal dependencies: PyTorch, SciPy, NumPy, and TorchMetrics. However, not all version combinations of these packages were tested. While these are stable libraries and compatibility issues are unlikely, users are encouraged to submit bug reports or request fixes for any incompatibilities through the GitHub repository.

Supporting Documentation

The package is hosted on GitHub (https://github.com/DIDSR/torchsurv). The user manual and detailed documentation is available at: https://opensource.nibr.com/torchsurv/index.html.

  1. M. Monod et al., “TorchSurv: A lightweight package for deep survival analysis.” Journal of Open Source Software, 9(104), 7341.
  2. D. R. Cox, “Regression models and life-tables,” J. R. Stat. Soc. Ser. B Methodol., vol. 34, no. 2, pp. 187–202, 1972.
  3. K. J. Carroll, “On the use and utility of the Weibull model in the analysis of survival data,” Control. Clin. Trials, vol. 24, no. 6, pp. 682–701, 2003.
  4. F. E. Harrell, R. M. Califf, D. B. Pryor, K. L. Lee, and R. A. Rosati, “Evaluating the yield of medical tests,” Jama, vol. 247, no. 18, pp. 2543–2546, 1982.
  5. P. Blanche, J.-F. Dartigues, and H. Jacqmin-Gadda, “Review and comparison of ROC curve estimators for a time-dependent outcome with marker-dependent censoring,” Biom. J., vol. 55, no. 5, pp. 687–704, 2013.
  6. E. Graf, C. Schmoor, W. Sauerbrei, and M. Schumacher, “Assessment and comparison of prognostic classification schemes for survival data,” Stat. Med., vol. 18, no. 17–18, pp. 2529–2545, 1999.
  7. M. Schumacher et al., “Randomized 2x2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. German Breast Cancer Study Group.,” J. Clin. Oncol., vol. 12, no. 10, pp. 2086–2093, 1994.

Contact

Tool Reference