New Machine Learning Approaches for Modeling Time-to-Event Data

National Science Foundation Award # 1527827

Project Duration: 09/01/15-08/31/18

Principal Investigator:

Chandan Reddy

Graduate Students:

Yan Li

Bhanukiran Vinzamuri

Ping Wang

Mahtab Fard

Project Summary:

Due to the advancements in recent data collection technologies, different disciplines have attained the ability to not only accumulate a wide variety of data but also to monitor observations over longer periods of time. In many real-world applications, the primary goal of monitoring these observations is to better estimate the time for a particular event of interest to occur. Examples of these events include disease recurrence in healthcare, time to default in finance, device failure in engineering, etc. A major challenge with such time-to-event data is that it is often incomplete; some data instances are either removed or become unobservable over a period of time before the event occurs. Due to this missing piece of information, standard statistical and machine learning tools cannot readily be applied to analyze such data. Survival analysis methods, primarily developed by the statistics community, aim to model time-to-event data and are usually more effective compared to the standard prediction algorithms as they directly model the probability of occurrence of an event in contrast to assigning a nominal label to the data instance. More importantly, they can implicitly handle missing data. However, in many practical scenarios, the missing data challenges are compounded by several other related complexities such as the presence of correlations within the data, temporal dependencies across multiple instances (collected over a period of time), lack of available information from a single source, and difficulty in acquiring sufficient event data in a reasonable amount of time. Such data poses unique challenges to the field of predictive analytics and thus creates opportunities to develop new algorithms to tackle these issues. This project provides innovative computational methods to assist novel scientific discoveries and bring practical transformational impact to the analysis and exploration of various time-to-event datasets and applications. The proposed methods are primarily being evaluated in the context of biomedical data, but are applicable to various other forms of time-to-event data that is often seen in other disciplines such as social science, engineering, finance, and economics. This project builds novel computational and analytical algorithms that can efficiently and accurately capture the underlying predictive patterns in time-to-event data. The project aims at building new algorithms for longitudinal data analysis, integrate multiple sources while building time-to-event models, and predict temporal events with limited amount of training data. The progress of the project and the research findings are disseminated via the project website (http://dmkd.cs.vt.edu/projects/survival/).

Journal Articles and Conference Papers:

• Yan Li, Lu Wang, Jie Wang, Jieping Ye, and Chandan K. Reddy, "Transfer Learning for Survival Analysis via Efficient L2,1-norm Regularized Cox Regression", In Proceedings of the IEEE International Conference on Data Mining (ICDM), Barcelona, Spain, December 2016. [pdf]
  

• Sattar Ameri, Mahtab J. Fard, Ratna B. Chinnam and Chandan K. Reddy, "Survival Analysis based Framework for Early Prediction of Student Dropouts", In Proceedings of the ACM Conference on Information and Knowledge Management (CIKM), Indianapolis, IN, October 2016. [pdf]
  

• Mahtab J. Fard, Ping Wang, Sanjay Chawla, and Chandan K. Reddy, "A Bayesian Perspective on Early Stage Event Prediction in Longitudinal Data", IEEE Transactions on Knowledge and Data Engineering (TKDE), 2016. [pdf] in press
  

• Yan Li, Jie Wang, Jieping Ye and Chandan K. Reddy, "A Multi-Task Learning Formulation for Survival Analysis", In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), San Francisco, CA, August 2016. [pdf]
  

• Yan Li, Kevin Xu, and Chandan K. Reddy , "Regularized Parametric Regression for High-dimensional Survival Analysis", In Proceedings of SIAM International Conference on Data Mining (SDM), Miami, FL, May 2016. [pdf]
  

• Yan Li, Bhanukiran Vinzamuri, Chandan K. Reddy, "Regularized Weighted Linear Regression for High-dimensional Censored Data", In Proceedings of SIAM International Conference on Data Mining (SDM), Miami, FL, May 2016. [pdf]
  

• Mahtab J. Fard, Sanjay Chawla, and Chandan K. Reddy, "Early-Stage Event Prediction for Longitudinal Data", In Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), Auckland, New Zealand, April 2016. [pdf]
  

• Yan Li, Vineeth Rakesh, and Chandan K. Reddy, "Project Success Prediction in Crowdfunding Environments", In Proceedings of the ACM International Conference on Web Search and Data Mining (WSDM), San Francisco, CA, February 2016. [pdf]
  

• Bhanukiran Vinzamuri, Yan Li and Chandan K. Reddy, "Active Learning Based Survival Regression for Censored Data", In Proceedings of the ACM Conference on Information and Knowledge Management (CIKM), Shanghai, China, November 2014. [pdf]
  

• Bhanukiran Vinzamuri and Chandan K. Reddy, "Cox Regression with Correlation based Regularization for Electronic Health Records", In Proceedings of the IEEE International Conference on Data Mining (ICDM), Dallas, TX, December 2013. [pdf]
  

Tutorial Presented:

Will be made available soon.

Software Download:

Machine Learning for Survival Analysis

Point of Contact: Chandan Reddy : reddy (at) cs (dot) vt (dot) edu

Date of Last Update: 9/30/16