High dimensional data analysis methods how to#

The rest of the chapter is arranged as follows. It aims to expand the application scope of sparse boosting and develop new methods of high-dimensional survival analysis, longitudinal data analysis, and subgroup analysis, which has great application prospects.

High dimensional data analysis methods how to#

This chapter intends to solve the problem of how to improve the accuracy and calculation speed of variable selection and parameter estimation in high-dimensional data. Finally, a multi-step sparse boosting method to identify patient subgroups that exhibit different treatment effects is studied for the high-dimensional dense longitudinal observations. Then, a two-step sparse boosting to carry out the variable selection and the model-based prediction is studied for the high-dimensional longitudinal observations measured repeated over time. First, a sparse boosting method to select important biomarkers is studied for the right censored survival data with high-dimensional biomarkers. In this chapter we review some sparse boosting based methods for the following high-dimensional problems based on three research papers. Besides these extensions, much effort has been made in understanding the advantages of boosting such as relatively lower over-fitting risk, smaller computational cost, and simpler adjustment to include additional constraints. See, for example, BГјhlmann and Hothorn for the twin boosting Komori and Eguchi for the pAUCBoost Wang for the twin HingeBoost Zhao for the GSBoosting and Yang and Zou for the ER-Boost.

Recently, more various versions of boosting algorithms have been proposed. For example, AdaBoost has the exponential loss function, L2 boosting has the squared error loss function, sparse boosting has the penalized loss function and HingeBoost has the weighted hinge loss function. The major difference among different boosting algorithms is the loss function. The original boosting algorithms were proposed by Schapire which is an ensemble method that iteratively combines weaker learners to minimize the expected loss.

In recent years, machine learning methods such as boosting have become very prominent for high-dimensional data settings since they can improve the selection accuracy substantially and reduce the chance of including irrelevant features. However, those penalized estimation approaches all have some tuning parameters required to be selected by computationally expensive methods like cross-validation. Among all developments, regularization procedures such as LASSO, smoothly clipped absolute deviation (SCAD), MCP and their various extensions have been thoroughly studied and widely used to perform variable selection and estimation simultaneously in order to improve the prediction accuracy and interpretability of the statistical model. Thus, variable selection is crucial to address the challenges. It is widely known that including irrelevant predictors in the statistical model may result in unstable estimation and dreadful computing issues. In the past decades, researchers have done a great deal of high-dimensional data analysis where the sample size nis relatively small but the number of features punder consideration is extremely large. High-dimensional model has become very popular in statistical literature and many new machine learning techniques have been developed to deal with data with very large number of features.

Then, a two-step sparse boosting method to carry out the variable selection and the model-based prediction is studied for the high-dimensional longitudinal observations measured repeatedly over time. This chapter discusses sparse boosting based machine learning methods in the following high-dimensional problems. However, these methods typically require time-consuming cross-validation methods to select tuning parameters and retain more false positives under high dimensionality. In high-dimensional data, penalized regression is often used for variable selection and parameter estimation.