Significance analysis of high-dimensional, low-sample size partially labeled data

摘要

Purpose:To propose a method for classification/clustering based on significance analysis of high dimensional, low-sample size data where only a small portion of the class labels (partially labeled) are available.Summary:It is highlighted about the role played by classification and clustering activities in statistical learning. Testing the difference between two classes is quite challenging when there is high-dimensional, low-sample size (HDLSS) data. While there are approaches to deal with such data, the problem becomes difficult when there are observations with many not having class labels (partially labeled data). The article develops a significance testing method for the HDLSS partially labeled data. Two significance analysis methods are considered:> DiProPerm test that is applicable when all class labels are known, and> Statistical significance clustering test (SigClust) that does not require a labelA detailed review of these test methods is presented from the perspective of their application to HDLSS data and the proposed test method for the significance analysis of HDLSS partially labeled (SigPal) data is presented. Some theoretical results are studied with an emphasize on an HDLSS data setting. In order to illustrate the proposed test method, a comprehensive simulation study is considered. A real time application to breast cancer data is also studied do demonstrate the usefulness of the proposed method and the results are discussed. (41 refs.) Results:While classification and clustering activities are important tools in statistical learning, their successful application depends on the nature of data on hand. Generally, in the case of classification, class labels are provided prior to the analysis, while such labels are unavailable in the clustering analysis. Also there are situations where the high-dimensional, low-sample size (HDLSS) data need to be dealt with, and this becomes more challenging.