Generalized Jaccard Similarity Based Multilevel Threshold Affinity Propagated Clustering For Big Data Analytics

摘要

Clustering the huge amount of information in very large datasets is a difficult problem to be solved in data mining. Few research works have been designed for grouping similar types of data in big dataset with aid of different data mining concepts. The computational cost of conventional affinity propagation clustering technique is expensive in terms of memory space and time complexities when considering large size of dataset as input. In order to overcome these limitations, Generalized Jaccard Similarity based Multilevel Threshold Affinity Propagated Clustering (GJS-MTAPC) Technique is proposed. The GJS-MTAPC Technique is an improved Affinity propagation (AP) algorithm to increase the clustering performance of big data with minimal false positive rate and minimal computational cost. The GJS-MTAPC Technique splits big dataset which is to be clustered into a number of subsets. After dividing dataset, GJS-MTAPC Technique chooses exemplars of each subset randomly. Then, GJS-MTAPC Technique identifies best exemplars by transmitting responsibility and availability messages among data samples in each subset. Finally, GJS-MTAPC Technique defines multiple threshold values in order to precisely cluster data samples in a large dataset based on similarity values. As a result, GJS-MTAPC Technique provides better big data clustering processes in terms of clustering accuracy, computational cost and space complexity and false positive rate. The experimental result show that GJS-MTAPC Technique is able to increases the clustering accuracy and also minimizes the computational cost of big data analytics as compared to state-of-the-art works.