A Unified Approach to Detect the Record Duplication Using BAT Algorithm and Fuzzy Classifier for Health Informatics

摘要

Generally, data cleaning techniques are employed in data engineering applications to improve the quality Of data. Besides, removing duplicate records after the identification from the medical data sets are performed by data cleansing for improving the quality of medical data and that lights up the disease diagnosis and health wellbeing to the next level. Likely, significant efforts have been already made by the different researchers to identify the duplicate records from medical data using soft computing techniques. Accordingly, a genetic programming approach was given recently into record de-duplication that combined several different pieces of evidence extracted from the data content. In this paper, it is planned to develop a data duplication identification technique using soft computing methods for improved data assessment in medical data. The major problem in the field of medical data and other health information is the persistence of multiple data or redundant data. Thus in the proposed method, an algorithms based on bat algorithm and fuzzy classifier is addressed. In the method, the selection of pieces can reduce the time consumption in testing since the small information is enough to do the duplication detection task using the defined techniques. The overall steps of the proposed data duplication technique is explained in three steps, such as Extraction of pieces of information from the input data, Selection of best set of pieces through BAT algorithm and detection of duplication using fuzzy classifier. At first, the input data is given to hashing-based algorithm and Levenshtein distance to find different set of features. Then, bat algorithm which is one of the recent optimization algorithm to select the best set of features from the initial sets. Then, from the optimal set of features, the detection will be done using fuzzy classifier. The experimentation will be done using four different datasets where two are from health informatics to evaluate the performance in medical application and the performance of the algorithm will be compared with the existing algorithm using detection accuracy. The experimentations showed that, the accuracy rate obtained for the proposed approach is .906 while .79 for the existing approach based on genetic algorithm.