Processing and Analyzing Big Data Generated from Data Communication and Social Networks: In-terms of Performance Speed and Accuracy

摘要

A multiple layer architecture for sentiment analysis has been discussed in the proposed work which achieves a best accuracy of 89.61% using SVM ML classifier. Obtained results conclude that, a better accuracy has been achieved in the proposed scheme compared to existing schemes in the literature. Predictive analytics has been done against tweets collected during IPL10 cricket 20 overs final match. Existing literature highlight conceptual relationships, physicaal locations or topical changes but the textual is not being visualized primarily. Proposed scheme works on exploring hidden semantics which helps in the visualization of real text content. Also, another work has been done where data processing environment is developed using Apache SPARk which is deployed above an existing YARN cluster. With the proposed setup and tuning of resource allocation strategies, we could analyze 85GB of network trace data within 78 seconds by using a distributed 32 node cluster, having a capacity of 1 terabyte RAM and 256 CPU cores. If processing the same amount of data in traditional systems, it will take around 30 minutes.