Cassandra and SQL database comparison for near real-time Twitter data warehouse

摘要

In the era of Big Data, social media analysis has grown extremely popular. Twitter, one of the most popular social media, is believed to contain many user opinion in its message. Thus, leading organizations start utilizing its data using social media analytic tools to get in-sight of their markets in real-time. However, many Twitter analytic tools are still specified only in some specific tasks. Therefore, in order to enhance the possibility of doing many analysis on Twitter, a data warehouse technology can be utilized to receive, process, and store a real-time Twitter streams. Nonetheless, data warehouse development using relational database start to show its limit on storing big data let alone real-time. Hence, NoSQL (Not Only SQL) technology has emerged as an alternative solution. In this paper, we try to develop near real-time Twitter data warehouse using NoSQL database, Cassandra, and compare its storing and querying performance with that developed using relational databases. The results show that Cassandra performs significantly better in storing data than the relational databases. Meanwhile, in its querying performance, Cassandra is slower while using small data but way faster on vast data.