Demographical gender prediction of Twitter users using big data analytics: an application of decision marketing

摘要

Twitter text is difficult to analyse due to the non-standard and unstructured data. Twitter does not accumulate user gender information as do other popular social media platforms. The demographic feature prediction and additional informative content are important for advertising, custom-made marketing and authorised investigation from the social medium. The proposed statistical method with real-time analysis using big data technologies is able to predict the gender of Twitter users. Gender prediction is performed using the naive Bayes classifier to address systemic issues, and Apache Hive is used to solve data cleaning, storage and processing issues. The proposed method is a speedy, easy-to-implement with pre-processing, close to state-of-the-art document text categorisation method using big data technologies.