Big Data: Technologies and Algorithms to Deal with Challenges

摘要

In this age, big data applications are increasingly becoming the main focus of attention because of the enormous increment of data generation and storage that has taken place in the last years, in science, business, . . . This situation becomes a challenge when huge amounts of data are processed to extract knowledge because the data mining techniques are not adapted to the new space and time requirements. We must consider the new paradigms to develop scalable algorithms. At this conference we will introduce briefly the technologies that have emerged strongly in recent years (Hadoop ecosystem, Spark, . . .) and the libraries such as Mahout, ML lib, . . . We will discuss some applications, and we will focus the attention on the steps to create a learning algorithms that was winner for the ECBDL'14 big data competition, processing an extremely imbalanced big data bioinformatics problem.