Applying software engineering processes for big data analytics applications development

摘要

Developing large scale software projects involves huge efforts at every stage of the software development life cycle (SDLC). This led researchers and practitioners to develop software processes and methodologies that will assist software developers and improve their operations. Software processes evolved and took multiple approaches to address the different issues of the SDLC. Recently big data analytics applications (BDAA) are in demand as more and more data is collected and stakeholders need effective and efficient software to process them. The goal is not just to be able to process big data, but also arrive at useful conclusions that are accurate and timely. Considering the distinctive characteristics of big data and the available infrastructures, tools and development models, we need to create a systematic approach to the SDLC activities for BDAA development. In this paper, we rely on our earlier work identifying the characteristic and requirements of BDAA and use that to propose appropriate models for their development process. It is necessary to carefully examine this domain and adopt the software processes that best serve the developers and is flexible enough to address the different characteristics of such applications.