Big data efficiency analysis: Improved algorithms for data envelopment analysis involving large datasets

摘要

In general, data sets are growing larger and larger, and handling related issues is topic of big data. Similar trends and tendencies are evident in data envelopment analysis (DEA). DEA is a well-known instrument for determining the efficiencies of decision-making units (DMUs), applying linear programming. Still, as we will show, DEA suffers notably from the curse of dimensionality. Therefore, we propose improved decomposition-based algorithms involving different termination criteria and multithreading to address this issue. For some of these criteria, we prove the convergence of the algorithm; to the best of our knowledge, we are the first to prove this. Ultimately, from a computational point of view, we study the performance of the new big data strategy by an extensive numerical analysis, thus demonstrating the algorithm's scalability.