Research of Distributed Computing Framework for Foreign Exchange Market Monitoring

摘要

Foreign exchange market monitoring is an important part of financial regulation, which amends the failures of market mechanism, protect the legal rights of market participants and maintain the equality, efficiency, transparency and stability of the foreign exchange market. This paper analyzes and summarizes the business characteristics and existing technology framework of foreign exchange market monitoring, and proposes a novel distributed computing framework for this task based on Spark. Considering the business features of single-market multi-indicator and multi-market multi-indicator, this framework involves market-level directed acyclic graph model (M-DAG) and Market-level resource allocation strategy (M-YARN) based on Spark's job scheduling mechanism and YARN's resource scheduling pool isolation mechanism, respectively. The experimental results prove the performance of our proposed framework in foreign exchange market monitoring, which effectively guarantee the completeness, accuracy and timeliness of foreign exchange market monitoring indicators under the background of big data.