A Big Data Driven Approach to Extracting Global Trade Patterns

摘要

Unlike roads, shipping lanes are not carved in stone. Their size, boundaries and content vary over space and time, under the influence of trade and carrier patterns, but also infrastructure investments, climate change, political developments and other complex events. Today we only have a vague understanding of the specific routes vessels follow when travelling between ports, which is an essential metric for calculating any valid maritime statistics and indicators (e.g. trade indicators, emissions and others). Whilst in the past though, maritime surveillance had suffered from a lack of data, current tracking technology has transformed the problem into one of an overabundance of information, as huge amounts of vessel tracking data are slowly becoming available, mostly due to the Automatic Identification System (AIS). Due to the volume of this data, traditional data mining and machine learning approaches are challenged when called upon to decipher the complexity of these environments. In this work, our aim is to transform billions of records of spatiotemporal (AIS) data into information for understanding the patterns of global trade by adopting distributed processing approaches. We describe a four-step approach, which is based on the MapReduce paradigm, and demonstrate its validity in real world conditions.