Utilising Amazon web services to provide an on demand urgent computing facility for climateprediction.net

摘要

climateprediction.net has traditionally been an activity that requires a large amount of computing resources from its volunteer network, whilst allowing a time-frame of weeks to months for simulations to be returned for each project. However, there is an increasing trend of projects requiring results in shorter and shorter timescales. Under no project is this clearer than in the World Weather Attribution (WWA) initiative, where we are aiming to provide in near to real-time an answer to how anthropogenic climate change has altered the frequency of occurrence of a particular type of extreme weather event, either as it happens or as soon after as is practical. As such we need the ability to run simulations on alternate resources when volunteer resources will not provide results within the necessary timeframe. This paper describes a workflow to distribute ensembles of climateprediction.net simulations in the Amazon Elastic Compute Cloud, to provide urgent compute capability for projects such as WWA. We propose a method of optimizing the use of cloud resources to minimize cost while maximising throughput. A case study is presented to provide a proof of concept of this methodology. As such, this is a clear example of beneficial utilisation of cloud resources to supplement those available through our volunteer community.