FAULT RECOVERY METHOD IN SPARK STREAMING BASED APPARATUS AND DISTRIBUTED APPARATUS SUPPORTING REAL-TIME PROCESSING

摘要

A fault recovery method in a spark streaming-based apparatus includes the steps of: storing input streaming data, cumulative deformation history and state data in a unit of a reference time by the spark streaming-based apparatus; accumulating, by the apparatus, the total fault recovery time while estimating a fault recovery time for the input streaming data based on a deformation time for the input streaming data; determining, by the apparatus, target state data which is state data for the most recently inputted streaming data among the stored state data, when the total fault recovery time exceeds a threshold; and deleting, by the apparatus, at least one among state data generated and stored before the target state data and at least one among input streaming data inputted and stored before the target state data is generated.