Determining desired job plan based on previous inquiries in a stream processing framework

摘要

A data stream processing system is provided that utilizes independent sites to process user-defined inquires over dynamic, continuous streams of data. A mechanism is provided for processing these inquiries over the continuous streams of data by matching new inquiries to previously submitted inquiries. The job plans containing sets of processing elements that were created for both the new inquiry and the previous inquiries are compared for consistency in input and output formatting and commonality of processing elements used. In accordance with the comparison, the new job plan, previous job plans or a combination of the new and previous job plans are used to process the new inquiry. Based on the results of processing the new inquiry, a determination is made regarding which job plans are used for future inquiries.