Method and apparatus for maintaining and processing provenance data in data stream processing system

摘要

Techniques are disclosed for maintaining and processing provenance data in such data stream processing systems. For example, a method for processing data associated with a data stream received by a data stream processing system, wherein the system comprises a plurality of processing elements, comprising the following steps. A portion of data associated with the data stream is maintained. The maintained data comprises inputs to each processing element that contributed to an output of each processing element. In response to an alert generated by one of the processing elements, a scheduler is triggered to determine when a pre-calculation of a prospective query related to the alert should be executed. In response to the scheduler, at least a portion of the maintained data is used to determine a set of data that contributed to the alert such that the alert-contributing set of data can be used to respond to the prospective query upon arrival thereof.