A physical operator algebra for prioritized elements in data streams

摘要

Data stream management systems are a natural choice to efficiently process continuous queries over high volume data streams, e.g., to monitor sensor data or transaction streams. An immediate reaction on detected critical or security relevant situations is essential for a secure and economic operation, as in our scenario of monitoring decentralized energy systems, which realize geographically distributed energy generation processes. Without further provisions existing processing approaches may lead to a delay of critical or security relevant messages in high load situations, e.g., caused by bursts.rnOne way to allow an adequate processing in such situations is to prioritize queries that handle critical situations. Unfortunately, problems are not always solely identifiable by a query. Sometimes certain - e.g., out of range - data values or error messages indicate situations, which urgerna faster processing of all queries processing these data. Traditional approaches on continuous query execution assume a stream order, typically based on timestamps, and a processing following this order. In this article we consider the prioritization of those elements and propose an out-of-order execution in the data stream.rnWe provide a comprehensive and formally founded approach for prioritizing data stream elements. Prioritized elements benefit twice from our approach. On the one hand, they are able to "overtake" lower prioritized elements, e.g., in queues. On the other hand, prioritized results can be produced earlier in stateful operators than this would be possible in other approaches. Still, the semantics of the queries remains unchanged. We implemented our approach and show with measurements that a very low latency of prioritized elements can be achieved - even under high load. As a result, all queries that process prioritized elements can benefit from our approach.