DE-MULTIPLEXING STREAMING DATA FROM A FAST WRITE AHEAD DURABLE LOG INTO SEGMENTS

摘要

Implementations are provided herein for managing streaming data that appended and multiplexed into a durable write ahead replicated log. By writing data in the durable log, large amounts of small writes can be processed quickly. Data in the durable log can be de-multiplexed and packaged into segment containers. Segment containers can serialize stream segment specific data and can be stored in long term storage. Data that has been stored in long term storage can be truncated form the durable write ahead log making room for new data.