As video delivery systems transition to all IP, they are becoming massively scaled distributed systems. Comcast's video delivery systems span hundreds of sites, and failures in one site can have unexpected negative imapcts on video delivery. When troubleshooting these sorts of systems, it's useful to have as broad, but often shallow, operational view of the system as a whole. In addition, these systems generate data essential for business intelligence, capacity planning, reccomendations, and a whole variety of other essential functions. Without a system to methodically collect data from across this infrastructure, data collection is usually done via a set of ad-hoc integrations, usually with log files, or, at best, custom telemetry collection schemes. This leads to a stew of ever-shifting data formats, which much be parsed and reconciled to make sense of the system as a whole In this paper, we present an architecture for a stream data platform which allows us to comphrehensively collect high value data, for both operational and other business purpouses, at large scale. In addition, we present a method of defining and evolving schemas for this data.
展开▼
