Correlation of written notes to digital content

摘要

A system and method receives a plurality of data streams from a smart pen device and a computing device, and indexes the data streams to a synchronized time index. A processor receives a first data stream representing gesture data captured by a smart pen device, and a second data stream representing a sequence of states associated with applications executing on a computing device, such that each state identifies content displayed by a computing device while the gesture data is captured. For example, a state could be a particular page of a digital document displayed by the computing device. After receiving the first and second data streams, the processor indexes the data streams to a synchronized time index, and stores the indexed data streams in a memory.