SEGMENTING CONTENT DISPLAYED ON A COMPUTING DEVICE INTO REGIONS BASED ON PIXELS OF A SCREENSHOT IMAGE THAT CAPTURES THE CONTENT

摘要

Methods and apparatus directed to segmenting content displayed on a computing device into regions, to facilitate user interactivity with graphical content. The segmenting of content displayed on the computing device into regions is accomplished via analysis of pixels of a "screenshot image" that captures at least a portion of (e.g., all of) the displayed graphical content. Individual pixels of the screenshot image may be analyzed to determine one or more regions of the screenshot image and to optionally assign a corresponding semantic type to each of the regions. Some implementations are further directed to generating, based on one or more of the regions, interactive content to provide for presentation to the user via the computing device.