Systems and/or methods for computer-automated execution of digitized natural language video stream instructions

摘要

Techniques for auto-executing instructions provided in a video on a computing platform are provided. A script is developed from audio provided in the video. Text shown in frames of the video is extracted. Simulated user interaction (UI) events present in the video are identified. A timeline representation is generated to include entries for elements of the script and the extracted text, and identified UI events. Like elements are collected into common entries. Each entry in the script that lacks an associated UI event but is likely to involve a user action prompt is identified. Each entry having an associated identified UI event, and each entry identified as likely to involve a user action prompt, is converted into a corresponding user action command representation. Each user action command representation is mapped to a computing platform executable command, each being performed using processing resources of the computing platform, automatically, without user intervention.