RECORDING MULTIMODAL USER INTERACTION WITH ITEMS IN REAL AND DIGITAL ENVIRONMENTS

摘要

A method and a corresponding system are disclosed for recording multimodal user interaction with an item in a networked system. The networked system includes a server device network- connected to one or more client user device(s). A data record is generated for the item, comprising data representative thereof and including at least image data depicting the item. A unique identifier code is associated with the data record, wherein the association of the unique identifier code with the image data in the record defines an avatar of the item in the system. Copies of the avatar are distributed to client user devices in the network, and the unique identifier code is encoded into a machine-readable tag secured to each physical embodiment of the item outside the network. At each client user device, each digital user interaction with an avatar and each analogue user interaction with a physical item is logged. At the server, each logged user interaction is matched against the item data record and recorded.