System and method for multimodal human-vehicle interaction and belief tracking

摘要

A method and system for multimodal human-vehicle interaction including receiving input from an occupant in a vehicle via more than one mode and performing multimodal recognition of the input. The method also includes augmenting at least one recognition hypothesis based on at least one visual point of interest and determining a belief state of the occupant's intent based on the recognition hypothesis. The method further includes selecting an action to take based on the determined belief state.