Deep Learning of User Behavior in Shared Spaces

摘要

Learning how road users behave is essential for the development of many intelligent systems, such as traffic safety control, intelligent transportation systems, and self-driving cars. However, automated and accurate recognition of road users' behavior is still one of the bottlenecks in realizing such systems in city traffic that is-compared to other types of traffic-especially dynamic and full of uncertainties. There are some urban environments which make detection and prediction of road users' behavior particularly challenging, e.g., temporarily shared spaces of intersections for vehicle turning or shared spaces as a traffic design. The former allow vehicles to turn and interact with other crossing road users, the latter intended to make different types of road users share the space, therefore reducing the dominance of vehicles and improving pedestrian movement and comfort. Direct interactions between vehicles and vulnerable road users (VRUs, e.g., pedestrians and cyclists) and ambiguous traffic situations (e. g., road users negotiating usage of the road) make road users' behavior stochastic and difficult to predict.With the development of deep learning techniques and the availability of large-scale real traffic data, this thesis proposes deep conditional generative models for automated interaction detection in the temporarily shared spaces of intersections, as well as for trajectory prediction in shared spaces as a traffic design. Models based on Conditional Variational Auto-Encoder (CVAE) are trained to map deterministic input (e.g., a sequence of video data or a segment of an observed trajectory) to many possible outputs of road users' behavior, characterized by their interaction or their movement in the next seconds.This thesis makes two main contributions to the research on modeling road users' behavior in shared spaces using deep learning approaches:(1) The interaction detection model takes the information of road users' type, position, and motion-all of which have been automatically extracted by deep learning object detectors and optical flow from video data-as input, and generates a frame-wise probability that represents the dynamics of interaction between a turning vehicle and any VRUs involved. The model's efficacy was proven by testing on real-world datasets acquired from two different intersections. It achieved an F1-score above 0.96 at a right-turn intersection in Germany and 0.89 at a left-turn intersection in Japan, both with very busy traffic flows.(2) Various factors and state-of-the-art deep learning architectures are investigated for trajectory prediction. In this thesis, three frameworks based on CVAE are proposed for accurate multi-path trajectory prediction of heterogeneous road users in shared spaces as a traffic design. The latent space of the CVAE is trained for encoding stochastic behavior patterns and the multi-sampling process from the trained latent space enables the frameworks to generate not only one deterministic future trajectory, but multiple possible future trajectories for each road user. The first framework focuses on studying multiple contexts, namely motion, interaction, pedestrian grouping, and different types of environmental scene context for trajectory prediction. The second and third frameworks focus on exploring dynamic context (i. e., motion and interaction) using attention mechanisms, and improving the models' generalizability-predicting trajectories of heterogeneous road users in various shared spaces that have not been used to train the models. All of the frameworks, but the second and third in particular, showed superior performance on various popular open-source datasets and benchmarks. The last two frameworks even took first place (in different submission times) in one of the most widely recognized open challenges (TrajNet online test) by reducing the overall average and final displacement errors of the predicted trajectories in the next 4.8 seconds to 0.353 mete