Skip navigation
Please use this identifier to cite or link to this item:
Title: Vehicular Trajectory Classification and Traffic Anomaly Detection in Videos Using a Hybrid CNN-VAE Architecture
Authors: Santhosh K.K.
Dogra D.P.
Pratim Roy, Partha
Mitra A.
Published in: IEEE Transactions on Intelligent Transportation Systems
Abstract: Visual surveillance has become indispensable in the evolution of Intelligent Transportation Systems (ITS). Video object trajectories are key to many of the visual surveillance applications. Classifying varying length time series data such as video object trajectories using conventional neural networks, can be challenging. In this paper, we propose trajectory classification and anomaly detection using a hybrid Convolutional Neural Network (CNN) and Variational Autoencoder (VAE) architecture. First, we introduce a high level features for varying length object trajectories using color gradient representation. In the next stage, a semi-supervised way to annotate moving object trajectories extracted using Temporally Incremental Gravitational Model (TIGM) is used for class labeling. For training, anomalous trajectories are identified using t-Distributed Stochastic Neighbor Embedding (t-SNE). Finally, a hybrid CNN-VAE architecture has been proposed for trajectory classification and anomaly detection. The results obtained using publicly available surveillance video datasets reveal that the proposed method can successfully identify traffic anomalies such as violations in lane driving, sudden speed variations, abrupt termination of vehicle during movement, and vehicles moving in wrong directions. The accuracy of trajectory classification improves by a margin of 1-6% against popular neural networks-based classifiers across various datasets using the proposed high-level features. The gradient representation also improves the anomaly detection accuracy significantly (30-35%). Code and dataset can be found at IEEE
Citation: IEEE Transactions on Intelligent Transportation Systems
Issue Date: 2021
Publisher: Institute of Electrical and Electronics Engineers Inc.
Keywords: Convolutional neural network
deep learning
Dirichlet process mixture model
traffic anomaly detection.
trajectory classification
variational autoencoder
visual surveillance
ISSN: 15249050
Author Scopus IDs: 56525012800
Author Affiliations: Santhosh, K.K., School of Electrical Sciences, Indian Institute of Technology Bhubaneswar, Odisha 752050, India (e-mail:
Dogra, D.P., School of Electrical Sciences, Indian Institute of Technology Bhubaneswar, Odisha 752050, India.
Roy, P.P., Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee 247667, India.
Mitra, A., Centre of Excellence in Artificial Intelligence (AI), Indian Institute of Technology Kharagpur, Kharagpur 721302, India.
Appears in Collections:Journal Publications [CS]

Files in This Item:
There are no files associated with this item.
Show full item record

Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.