Human action recognition using convolutional LSTM with three-time variables

Subject Areas : Journal of Computer & Robotics

Arash Asefnejad ¹ , Javad Mohammadzadeh ² , Mitra Mirzarezaee ³

1 - Department of Computer Engineering, Science and Research branch, Islamic Azad University, Tehran, Iran
2 - Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, Iran
3 - Department of Computer Engineering, Science and Research branch, Islamic Azad University, Tehran, Iran

Received: 2023-10-11 Accepted : 2024-05-02 Published : 2024-08-12

Keywords: Action recognition, deep neural networks, LSTM, CNN,

Abstract :

With the appearance of deep neural networks, and at the head of it, convolutional neural networks, a great revolution in machine vision was created. Also, the growth of video data and the need for automated processing of this data type have made deep neural network usage increasingly important. There are several methods to recognize the type of movement in the videos. One of the methods is using LSTM and a convolutional neural network in order to extract the time dependencies from video images more accurately. In this study, we present an extended version of the LSTM that can learn longer temporal dependencies. Besides the convolutional neural network, our extended version of the LSTM forms a strong structure to recognize human activity. The results of this study on data set UCF 101 and HMDB51 show that the presented architecture, with a performance accuracy of 96.28 on data set UCF101 and 78.02 on data set HMDB51, performs better than the most similar methods.