Journal article
Continuous gesture segmentation and recognition using 3DCNN and convolutional LSTM
IEEE Transactions on Multimedia, Vol.21(4), pp.1011-1021
2019
Abstract
Continuous gesture recognition aims at recognizing the ongoing gestures from continuous gesture sequences and is more meaningful for the scenarios, where the start and end frames of each gesture instance are generally unknown in practical applications. This paper presents an effective deep architecture for continuous gesture recognition. First, continuous gesture sequences are segmented into isolated gesture instances using the proposed temporal dilated Res3D network. A balanced squared hinge loss function is proposed to deal with the imbalance between boundaries and nonboundaries. Temporal dilation can preserve the temporal information for the dense detection of the boundaries at fine granularity, and the large temporal receptive field makes the segmentation results more reasonable and effective. Then, the recognition network is constructed based on the 3-D convolutional neural network (3DCNN), the convolutional long-short-term-memory network (ConvLSTM), and the 2-D convolutional neural network (2DCNN) for isolated gesture recognition. The “3DCNN-ConvLSTM-2DCNN” architecture is more effective to learn long-term and deep spatiotemporal features. The proposed segmentation and recognition networks obtain the Jaccard index of 0.7163 on the Chalearn LAP ConGD dataset, which is 0.106 higher than the winner of 2017 ChaLearn LAP Large-Scale Continuous Gesture Recognition Challenge.
Details
- Title
- Continuous gesture segmentation and recognition using 3DCNN and convolutional LSTM
- Authors/Creators
- G. Zhu (Author/Creator)L. Zhang (Author/Creator)P. Shen (Author/Creator)J. Song (Author/Creator)S.A.A. Shah (Author/Creator)M. Bennamoun (Author/Creator)
- Publication Details
- IEEE Transactions on Multimedia, Vol.21(4), pp.1011-1021
- Publisher
- IEEE
- Identifiers
- 991005545198907891
- Copyright
- © 2019 IEEE
- Murdoch Affiliation
- Murdoch University
- Language
- English
- Resource Type
- Journal article
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Metrics
105 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- International collaboration
- Citation topics
- 4 Electrical Engineering, Electronics & Computer Science
- 4.116 Robotics
- 4.116.1097 Gesture Recognition
- Web Of Science research areas
- Computer Science, Information Systems
- Computer Science, Software Engineering
- Telecommunications
- ESI research areas
- Computer Science