Journal article
Leveraging structural context models and ranking score fusion for human interaction prediction
IEEE Transactions on Multimedia, Vol.20(7), pp.1712-1723
2017
Abstract
Predicting an interaction before it is fully executed is very important in applications such as human-robot interaction and video surveillance. In a two-human interaction scenario, there often contextual dependency structure between the global interaction context of the two humans and the local context of the different body parts of each human. In this paper, we propose to learn the structure of the interaction contexts, and combine it with the spatial and temporal information of a video sequence for a better prediction of the interaction class. The structural models, including the spatial and the temporal models, are learned with Long Short Term Memory (LSTM) networks to capture the dependency of the global and local contexts of each RGB frame and each optical flow image, respectively. LSTM networks are also capable of detecting the key information from the global and local interaction contexts. Moreover, to effectively combine the structural models with the spatial and temporal models for interaction prediction, a ranking score fusion method is also introduced to automatically compute the optimal weight of each model for score fusion. Experimental results on the BIT-Interaction and the UT-Interaction datasets clearly demonstrate the benefits of the proposed method.
Details
- Title
- Leveraging structural context models and ranking score fusion for human interaction prediction
- Authors/Creators
- Q. Ke (Author/Creator)M. Bennamoun (Author/Creator)S. An (Author/Creator)F. Sohel (Author/Creator)F. Boussaid (Author/Creator)
- Publication Details
- IEEE Transactions on Multimedia, Vol.20(7), pp.1712-1723
- Publisher
- IEEE
- Identifiers
- 991005543914907891
- Copyright
- © Copyright 2017 IEEE
- Murdoch Affiliation
- School of Engineering and Information Technology
- Language
- English
- Resource Type
- Journal article
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Metrics
36 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- Citation topics
- 4 Electrical Engineering, Electronics & Computer Science
- 4.116 Robotics
- 4.116.1097 Gesture Recognition
- Web Of Science research areas
- Computer Science, Information Systems
- Computer Science, Software Engineering
- Telecommunications
- ESI research areas
- Computer Science