Journal article
A Multi-modal, discriminative and spatially invariant CNN for RGB-D object labeling
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.40(9), pp.2051-2065
2017
Abstract
While deep convolutional neural networks have shown a remarkable success in image classification, the problems of inter-class similarities, intra-class variances, the effective combination of multimodal data, and the spatial variability in images of objects remain to be major challenges. To address these problems, this paper proposes a novel framework to learn a discriminative and spatially invariant classification model for object and indoor scene recognition using multimodal RGB-D imagery. This is achieved through three postulates: 1) spatial invariance - this is achieved by combining a spatial transformer network with a deep convolutional neural network to learn features which are invariant to spatial translations, rotations, and scale changes, 2) high discriminative capability - this is achieved by introducing Fisher encoding within the CNN architecture to learn features which have small inter-class similarities and large intra-class compactness, and 3) multimodal hierarchical fusion - this is achieved through the regularization of semantic segmentation to a multi-modal CNN architecture, where class probabilities are estimated at different hierarchical levels (i.e., imageand pixel-levels), and fused into a Conditional Random Field (CRF)- based inference hypothesis, the optimization of which produces consistent class labels in RGB-D images. Extensive experimental evaluations on RGB-D object and scene datasets, and live video streams (acquired from Kinect) show that our framework produces superior object and scene classification results compared to the state-of-the-art methods.
Details
- Title
- A Multi-modal, discriminative and spatially invariant CNN for RGB-D object labeling
- Authors/Creators
- U. Asif (Author/Creator)M. Bennamoun (Author/Creator)F. Sohel (Author/Creator)
- Publication Details
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.40(9), pp.2051-2065
- Publisher
- IEEE
- Identifiers
- 991005545386907891
- Copyright
- © 2017 IEEE
- Murdoch Affiliation
- School of Engineering and Information Technology
- Language
- English
- Resource Type
- Journal article
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Metrics
36 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- Citation topics
- 4 Electrical Engineering, Electronics & Computer Science
- 4.17 Computer Vision & Graphics
- 4.17.128 Deep Visual Recognition
- Web Of Science research areas
- Computer Science, Artificial Intelligence
- Engineering, Electrical & Electronic
- ESI research areas
- Engineering