Journal article
Exploiting layerwise convexity of rectifier networks with sign constrained weights
Neural Networks, Vol.105, pp.419-430
2018
Abstract
By introducing sign constraints on the weights, this paper proposes sign constrained rectifier networks (SCRNs), whose training can be solved efficiently by the well known majorization–minimization (MM) algorithms. We prove that the proposed two-hidden-layer SCRNs, which exhibit negative weights in the second hidden layer and negative weights in the output layer, are capable of separating any number of disjoint pattern sets. Furthermore, the proposed two-hidden-layer SCRNs can decompose the patterns of each class into several clusters so that each cluster is convexly separable from all the patterns from the other classes. This provides a means to learn the pattern structures and analyse the discriminant factors between different classes of patterns. Experimental results are provided to show the benefits of sign constraints in improving classification performance and the efficiency of the proposed MM algorithm.
Details
- Title
- Exploiting layerwise convexity of rectifier networks with sign constrained weights
- Authors/Creators
- S. An (Author/Creator)F. Boussaid (Author/Creator)M. Bennamoun (Author/Creator)F. Sohel (Author/Creator)
- Publication Details
- Neural Networks, Vol.105, pp.419-430
- Publisher
- Elsevier
- Identifiers
- 991005544654707891
- Copyright
- © 2018 Elsevier Ltd.
- Murdoch Affiliation
- School of Engineering and Information Technology
- Language
- English
- Resource Type
- Journal article
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Metrics
42 File views/ downloads
131 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- Citation topics
- 4 Electrical Engineering, Electronics & Computer Science
- 4.17 Computer Vision & Graphics
- 4.17.128 Deep Visual Recognition
- Web Of Science research areas
- Computer Science, Artificial Intelligence
- Neurosciences
- ESI research areas
- Computer Science