Logo image
Back
Using an effective Boltzmann machine to learn context dependencies of a sequence
Conference paper   Open access

Using an effective Boltzmann machine to learn context dependencies of a sequence

M.I. Bellgard and C.P. Tsang
Proceedings of ICNN'95 - International Conference on Neural Networks, Vol.5, pp.2841-2846
1995 IEEE International Conference on Neural Networks (ICNN '95) (Perth, W.A, 27/11/1995–01/12/1995)
1995
DOI: https://doi.org/10.1109/ICNN.1995.488184
pdf
effective_boltzmann_machine.pdfDownloadView
Published (Version of Record) Open Access
url
Link to Published Version *Subscription may be requiredView

Abstract

It has been recognised that current Artificial Neural Network (ANN) systems that employ windowing techniques to learn context dependencies in sequences have many deficiencies. One variant of this type of system that attempts to overcome many of these deficiencies is the Effective Boltzmann Machine(EBM). The EBM which is based on the Boltzmann Machine (BM) has the ability to perform completion and to provide an energy measure for the solution. In this paper, we extend the EBM and show that the system has many desirable properties. This paper reports two major improvements to the EBM First, whereas in the past, a BM is used to learn the local contexts of a sequence, we show that the EBM itself is an architecture suitable for learning local contexts. Our initial experiments show that it outperforms a window based BA4 Second, we demonstrate that, with this new training scheme, a multilayer EBM can be constructed to extend the effective window size. This means that learning contextual dependencies is not limited by the window size. This is done by utilising the state information of the first EBM hidden layer to train that of the second EBM layer. The effect of this long range effective window is demonstrated by experiments.

Details

Metrics

192 File views/ downloads
94 Record Views
Logo image