Journal article
Connection between BPTT and RTRL
Computational Intelligence and Applications, pp.97-102
1999
Abstract
This paper shows the connection between the Backpropagation Through Time (BPTT) algorithm, its truncated forms with truncation depth h, and the Recurrent Real Time Learning (RTRL) algorithm. The comparison is done by looking at a fully connected recurrent network, which is based on the same error function and calculations, using exact ordered derivatives. Two kind of formulas, based on total ordered derivatives, for BPTT(h) are given and proven to be equivalent. Of the two formulae, the second one, can be interpreted by a target modification in the case of h. Further, a combination of BPTT and RTRL is proposed to account for possible instabilities caused by weight adaptation. An overview of BPTT and RTRL and their implementations as well as their interpretations and uses are outlined.
Details
- Title
- Connection between BPTT and RTRL
- Authors/Creators
- T. Hanselmann (Author/Creator)A. Zaknich (Author/Creator)Y. Attikiouzel (Author/Creator)
- Publication Details
- Computational Intelligence and Applications, pp.97-102
- Publisher
- World Scientific and Engineering Academy and Society
- Identifiers
- 991005540273707891
- Murdoch Affiliation
- Murdoch University
- Language
- English
- Resource Type
- Journal article
Metrics
113 Record Views