Logo image
Back
SubICap: Towards Subword-informed Image Captioning
Conference paper

SubICap: Towards Subword-informed Image Captioning

N. Sharif, M. Bennamoun, W. Liu and S.A.A. Shah
2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
IEEE Winter Conference on Applications of Computer Vision (WACV) 2021 (Waikoloa, HI, USA, 03/01/2021–08/01/2021)
2021
DOI: https://doi.org/10.1109/WACV48630.2021.00358
url
Link to Published Version *Subscription may be requiredView

Abstract

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units `subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.

Details

Metrics

47 Record Views
Logo image