Journal article
LCEval: Learned Composite Metric for Caption Evaluation
International Journal of Computer Vision, Vol.127(10), pp.1586-1610
2019
Abstract
Automatic evaluation metrics hold a fundamental importance in the development and fine-grained analysis of captioning systems. While current evaluation metrics tend to achieve an acceptable correlation with human judgements at the system level, they fail to do so at the caption level. In this work, we propose a neural network-based learned metric to improve the caption-level caption evaluation. To get a deeper insight into the parameters which impact a learned metric’s performance, this paper investigates the relationship between different linguistic features and the caption-level correlation of the learned metrics. We also compare metrics trained with different training examples to measure the variations in their evaluation. Moreover, we perform a robustness analysis, which highlights the sensitivity of learned and handcrafted metrics to various sentence perturbations. Our empirical analysis shows that our proposed metric not only outperforms the existing metrics in terms of caption-level correlation but it also shows a strong system-level correlation against human assessments.
Details
- Title
- LCEval: Learned Composite Metric for Caption Evaluation
- Authors/Creators
- N. Sharif (Author/Creator) - The University of Western AustraliaL. White (Author/Creator) - Invenia Labs, Cambridge, UKM. Bennamoun (Author/Creator) - The University of Western AustraliaW. Liu (Author/Creator) - The University of Western AustraliaS.A.A. Shah (Author/Creator) - Murdoch University
- Publication Details
- International Journal of Computer Vision, Vol.127(10), pp.1586-1610
- Publisher
- Springer US
- Identifiers
- 991005540932107891
- Copyright
- © 2019 Springer Science+Business Media, LLC, part of Springer Nature
- Murdoch Affiliation
- Information Technology, Mathematics and Statistics
- Language
- English
- Resource Type
- Journal article
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Metrics
32 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- International collaboration
- Citation topics
- 4 Electrical Engineering, Electronics & Computer Science
- 4.17 Computer Vision & Graphics
- 4.17.128 Deep Visual Recognition
- Web Of Science research areas
- Computer Science, Artificial Intelligence
- ESI research areas
- Engineering