Abstract
The attention mechanism in neural networks is often viewed as a means of gaining insight into model behavior, particularly by highlighting which parts of the input the model focuses on when generating predictions. This has led to the assumption that attention distributions can serve as explanations of model decisions. Recent studies, however, present conflicting views on whether attention weights truly provide faithful and meaningful interpretations, raising concerns about their transparency. To address this debate, this paper systematically reviews 191 studies published between 2016 and 2025, categorized into three methodological groups: visualization-based, probing based, and functional-based approaches. Visualization methods convert high-dimensional attention tensors into interpretable visual forms that reveal relational patterns and information flow across Transformer layers. Probing methods analyze the linguistic knowledge encoded in attention while considering operational factors. Functional-based methods directly intervene in model internals to uncover the causal and mechanistic roles of attention. Together, these paradigms provide complementary perspectives on what attention captures, how it represents information, and the limitations of current interpretability techniques. The paper also discusses emerging trends and directions for future work aimed at improving the transparency of attention mechanisms.
