Poster
in
Workshop: Methods and Opportunities at Small Scale (MOSS)
Understanding Attention Glitches with Threshold Relative Attention
Mattia Opper · Roland Fernandez · Paul Smolensky · Jianfeng Gao
Keywords: [ algorithmic reasoning ] [ length generalisation ] [ attention glitches ] [ flip-flops ]
Transformers struggle with generalisation, displaying poor performance even on basic yet fundamental tasks, such as flip-flop language modeling. We test whether these limitations can be explained through two key failures of self-attention. The first is the inability to fully remove irrelevant information. The second concerns position, even when a key is completely irrelevant learned positional biases may unintentionally up-weight it - dangerous when distances fall out of distribution. To probe this we propose TRA, a novel attention mechanism with which we demonstrate that these issues underlie generalisation failures on the flip-flop task.