If I can think of two relevant things that bear upon your attention here.

Stress, which makes your attention kind of jump around.

Temptation, which makes your attention go places without you choosing it. It’s a kind of loss of control.