I do listen to books on tape now, but I have the same problem with those. ..I lose focus after a bit, my mind starts wandering over something that is said...and I have to rewind and listen to what I missed...over and over and over...it is aggravating as heck.
It also effects me watching movies and TV shows...I used to binge watch my favorite shows..but now it takes me a few watches to get through a good movie I enjoy, and sometimes months to finish a season of a series.
It started 3-4 years ago and has gotten severely worse...sucks getting old.