The way I heard it in Fort Worth, where I have lived for 25 years, is that Tennessee was looking for a coach and an ambassador for the school. Tennessee wanted someone who would coach the team and participate in statewide events, represent the school to the boosters, work on fundraisers, host the radio and tv shows, etc. Patterson wanted to be a coach. The concept of doing all of the other things was new to him. He didn’t do those things at the time or since at TCU. That was my understanding of the idea that the job was too big for him or that he wasn’t ready. It wasn’t about simply coaching the team.
Edit:
TCU is a small, Christian school with about 11,000 students now, but with about 9,000 (or perhaps fewer) in 2008. The stadium holds less than 45,000 and neither Texas nor Fort Worth are rabid about the Horned Frogs. TCU is not a globally recognizable brand. I have traveled about 50% of the time for the last 20+ years and I have never seen a TCU shirt outside of Texas. I have seen Tennessee shirts worn all over the US and in several countries in Europe and Asia. The reach of Tennessee far exceeds that of TCU.
On Saturdays in Fort Worth, traffic around the immediate area of the stadium is the only thing that would indicate a game is going on. It isn’t like Knoxville (or other parts of the state) where you see all of the shirts, stickers, car flags, etc. even from people not attending the game.
Is he a very good coach? Absolutely! Could he have been successful at Tennessee? I think so, if he accepted the other job responsibilities in addition to coaching.