By the stadium, its fine, because you have crowds and police around and that tends to tame everyone quite a bit. There's usually at least one loud mouth drunk within about 7 or 8 rows of you, and they will be looking for a fight. I think I've seen at least one guy like that at every game I've been to for my entire life -- and that's not just at Gator football games. I think its in the Constitution or something.
At any rate, my advice to minimize chances of anything other than a little good nature ribbing: 1) either pay for the parking pass right next to the stadium or park out at the mall and take a shuttle bus. Just avoid walking by fraternities or tailgating parties as much as you possibly can; 2) If some guy in the stands starts yelling at you from 30 seats away, assume he's a drunk and just don't even acknolwedge him. For God's sake don't yell back, because that's what he want so he can show off for his friends. If you ignore him, he'll call you a few choice names for 30 seconds but then he'll give up and look for someone else to taunt so he can be a bigshot; 3) if UT happens to pull the upset, be happy and joyous and what not, but remember where you are and save the singing "It's great" yada yada yada for the drive home.