I say keep them in. They brought a TON of SEC media coverage to my local media market, Plus, Before they joined, I could wear my orange and might get a "GBO" a few times a year. Since, I get to tick off dozens a year wearing it.
College football interest here is actually pretty high and has gotten higher since the Rams deal really soured the market on the NFL, and the SEC is the dominant interest. There is a TON of Mizzou graduates in St. Louis, and it's the second largest media market in the league.
As for the "Southerness" of the state, I have ridden my MC all over the state and back more times than I can count, and I can only say that anywhere outside of St. Louis and KC feels about as "Southern" as you can get. I find the people to share values and beliefs quite similar to my East Tennessee upbringing.
All that said, it is true that you don't get that "breath, eat and sleep" all things Mizzou vibe like you get with the Volunteers in Tennessee.
And for what it is worth, there is a significant percentage of Mizzou fans that agree with Freak. They don't like being in the SEC either. They would prefer to go back to the Big 12 — partly for nostalgia's sake, and partly because they feel like they have a better chance of being competitive.