Personally, I wouldn't mind them hiring Herman. He's had a heck of a year at Houston and a solid track record as an OC, but he's still pretty inexperienced as a HC. Furthermore, it's not like Houston is a terribly difficult place to win football games. Kevin Sumlin had much more sustained success there and his SEC results have been up and down at a much better program with a better recruiting base near all the schools he alreay had recruiting in-roads.
I'm not saying that I don't think Herman can or will be a good coach one day, but he's just not that experienced as a HC yet and USC is far from the area he's most familiar with recruiting. He hasn't shown that he can build up a program in the way that, say, Jim McCelwain did at CSU. And USC will have a LONG way to go before being back to where Spurrier had them in 2013.
I'd be much, much more worried if they'd hired Fuente.