On board with the general theme and overall success of using this and similar models as predictors to outcomes. Believe the link I provided above lays it out exacto as per your posts. My main point however was to highlight the fact that there are "outliers" to these models, and much more importantly, consistent outliers. It may be the case that this group represents a small percentage when gauged against the pool of 120, but it is significant in percentage amongst the top teams. Those teams use advantages outside of recruiting rankings in order to achieve their success, and in those cases, I believe it would be difficult to argue against coaching/development as the key to such. Though the number of these teams within a group of 120 or so would seem small, this group of outliers has a significant presence within the top 25 of teams, which of course is due to the fact that it is these schools that have the unique advantage or ability to acquire superior coaching talent. Beating a dead horse with the Mich St. example, but they rarely recruit top 30, yet with current staff are lately a top 5-10 type team. I can think of 4-5 teams which consistently break the mold in the past 3-4 years final top 25 teams like that. The issue to consider here is that while the "talent" formulas work often (other models/formulas do the same, see below) a school with the resources and ability to acquire and retain a superior staff can and do break the mold, the outliers to the overall formulas per se. Those are the teams that do it with a different recipe but certainly break the mold, consistently. So yes, in a pool of 120 teams, it is not significant, but within the top 25-30 teams it rears it's head often. Also, the 70/30 model seems to do well enough, not unlike Newton's initial theory of gravity, works most of the time, but it feels like subjective figures and in truth, all of the variables not mentioned yet which are in play as well as those I mentioned above (turnovers, penalties, weather, bounce of ball, etc) have a larger play than is given credit in this analysis. How many times each week does a team dominate statistically and find a way to lose? Why? And how does OK State beat UTexas even though UTexas recruits much better every year? There are stats/formulas that indicate a positive correlation to teams with fewest turnovers, most rushing yards, etc. which win 70% of the time as well. Who is to say which factors are the ultimate answer if there is such a thing? Many many factors involved, and as the Prof stated above, the outlier coaches have both positive and negative impacts. But my beef with the 70/30 model based upon recruiting rankings is that amongst top 25-30 teams, recruiting rankings seem to take a lesser role than the 70 percent rule. Witness the two exceptions highlighted in the above article for NC teams. If you really want to expound on this topic, when one looks at teams that did not win the NC but played in a BCS. bowl the since that format, the outliers grow in numbers exponentially. Bottom line? Get the best players and coaches available at the time and DEVELOP/RETAIN those players to their utmost ability. Then and only then, with a little luck, the right bounces etc., your team is in play for good things, maybe greatness. The formulas will not make a championship team, that is left to the players and coaching staff.
You have made many cogent points. I appreciate you taking the time to respond.
First let me begin by saying that this 70/30 relationship isn't the end of the road to understanding college football. I view it as the beginning. There are many people and organizations that do a far superior job. My numbers indicate that Vegas, for instance, clicks along at about an 80% correct prediction rate if you simply look at it as the favored team being the winner and ignore the spread. There are even people that do it better than that. Those sorts of systems are incredibly valuable and are unlikely to be shared on a free message board.
One of the problems that I see with your view is that you are mixing several different paradigms. Polls, for instance, are a jumbled but predictable mix of inconsistencies. Somewhere on
CFB Matrix - Setting College Football's Expectations you can find an explanation of how he can predict the final top 25, starting with talent evaluations, extrapolating out likely wins and losses for each team accounting for "coach effect", weighing these records by a modifier applied for conference affiliation, and so on. To simplify the conclusion: polls end up ranking teams by the numbers of wins and losses with preference given to conferences in a predictable order. This system is summarily flawed. Polls don't create a ranking of teams best to worst. The best example I can give to show you how badly polls are flawed is this: If polls are ranked best to worst, shouldn't the 8th best team lose more often than not to the top 7 teams and win more often than not against teams 9-25? I would submit that they should. Suppose team number 8 actually plays all of teams 1-7, and loses every game but beats teams 9-13 for a record of 5-7. Is there any way possible that under our current or any foreseeable system that polls would allow a 7 loss team to be ranked number 8?
Isn't it possible then, that a 7 loss team could still be the 8th best team in the country and not only be well outside of the top 25, but also ineligible for a bowl?
You commented how the exceptions seem to be more prevalent in the top 25. My samples tend to indicate the opposite is true. When a team is in the top 25, if you pick the higher ranked team (using recruiting numbers) the prediction rate begins to approach 80% (90% in title games, as I mentioned previously). It is when teams are in the bottom half of recruiting that the correlation drops below 70%. My hypothesis is that recruiting services are really good at distinguishing between higher ranked recruits (the kind that top teams viewed by recruiting rankings get) as opposed to the types of recruits that bottom teams recruit. Is this because recruiting services don't spend as much time, effort or resources on a NR through 2 star as they do a 3-5 star? Possibly.
Take for instance the SEC last year. 2013 was actually a relatively down year for talent predictions. Admittedly showing just one conference over one year is a small sample. But, as this year was a weaker correlation it would be the most helpful to your assertion that the 70/30 model fails more frequently with "better" coaching. Here is a chart showing actual wins and losses versus seasonal predicted wins and losses. Remember that using talent, each team (highlighted) should lose the games to teams above, and win games against teams below.
So what does this show? Talent averages correctly indicated 69.64% of the conference games played. Again, my numbers indicate that this was a relatively down year in the SEC.
If you look at seasonal predictions, only 4 teams turned out exactly as predicted (including Auburn), BUT 9 of 14 finished within 1 game of predictions (64.3%).
Only 3 teams had a swing of 3 or more games outside of predictions. All of those teams were in the SEC east. Vanderbilt and Mizzou were the two over-performers, and Florida was the under-performer. This means, at best in the SEC, 21% of teams are either positively or negatively effected by coaching in an appreciable way. Further examination might indict those over-performances. Did UGA's rash of injuries help Vandy and Mizzou? Did Florida's coaching AND injuries contribute to Vandy and Mizzou's over-performance totals? Did UT's depth issues or other factors contribute to withering late in the season? All of those are legitimate questions, I believe. If the answer is maybe or yes, that tends to point even more strongly to talent as the greatest indicator of success, does it not? Or, maybe variation is already accounted for in the probabilistic system?
It is hard for most to visualize (myself included) what it actually means that a team is predicted, by these matrices, to win 8 games and lose 4. What this chart shows is that each game is about a 70% likelihood for an individual win. Knowing the probability of each win or loss, one can calculate the actual likelihood that a team wins X number of games in that series. Take a hypothetical team predicted to go 8-4 using this metric. Probability calculates the following (shown in the thread above):
Probability of Winning at least 1 of 12 games: 100 %
Probability of Winning at least 2 of 12 games: 99.97 %
Probability of Winning at least 3 of 12 games: 99.67 %
Probability of Winning at least 4 of 12 games: 98.11 %
Probability of Winning at least 5 of 12 games: 92.65 %
Probability of Winning at least 6 of 12 games: 79.63 %
Probability of Winning at least 7 of 12 games: 58.09 %
Probability of Winning at least 8 of 12 games: 33.35 %
Probability of Winning at least 9 of 12 games: 13.95 %
Probability of Winning at least 10 of 12 games: 3.9 %
Probability of Winning at least 11 of 12 games: 0.64 %
Probability of Winning at least 12 of 12 games: 0.05 %
This proves that even with a 70% probability to win 8 of the 12 games, there is only a 1/3 shot that this team would actually win that many games or more. Many would claim this variation is due to coaching or other variables that arent already accounted for.
The only win loss totals that are virtually assured are 1 game, and 12 games. The rest are varying degrees of likelihood that are actually quantifiable and seem to make sense when viewed in context of actual results as shown.
As a final example here are the top 40 teams ranked by four year recruiting averages for 2013 and grouped by conference. Clear your mind of how good some poll tells you that these teams are, and use this as a guide to determine how teams actually performed in relation to their talent. Pick a team. look at their schedule. Chalk up a predicted loss to any team that they play that falls above them on this chart, and a win to any team below them. If two teams are adjacent, some calculations show to favor the home team 60/40.
Admittedly this is only an incomplete snapshot, but I believe you will find that it is a reasonable sample. Most teams perform within a game or so of expectations, and there are only a handful of teams that tend to track significantly outside of that range. Three teams on this list immediately jump out at me as performing outside of what predictions would indicate. 1)Florida, 2)Mizzou, 3) Michigan State (and perhaps a fourth in Stanford).
Ultimately I have no qualms with your conclusion that the key is to recruit, retain, and coach. Of those three, recruiting appears to be the most stable predictor of success. Logically I would conclude you believe this too. Do you believe the best coach (pick one who you believe is best) could take Navy's roster and win a national championship? I doubt you believe that. Doesn't that in itself tend to indicate that even the best coaching is limited by talent? Further, getting to a championship game requires recruiting in the top 20, and then having the better roster than your opponent (remember 90% of title games are won by the team with the better four year talent average). Knowing that, give me a great roster, and a coach with no apparent positive or negative effect on talent over the greatest coach in the land and a bottom tier roster.
