Recruiting Ranking and Results in 3-4 Years

#29

knoxvol16

The Volifilator
Joined
Dec 31, 2010
Messages
1,509
Likes
497
#29
I did something similar, and posted it in a thread in the recruiting forum.

All the other stuff I did is here.
I read through your analysis on your personal site, and I must say "Nice work RiotVol."

I think your editorial and number crunching really highlights the fact that using predictive analyses in sports is wrought with concomitant variables. I think the real trick is using some sort of principal component regression technique to find the impact variables, then post hoc evaluate the weight of each. Of course, you somewhat corrupt the p-value by doing the post hoc weighting, but blah, blah, blah . . . with your background you know where I'm going.

Bottom line - nice work, a truly insightful use of predicted versus residuals to capture the high and low expectations; and, your log-log transformation . . . well, nice "voodoo."
 
#31

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#31
I read through your analysis on your personal site, and I must say "Nice work RiotVol."

I think your editorial and number crunching really highlights the fact that using predictive analyses in sports is wrought with concomitant variables. I think the real trick is using some sort of principal component regression technique to find the impact variables, then post hoc evaluate the weight of each. Of course, you somewhat corrupt the p-value by doing the post hoc weighting, but blah, blah, blah . . . with your background you know where I'm going.

Bottom line - nice work, a truly insightful use of predicted versus residuals to capture the high and low expectations; and, your log-log transformation . . . well, nice "voodoo."
Thanks, man. I'm a nerd. Huge nerd. Like a "I watch Lord of the Rings while working on my dissertation" nerd.

If you want to fool around with getting a pc regression to work, I'll email you the data. I kind of shy away from going through those hurdles just for the simple fact that after you've transformed the data so many times, it's hard to tell when you're really looking at when you're done. Seriously though, if you want the data shoot me a message.

I fooled around the win/loss records for the SEC today...



Here's the link to the rest of it...https://sites.google.com/a/email.arizona.edu/d-shane-miller/blog/12212-chiefly-cycling-and-sec-football
 
#34

wbailey7

Space Mountain
Joined
Oct 21, 2007
Messages
1,252
Likes
175
#34
Thanks, man. I'm a nerd. Huge nerd. Like a "I watch Lord of the Rings while working on my dissertation" nerd.

If you want to fool around with getting a pc regression to work, I'll email you the data. I kind of shy away from going through those hurdles just for the simple fact that after you've transformed the data so many times, it's hard to tell when you're really looking at when you're done. Seriously though, if you want the data shoot me a message.

I fooled around the win/loss records for the SEC today...



Here's the link to the rest of it...https://sites.google.com/a/email.arizona.edu/d-shane-miller/blog/12212-chiefly-cycling-and-sec-football
I worked on something in one of my graduate level classes where I used principal component analysis to cluster (by fuzzy methods) teams together based on various BCS criteria (computer input, AP, etc.).

What's cool, and it depends on the method, is when you use PC analysis for inputs, you can typically go back and use some sort of variable selection routine -forward if you have collinearity issues, all-possible routines if you don't have many variables, or even an genetic algorithm (I prefer it, but the coding usually sucks) to do a variable selection and then use those variables to predict the groups you have classified into (or values if you do not have a categorical response). This would go hand-in-hand with also having a validation sample as well.

Could you send me the time series data of the plots you posted above? Or whatever you have. This thread just made me happy. I'll throw it into SAS and have some fun with it for the night.
 
#35

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#35
I worked on something in one of my graduate level classes where I used principal component analysis to cluster (by fuzzy methods) teams together based on various BCS criteria (computer input, AP, etc.).

What's cool, and it depends on the method, is when you use PC analysis for inputs, you can typically go back and use some sort of variable selection routine -forward if you have collinearity issues, all-possible routines if you don't have many variables, or even an genetic algorithm (I prefer it, but the coding usually sucks) to do a variable selection and then use those variables to predict the groups you have classified into (or values if you do not have a categorical response). This would go hand-in-hand with also having a validation sample as well.

Could you send me the time series data of the plots you posted above? Or whatever you have. This thread just made me happy. I'll throw it into SAS and have some fun with it for the night.

Dude, tear it up. I think I follow what you're saying, and I'm kind of curious to see what comes out.

Here's the file as a google doc. Let me know if you have any problems accessing it.

https://docs.google.com/spreadsheet/ccc?key=0ApINaFdTJ7WxdDBwLVpzYzROVjZTX3NUVFl0YV9wbFE
 
#36

knoxvol16

The Volifilator
Joined
Dec 31, 2010
Messages
1,509
Likes
497
#36
I worked on something in one of my graduate level classes . . .
Dude, tear it up . . .
Cool stuff from both of you guys. I've unfortunately been away from using a lot of Statistics going on a few years now. So, I'd have to have use/write R routines because I no longer have access to SAS (which would be okay if I had the time, because I really like R).

Anyway, my current position doesn't offer me the need to use my Stat background as much as in years past.

I'm interested in seeing the results.
 
Last edited:
#37

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#37
Cool stuff from both of you guys. I've unfortunately been away from using a lot of Statistics going on a few years now. So, I'd have to have use/write R routines because I no longer have access to SAS (which would be okay if I had the time, because I really like R).

Anyway, my current position doesn't offer me the need to use my Stat background as much as in years past.

I'm interested in seeing the results.

I used SAS when I took stats at UT back in the day. Then I stumbled across JMP and have really taken to that. I should probably learn R, but then again I should probably be working on a lot of things beyond crunching numbers on college football.
 
#38
Joined
Nov 30, 2011
Messages
30
Likes
0
#38
Riot, I don't think you can use your graph to explain the effectiveness of recruiting good players. There is a slightly positive slope on the trendline but I would say that your graph reflects the ability that coaching and growth have on highschool players than highschool player rankings have on team rankings. Look at k state and lsu and that left-most point (alabama). They are so far above the trendline that it indicates something else is affecting performance.

Then look at UT, Ole Miss, and Indiana. All of them are known to have rather dismal coaching staffs during the time period and although UT and Ole Miss are in the left half of the graph, they are pretty far below the trendline. It's obvious that player potential has something to do with it, but without a coach to unleash the potential, you end up, well, like us: less than average.
 
#40

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#40
Riot, I don't think you can use your graph to explain the effectiveness of recruiting good players. There is a slightly positive slope on the trendline but I would say that your graph reflects the ability that coaching and growth have on highschool players than highschool player rankings have on team rankings. Look at k state and lsu and that left-most point (alabama). They are so far above the trendline that it indicates something else is affecting performance.

Then look at UT, Ole Miss, and Indiana. All of them are known to have rather dismal coaching staffs during the time period and although UT and Ole Miss are in the left half of the graph, they are pretty far below the trendline. It's obvious that player potential has something to do with it, but without a coach to unleash the potential, you end up, well, like us: less than average.
I talked about this a bit in my blog entry. I actually agree with you.
 
#41

wbailey7

Space Mountain
Joined
Oct 21, 2007
Messages
1,252
Likes
175
#41
I used SAS when I took stats at UT back in the day. Then I stumbled across JMP and have really taken to that. I should probably learn R, but then again I should probably be working on a lot of things beyond crunching numbers on college football.
What Stats class? I noticed on your blog that you're doing anthropology. There were a ton of anthropology Ph.D's in my Multivariate class. Same goes for Time Series.
 
#42

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#42
What Stats class? I noticed on your blog that you're doing anthropology. There were a ton of anthropology Ph.D's in my Multivariate class. Same goes for Time Series.
I took Stats 537 and 538 from Schmidhammer. At that point in time, all of us Anthro folks were required to take 537.

That dude was a trip. I remember him giving us homework on stuff he wouldn't talk about for two more weeks.

Then I took multivariate when I got to Arizona.
 
#47

TrueOrange

Well-Known Member
Joined
Jul 7, 2008
Messages
43,610
Likes
2,216
#47
The Mr SEC blog, actually, just made a really good article on this topic:

Blue Chip Stories | MrSEC


Next Wednesday, one group of SEC fans will celebrate a signing day “championship.”* That same evening, a larger group of SEC-backers will claim that recruiting rankings aren’t accurate.

Both groups will be right.* Sort of.

In order to get a grip on just how accurate recruiting rankings are when it comes to predicting success in the rough and tumble SEC, we went back through 10 years of signing day grades and rankings.* Then we compared those rankings to the actual on-field results from 2006 through 2011.

We found — as we have before — that recruiting rankings do provide a pretty good ballpark indicator of a program’s future success.* But they are far from infallible.

As usual, we pored over the rankings as put together by Rivals.com.* Some prefer ESPNU’s rankings, others Scout.com and so on.* We like Rivals.* And the data you’ll see will explain why.

The general process was as follows:

1.* Tally up the recruiting rankings for all the signing classes that would normally impact a season.* Let’s use this past 2011 season as an example.* True freshman signed in ’11, sophomores in ’10, juniors in ’09, seniors in ’08, and a few redshirt seniors might’ve still been around from the ’07 class.* We used Rivals.com’s SEC rankings, 1 through 12.

2.* Add up the SEC records for each program in a given year.* The SEC title game didn’t count.* We didn’t knock off Alabama’s numbers due to NCAA penalties.* We wanted to know which teams won the most in-league games only… and we wanted to know who actually won on the field, not who was stripped by the NCAA later.

3.* Finally, we compared the combined recruiting rankings with the SEC records from the season in question.* Pretty simple.

Now, signing classes can be affected — obviously — by coaching changes, attrition, injuries, transfers, flunk outs, drop outs, dismissals and the like.* So the system isn’t perfect.* But it’s close enough to give us an idea of how accurate the recruiting rankings work.

Looking at the recruiting classes from 2002, 2003, 2004, 2005 and 2006, here’s what we found when comparing rankings to actual on-field SEC results in the fall of ’06:

School 2002 Rank 2003 Rank 2004 Rank 2005 Rank 2006 Rank Combined Recruiting Rank 2006 SEC Record
Georgia 2 3 2 2 2 11 4-4
LSU 5 1 1 6 3 16 6-2
Florida 7 2 3 4 1 17 7-1
Tennessee 1 7 4 1 7 20 5-3
Auburn 3 6 6 3 4 22 6-2
S. Carolina 4 4 9 7 8 32 3-5
Alabama 9 10 5 5 5 34 2-6
Arkansas 8 8 7 8 9 40 7-1
Ole Miss 10 9 8 9 6 42 2-6
Miss. State 6 5 11 10 11 43 1-7
Kentucky 12 11 10 11 10 54 4-4
Vanderbilt 11 12 12 12 12 59 1-7
Okay, right off the bat you’ll see that the combined recruiting rankings from ’02 through ’06 don’t provide a perfect team-by-team indicator of success.* Georgia had the five best classes leading up to 2006, yet the Dawgs managed only a 4-4 SEC record.* Arkansas, on the other hand, finished with a 7-1 league mark despite ranking 8th in the SEC for that five-year recruiting window.

Looking at the six seasons from 2006 through 2011, we found that there were always some schools that finished much better or much worse than the recruiting rankings would have suggested.

So recruiting rankings don’t work.* Right?* Not exactly.

For kicks we broke the league into fourths.* The idea was to see if recruiting rankings worked on a general basis.* Boy, did they:

The top three teams in recruiting rankings from ’02-’06 (Georgia, LSU and Florida) combined for a 17-7 SEC record.* That’s a winning percentage of .708

The next three teams in the recruiting rankings (Tennessee, Auburn and South Carolina) combined for a 14-10 SEC record.* That’s a winning percentage of .583.

The next three teams down the list (Alabama, Arkansas, and Ole Miss) notched an 11-13 SEC record.* That’s a winning percentage of .458.

And the three worst teams in Rivals’ ’02 to “06 recruiting rankings (MSU, Kentucky and Vandy) combined for an 8-24 SEC record.* That’s a .250 winning percentage.

In other words, recruiting rankings might not tell you exactly how your team will finish in SEC play, but they will give you a pretty good idea.* And we found that to be the case in 2006, 2007, 2008, 2009, 2010 and 2011.

Below are the group results for each of those seasons:

2007 Season (2003-2007 recruiting rankings)
1.* Top Three Schools (Florida, LSU, Georgia): 17-7 in SEC, .708
2.* Next Three Schools (Tennessee, Auburn, S. Carolina): 14-10 in SEC, .583
3.* Next Three Schools (Alabama, Ole Miss, Arkansas): 8-16 in SEC, .333
4.* Bottom Three Schools (MSU, Kentucky, Vanderbilt): 9-15 in SEC, .375

2008 Season (2004-2008 recruiting rankings)
1.* Top Three Schools (Florida, Georgia, LSU): 16-8 in SEC, .666
2.* Next Three Schools (Tennessee, Auburn, Alabama): 13-11 in SEC, .541
3.* Next Three Schools (S. Carolina, Ole Miss, Arkansas): 11-13 in SEC, .458
4.* Bottom Three Schools (MSU, Kentucky, Vanderbilt): 8-16 in SEC, .333

2009 Season (2005-2009 recruiting rankings)
1.* Top Three Schools (Florida, Georgia, LSU): 17-7 in SEC, .708
2.* Next Three Schools (Alabama, Tennessee, Auburn): 15-9 in SEC, .625
3.* Next Three Schools (S. Carolina, Ole Miss, Arkansas): 10-14 in SEC, .416
4.* Bottom Three Schools (MSU, Kentucky, Vanderbilt): 6-18 in SEC, .250

2010 Season (2006-2010 recruiting rankings)
1.* Top Three Schools (Florida, LSU, Alabama): 15-9 in SEC, .625
2.* Next Three Schools (Georgia, Auburn, Tennessee): 14-10 in SEC, .583
3.* Next Three Schools (S. Carolina, Ole Miss, Arkansas): 12-12 in SEC, .500
4.* Bottom Three Schools (MSU, Kentucky, Vanderbilt): 7-17 in SEC, .291

2011 Season (2007-2011 recruiting rankings)
1.* Top Three Schools (Alabama, Florida, LSU): 18-6 in SEC, .750
2.* Next Three Schools (Georgia, Tennessee, Auburn): 12-12 in SEC, .500
3.* Next Three Schools (S. Carolina, Ole Miss, Arkansas): 12-12 in SEC, .500
4.* Bottom Three Schools (MSU, Kentucky, Vanderbilt): 6-18 in SEC, .250

That’s pretty impressive.* Breaking six seasons into fourths, there were 24 “slots” and only two of those slots were a tad off:

* In 2007, the bottom four schools in the recruiting rankings actually outperformed teams 7, 8 and 9 by one game in the SEC standings.

* Last season, teams 7, 8 and 9 finished with the same .500 record in conference play that teams 4, 5 and 6 did.

Other than those two tiny differences, the recruiting rankings provided a good ballpark indicator of teams’ SEC success.

The trick to reading recruiting rankings, therefore, is to use them as a compass.* Over a span of years, you’ll find that the teams getting the highest marks on signing day in the SEC will usually do pretty well.* Those that score poorly, usually really won’t have good results inside the league.

But recruiting rankings cannot be used as a GPS.* They aren’t precise.* They aren’t perfectly accurate.* Almost every year, Arkansas outperforms its recruiting grades.* Meanwhile, a school like Tennessee — that has seen massive attrition thanks to back-to-back coaching changes — has underperformed based on the caliber of its signing classes.

Recruiting rankings do matter.* The more four-stars your schools signs, the better the odds you’ll find a great difference-maker.* It’s a bit like buying raffle tickets.* The more you have, the better your odds of winning the prize.

Just remember — and we can’t say it enough — these rankings have to be used as a compass to point you in the right direction.* They can’t be used as GPS to tell you exactly where your favorite team will finish in a given year.

Finally, as a bonus, we’ve provided the combined class rankings for each school from 2002 through 2011 below.* Also listed are each school’s SEC record for the years 2006 through 2011.* Once again… even these rankings over a such a long period of time delivered a good ballpark read on how things would actually play out on the field over that six-season span of games:

School Combined 2002-2011 Recruiting Rank Combined 2006-2011 SEC Record Group Record Group Winning Pct.
Florida 1st (31 total points) 34-14
Georgia 1st (31 total points) 30-18
LSU 3rd (32 total points) 34-14 98-46 .680
Tennessee 4th (45 total points) 22-26
Alabama 5th (47 total points) 34-14
Auburn 5th (47 total points) 28-20 84-60 .583
S. Carolina 7th (63 total points) 24-24
Ole Miss 8th (80 total points) 12-36
Arkansas 9th (84 total points) 28-20 64-80 .444
Miss. State 10th (92 total points) 16-32
Kentucky 11th (109 total points) 16-32
Vanderbilt 12th (119 total points) 10-38 42-102 .291
And in case you’re wondering, the Rivals.com currently ranks the SEC recruiting classes as follows (from first to 14th): Alabama, Florida, Texas A&M, Tennessee, South Carolina, LSU, Auburn, Georgia, Vanderbilt, Arkansas, Mississippi State, Ole Miss, Missouri and Kentucky.

Might show up better on the actual site...charts/columns likely much clearer
 
#48

RiotVol

Well-Known Member
Joined
Dec 25, 2011
Messages
766
Likes
819
#48
The Mr SEC blog, actually, just made a really good article on this topic:

Blue Chip Stories | MrSEC





Might show up better on the actual site...charts/columns likely much clearer


I think what I did was similar but I focused my sample more temporally, and looked at teams from all of the major conferences. I had to then scale fore schedule strength, which I'm not sure you need to do necessarily if you're just looking at SEC teams like they did (although it would help if you just stuck with the conference record vs. overall record).

Beyond that - some people prefer tables. I prefer graphs.

Cool stuff. Thanks for re-posting.
 
#50

TrueOrange

Well-Known Member
Joined
Jul 7, 2008
Messages
43,610
Likes
2,216
#50
I think what I did was similar but I focused my sample more temporally, and looked at teams from all of the major conferences. I had to then scale fore schedule strength, which I'm not sure you need to do necessarily if you're just looking at SEC teams like they did (although it would help if you just stuck with the conference record vs. overall record).

Beyond that - some people prefer tables. I prefer graphs.

Cool stuff. Thanks for re-posting.
:hi: no prob. Glad you liked it
 


VN Store

Sponsors
 

Top