Tannehill rated the 10th best quarterback in the league

[fyo]

Then you should've done the analysis I just did when looking at the data to begin with, and determined for yourself that Tannehill, despite being ranked 10th overall, was nonetheless in the average range.

No, that's incorrect. I don't dispute that performance most likely follows a normal distribution. That's the assumption we all make when dealing with these numbers all the time. Thus it's completely ridiculous for you to come out and say on the one hand that, well, he couldn't possibly be the 10th best. And then, on the other hand, say that, well, 10th best is really not significantly different than average.

Your point here rests on the idea that the sample should extend beyond 32 quarterbacks.

No. No. No.

I'm saying that if you're 1 standard deviation behind Usain Bolt, you may be pretty f'n fast, but you'll get smoked every time.

[MyGodWearsAHoodie]

Then you should've done the analysis I just did when looking at the data to begin with, and determined for yourself that Tannehill, despite being ranked 10th overall, was nonetheless in the average range

If we take the 32 starting QBs 8 will be in the top 25%, 8 in the bottom and 16 would be in "average range". 10 would in fact be the second best among the 16 average qbs

[Dolfanalyst]

No, that's incorrect. I don't dispute that performance most likely follows a normal distribution. That's the assumption we all make when dealing with these numbers all the time. Thus it's completely ridiculous for you to come out and say on the one hand that, well, he couldn't possibly be the 10th best. And then, on the other hand, say that, well, 10th best is really not significantly different than average.

I didn't say he couldn't possibly be the 10th best.  What I said was that the ranking is inconsistent with the statistics we've mentioned in the thread, and that there is an insufficient basis for deeming the ranking to be valid, since we're unaware of the methodology used and whether it was applied consistently across quarterbacks.

The issue that 10th best isn't significantly better than the average QB, when using that ranking information on the webpage, is yet another issue to contend with, in addition to the others mentioned.

No. No. No.

I'm saying that if you're 1 standard deviation behind Usain Bolt, you may be pretty f'n fast, but you'll get smoked every time.

Certainly.  And likewise, if you're 1 standard deviation behind the best QB in the league, you may be pretty f'n good, but you won't give your team nearly the probability of winning.

[Dolfanalyst]

If we take the 32 starting QBs 8 will be in the top 25%, 8 in the bottom and 16 would be in "average range". 10 would in fact be the second best among the 16 average qbs

And now you're back into rankings (i.e., "second-best"), which again can be misleading.

The proper way to do this is to determine how deviant from the league norm Tannehill is.

In the case of these rankings of overall performance, he's little more than a half a standard deviation (0.58) from the mean of the top 32 QBs ranked.  That ain't all that great.

[fyo]

And now you're back into rankings (i.e., "second-best"), which again can be misleading.

The proper way to do this is to determine how deviant from the league norm Tannehill is.

In the case of these rankings of overall performance, he's little more than a half a standard deviation (0.58) from the mean of the top 32 QBs ranked.  That ain't all that great.

If we're assuming that the population follows a normal distribution for the metric we're measuring (which is likely the case), then rankings map directly to z-scores. You calculate the standard deviation with the explicit assumption that the distribution is normal. If the rankings DON'T map directly to z-scores (in a significant way), then the distribution likely wasn't normal to begin with.

In other words, for a normal distribution rank and z-score are equivalent, by definition, so rank isn't any more or less misleading than looking at standard deviations.

For such small populations (e.g. 32 teams) there are bound to be slight irregularities, so the mapping may not be completely exact, although I have to say it fits really well for most of the metrics I've looked at.

[fyo]

if you're 1 standard deviation behind the best QB in the league, you may be pretty f'n good, but you won't give your team nearly the probability of winning.

Assuming similar variances in individual performance, being half or a full standard deviation above average results in the same win-probability as being half or a full standard deviation behind the best. In other words, half a standard deviation is very significant. Your specifically stated that any difference of less than a full standard deviation was insignificant. That is something that I disagree with completely and utterly -- and, your original statement to the contrary, you seem have come around (see quoted text above). If so, let's just leave the statistical aspects of this discussion behind. I have more problems with ESPN's QBR and am not at all convinced it is the least bit more objective than the scores I linked in the OP. These issues are systemic and, in my opinion, largely invalidate QBR as a meaningful metric when comparing dissimilar quarterbacks. I'll leave it at that teaser for now, though. I'm sure QBR will pop up again soon .

[Dolfanalyst]

If we're assuming that the population follows a normal distribution for the metric we're measuring (which is likely the case), then rankings map directly to z-scores. You calculate the standard deviation with the explicit assumption that the distribution is normal. If the rankings DON'T map directly to z-scores (in a significant way), then the distribution likely wasn't normal to begin with.

In other words, for a normal distribution rank and z-score are equivalent, by definition, so rank isn't any more or less misleading than looking at standard deviations.

For such small populations (e.g. 32 teams) there are bound to be slight irregularities, so the mapping may not be completely exact, although I have to say it fits really well for most of the metrics I've looked at.

The problem is that the rankings are perceived by the average fan to be interval data, with no knowledge of the size of the intervals between each datum.

[Dolfanalyst]

Assuming similar variances in individual performance, being half or a full standard deviation above average results in the same win-probability as being half or a full standard deviation behind the best. In other words, half a standard deviation is very significant. Your specifically stated that any difference of less than a full standard deviation was insignificant. That is something that I disagree with completely and utterly -- and, your original statement to the contrary, you seem have come around (see quoted text above). If so, let's just leave the statistical aspects of this discussion behind. I have more problems with ESPN's QBR and am not at all convinced it is the least bit more objective than the scores I linked in the OP. These issues are systemic and, in my opinion, largely invalidate QBR as a meaningful metric when comparing dissimilar quarterbacks. I'll leave it at that teaser for now, though. I'm sure QBR will pop up again soon .

Really what you're saying here highlights yet another problem with these rankings, in that we don't know how they correlate with winning, which would determine the expected increase in win probability for every SD increase in the measure used to make the ranking.

What I've been doing here is generously superimposing the correlations with winning we know exist for other measures (which are in the .50 range, give or take), and which give meaning to 1 SD changes either way in QB statistics.

Hell, these rankings could be correlated with winning at 0.00, in which case a change of a standard deviation (or more) would mean nothing!  Or, they could be correlated with winning at 1.0, in which case much smaller differences in SD would be associated with far bigger increases (or decreases) in win probability.

Either way, we would need to know how any measure correlates with winning to determine the meaning of a 1 SD change in the measure in either direction.

[fyo]

The problem is that the rankings are perceived by the average fan to be interval data, with no knowledge of the size of the intervals between each datum.

Well, then let that be a lesson to "the average fan"

(The point in your subsequent post opens a whole 'nother can o' worms and I'm just going to leave it at "very small sample size" for the time being.)

[MyGodWearsAHoodie]

So Fyo when ya revealing who is who?

[Spider-Dan]

That should be after Dolfanalyst announces his verdict for Tannehill's position on the chart, right?

[fyo]

That should be after Dolfanalyst announces his verdict for Tannehill's position on the chart, right?

I think he's already said what he's going to say about that, so I'll go ahead and put up the uncensored image:



The takeaways for me are:

- Dan Marino was really good. I know we all know this, but to compile those stats in that era is astonishing.
- Russell Wilson has gotten off to one heck of a start to his career. I don't get to see that many Seahawks games, but maybe I should.
- It's just fun to see a list headed by Peyton Manning, Dan Marino, and Ryan Tannehill.
- Quarterbacks that were asked to throw a BOATLOAD from day 1 generally turned out to be very good quarterbacks.

[Dolfanalyst]

I think he's already said what he's going to say about that, so I'll go ahead and put up the uncensored image:



The takeaways for me are:

- Dan Marino was really good. I know we all know this, but to compile those stats in that era is astonishing.
- Russell Wilson has gotten off to one heck of a start to his career. I don't get to see that many Seahawks games, but maybe I should.
- It's just fun to see a list headed by Peyton Manning, Dan Marino, and Ryan Tannehill.
- Quarterbacks that were asked to throw a BOATLOAD from day 1 generally turned out to be very good quarterbacks.

This sounds like it's reached the "just for fun" stage, and just for fun I'll try to give this a little more perspective here.

During the Tannehill era (2012 to 2016), teams with a QB rating between 84.2 and 86.2 (roughly his career QB rating) are 36-45 -- a 45.5 win percentage.  That translates to roughly a 7-9 regular season record.

During the same era, teams with a QB rating between 100.8 and 102.8 (roughly Russell Wilson's career QB rating) are 39-22 -- a 63.5 win percentage.  That translates to roughly a 10-6 regular season record.

During the Dan Marino era alluded to here (1983 to 1986), teams with a QB rating between 94.2 and 96.2 (roughly Marino's overall rating during that period) were 36-15 -- a 70.6 win percentage.  That translates to roughly an 11-5 regular season record.

Yet Tannehill has only slightly less passing yardage than Marino did during these time periods, and a good bit more than Wilson.

So again, if we focus exclusively on (or said differently, cherry-pick -- and I'm not saying that was fyo's intention here) passing yardage alone, we can easily go astray in terms of what's most important with regard to winning, and come away with a misperception of the quality of a QB.

Again, my intent here (and in all of these sorts of analytical discussions about Tannehill and QBs) is to refrain from setting myself up for disappointment by making a misguided conclusion in this way.

[MyGodWearsAHoodie]

My takeaway Dolphin fans that hate on Tannehill should STFU.  Is he the single best QB in the history of the NFL? Absolutely not.

But, he does match up pretty favorablely with other very good/great QB. 

A common refrain among those making the case for Marino was that fins didn't have much of a running game and put the game on Dan.  Well nobody on this list has more attempts, that speaks volumes about what share of the workload falls to Ryan each game.

He has a lower int % than Brady, a player known for his focus on ball security.

Reaffirms my opinion that Bledslow wasn't all that great.  Clearly the worst of the list. Tannehill is head and shoulders better than a QB that won the Patriots two AFCCGs

[Dolfanalyst]

My takeaway Dolphin fans that hate on Tannehill should STFU.  Is he the single best QB in the history of the NFL? Absolutely not.

But, he does match up pretty favorablely with other very good/great QB.

How so?