Tannehill rated the 10th best quarterback in the league

[fyo]

DVOA and DYAR makes no attempt to distinguish the different components of the offense, except for a split into running plays and passing plays. When applied to quarterbacks, for example, perfectly thrown footballs dropped in the endzone (or on the way into the endzone **cough** Stills **cough**) are counted as negative plays for the quarterbacks. Nothing that actually happens on the play, other than the single line that is generated in the play-by-play, is taken into account. E.g. there is no difference between an interception thrown directly to an opposing receiver and a missed assignment from an offensive lineman allowing a hit by a defender that causes the ball to go way off target. A turnstile "tackle" allowing a blind side sack resulting in a fumble is graded the same as a botched snap or a fumble caused by the quarterback simply dropping the ball untouched. A short screen pass to a running back or tight end that is subsequently taken for 50 yards and a touchdown is graded exactly the same as a perfect backshoulder throw 50 yards downfield.

The result is that DVOA and DYAR are excellent measures of the offense as a whole, but should be applied very cautiously to individual players or components of an offense. Comparing a quarterback+offensive-line+receiver-combo from week to week for a given team is still valid -- and a very useful tool. However, comparing the DVOA of "a quarterback" from one team with the DVOA of "a quarterback" from another team is not likely to yield a valid result. "A quarterback" is in quotes here since no actual attempt is made to isolate the QUARTERBACK's performance from that of the rest of the offense and, as such, the metric isn't really measuring quarterback performance, despite the name.

If you look specifically at quarterbacks, DVOA will result in a overly positive score for quarterbacks who throw high-percentage screens to running backs and tight ends capable of churning out tons of yards after catch. As a whole, YAC becomes extremely important and quarterbacks without receivers capable of generating YAC will receive significantly lower grades than those who have such receivers. All indications are that YAC are mostly dependent on the receiver himself and how he is used, but fairly independent of the quarterback (I note that ESPN has acknowledged this and disregards actual YAC and instead adds an "expected YAC"). Quarterbacks with solid offensive lines will also produce higher DVOA ratings as the "fault" of a sack (and any fumble that might occur) rests fully on the quarterback.

The (subjective) decision has also been made to exclude interceptions (and fumbles) from the "situational awareness" (time remaining, score, down, distance) otherwise used in deriving DVOA, with a few exceptions (notably on fourth down inside 2 minutes and some rather arbitrary adjustments depending on length of throw, punishing interceptions on short throws more than those on long throws). This punishes risk-taking, which would otherwise increase win-probability in games where teams are down multiple touchdowns (outside last two minutes). Interceptions stemming from mistakes by the offensive line are also punished MORE severely than throws that are simply bad quarterback decisions, since hits on the quarterback (or arms) are likely to cause shorter interceptions (shorter "throws") than if the quarterback were not hit and just threw a "clean" interception.

I'm going to make a quick back-of-the-envelope calculation here, partly because I haven't bothered yet, but also because it applies to QBR as well, which I'll get to at some point, although this post is ballooning in length as is.

According to ESPN and other sites (link), about 4% of passes are listed as dropped (3.6% for 2015). So far this year there have been a total of 6127 completed passes league wide, resulting in a total of 411 (passing) touchdowns, so 6.7% of all completed passes result in touchdowns. Assuming receiver drops are distributed randomly among passes, a dropped pass would also have a 6.7% chance of being a dropped touchdown pass. League average pass attempts (9666/32 = ) 302 results in about 12 dropped passes per team so far this year. With (12*0.067 = ) 0.8 of those expected to have gone for a touchdown. The Dolphins thus far have 3 dropped touchdown passes, including the Stills drop.

Dropped passes may or may not be a receiver specific, or just bad luck, but Tannehill has clearly gotten the wrong end of that particular stick.

The same goes with every single one of the other deficiencies in DVOA (when used to compare quarterbacks across teams) I listed above.

That's *remarkable*.

I will note that I am a huge fan of Football Outsiders and DVOA, but as the saying goes "lies, damned lies, and statistics". In this case, you have to remember what it is you are actually measuring; and what you are not measuring.

In other words, I honestly believe that DVOA massively underestimates Tannehill's performance this year (and, to a somewhat lesser extent in that I'm assuming he doesn't always suffer from such a ridiculous number of dropped touchdown passes, in other years).

mod edit: fixed broken link

[MyGodWearsAHoodie]

You say that as if DVOA and QBR are objective data.

They are objective data.  Measurable calculated data is by definition objective.  Just like shoe size of a WBA is objective data.  What is subjective is if you feel those measurements are a good or bad way to judge a QB.

[Dolfanalyst]

I'm going to make a quick back-of-the-envelope calculation here, partly because I haven't bothered yet, but also because it applies to QBR as well, which I'll get to at some point, although this post is ballooning in length as is.

According to ESPN and other sites (link), about 4% of passes are listed as dropped (3.6% for 2015). So far this year there have been a total of 6127 completed passes league wide, resulting in a total of 411 (passing) touchdowns, so 6.7% of all completed passes result in touchdowns. Assuming receiver drops are distributed randomly among passes, a dropped pass would also have a 6.7% chance of being a dropped touchdown pass. League average pass attempts (9666/32 = ) 302 results in about 12 dropped passes per team so far this year. With (12*0.067 = ) 0.8 of those expected to have gone for a touchdown. The Dolphins thus far have 3 dropped touchdown passes, including the Stills drop.

Dropped passes may or may not be a receiver specific, or just bad luck, but Tannehill has clearly gotten the wrong end of that particular stick.

The same goes with every single one of the other deficiencies in DVOA (when used to compare quarterbacks across teams) I listed above.

That's *remarkable*.

I appreciate your in-depth analysis, but it's hard to believe that Tannehill's ability is being underestimated a great deal by the most sophisticated statistics we have available when there has been no season in his career that he's been any better than average with regard to them, and instead he's typically been below average with regard to them.

This year may be an anomaly with regard to the ability of DVOA to accurately represent his play, but when you look at how he has fared with regard to DVOA over his career, adjusting for this year's anomaly is likely to bring him from the depths of the league to somewhere between average and below average.

It's very unlikely that making such an adjustment would vault him from the depths of the league to being ranked 10th overall, a la the subjective material in the OP.

[Spider-Dan]

They are objective data.

Whether or not a quarterback is under pressure is not objective.

[Dolfanalyst]

Whether or not a quarterback is under pressure is not objective.

Indeed it's not, but it's easier for me to imagine a group of people's being able to define it in such a way that they can reach sufficient agreement about when it's happening and when it isn't, and then apply that definition consistently across QBs and situations, than it is for me to imagine a group of people's being able to take the entirety of QB play and do the same thing.

One thing being measured is relatively narrowly defined, simple, and manageable, and the other is very broad, complex, and dynamic.

I suspect that if you took a sample of let's say 50 passing plays and had three people here decide which ones Tannehill was pressured on, you'd get a far greater degree of agreement than if you asked three people here to simply evaluate Tannehill in his entirety by assigning him a percentile rank among the league's QBs.

[masterfins]

^^^

I suspect if one were to put a variety of QB stats out there, without the players' names attached, your opinion of Tannehill would be quite different.

[Dolfanalyst]

^^^

I suspect if one were to put a variety of QB stats out there, without the players' names attached, your opinion of Tannehill would be quite different.

Try me.

You ought to know by now that I've done my homework.

[fyo]

I appreciate your in-depth analysis, but it's hard to believe that Tannehill's ability is being underestimated a great deal by the most sophisticated statistics we have available when there has been no season in his career that he's been any better than average with regard to them, and instead he's typically been below average with regard to them.

This year is a bit extreme with the statistical outliers of dropped touchdowns, but DVOA consistently underrates quarterbacks with bad offensive lines, quarterbacks with receivers who don't generate YAC, and quarterbacks who play from behind a lot. That isn't specific to Tannehill, although it has plagued Tannehill in each of his first four years. Part of that is personnel, but part is probably also that they keep changing coaches (and therefore blocking schemes, not just for the o-line, but the running backs and tight ends as well).

As for QBR, I disagree with your assessment of it being objective. The biggest issue is that we just don't know enough about how it is generated. I've read all the descriptions on ESPN before, but I've reread them a few times now and there's enough wiggle room to drive a fleet of trucks through in their descriptions. Oh, and you'll appreciate that they frequently refer to QBR as a measure of quarterback efficiency.

Apart from the obvious bit about determining pressure, it's not at all clear how the "division of credit" works. The only substantive statement is that it is based on analysis of thousands and thousands of plays. It might be a fixed ratio (with a boolean depending on pressure or not pressure), but that isn't actually stated. If it isn't fixed, then a whole boatload of subjective judgement comes into play. On the other hand, if it is fixed, it effectively averages out things that are important for the individual plays. The hope is that these aspects of the play average out over a game or a season, but without giving any supporting evidence that's a bit of a leap of faith.

The section on division of credit explicitly states that's what happens on completed passes, leading one to believe that no division of "credit" (blame) happens negative plays. However, some of the older descriptions of QBR (from when ESPN introduced it) included judgement calls on drops, underthrows, and overthrows, among other things. These are clearly subjective to some degree, but also important to include. It's entirely unclear if they are still used in QBR since ESPN won't show us the recipe.

[fyo]

Try me.

You ought to know by now that I've done my homework.

Below is a chart with the 13 quarterbacks who have passed for 13000 or more yards in their first four years (I added Brady to the bunch manually, as he had 3 passes or something his first year). The data is from Pro Football Reference.




Sacks, wins and losses give too much away, so that isn't included . For reference, about half the players on the list are pretty close to .500 in W/L during their first 4 years.

The quarterbacks are, in alphabetical order:

Drew Bledsoe
Tom Brady
Andy Dalton
Joe Flacco
Jeff Garcia
Andrew Luck
Peyton Manning
Dan Marino
Cam Newton
Carson Palmer
Matt Ryan
Ryan Tannehill
Russell Wilson

[Dolfanalyst]

Below is a chart with the 13 quarterbacks who have passed for 13000 or more yards in their first four years (I added Brady to the bunch manually, as he had 3 passes or something his first year). The data is from Pro Football Reference.




Sacks, wins and losses give too much away, so that isn't included . For reference, about half the players on the list are pretty close to .500 in W/L during their first 4 years.

The quarterbacks are, in alphabetical order:

Drew Bledsoe
Tom Brady
Andy Dalton
Joe Flacco
Jeff Garcia
Andrew Luck
Peyton Manning
Dan Marino
Cam Newton
Carson Palmer
Matt Ryan
Ryan Tannehill
Russell Wilson

Thanks.  This ought to be fun.

I'm well aware of what Tannehill has done in the way of passing yardage, which is a volume statistic, as opposed to an efficiency statistic.

The issues we're going to face in this particular exercise are that 1) passing yardage isn't anywhere near as predictive of winning as passing efficiency (YPA, or Y/A as it's referred to in the table) or overall play (QB rating), and 2) we have a mixture of time periods, both before and after 2004, when the rules were changed to favor the passing game.

The average QB rating in the league has steadily increased over the years, and jumped a great deal after 2004.  This year for example it's almost 90.

What stands out to me in that table is that you have an absolute stud with a QB rating of 101.8 and a YPA of roughly 8.  I know who that is, and I know he compiled those statistics after 2004.  Had he done it prior to 2004 it would be mind-boggling.

[Dolfanalyst]

This year is a bit extreme with the statistical outliers of dropped touchdowns, but DVOA consistently underrates quarterbacks with bad offensive lines, quarterbacks with receivers who don't generate YAC, and quarterbacks who play from behind a lot. That isn't specific to Tannehill, although it has plagued Tannehill in each of his first four years. Part of that is personnel, but part is probably also that they keep changing coaches (and therefore blocking schemes, not just for the o-line, but the running backs and tight ends as well).

As for QBR, I disagree with your assessment of it being objective. The biggest issue is that we just don't know enough about how it is generated. I've read all the descriptions on ESPN before, but I've reread them a few times now and there's enough wiggle room to drive a fleet of trucks through in their descriptions. Oh, and you'll appreciate that they frequently refer to QBR as a measure of quarterback efficiency.

Apart from the obvious bit about determining pressure, it's not at all clear how the "division of credit" works. The only substantive statement is that it is based on analysis of thousands and thousands of plays. It might be a fixed ratio (with a boolean depending on pressure or not pressure), but that isn't actually stated. If it isn't fixed, then a whole boatload of subjective judgement comes into play. On the other hand, if it is fixed, it effectively averages out things that are important for the individual plays. The hope is that these aspects of the play average out over a game or a season, but without giving any supporting evidence that's a bit of a leap of faith.

The section on division of credit explicitly states that's what happens on completed passes, leading one to believe that no division of "credit" (blame) happens negative plays. However, some of the older descriptions of QBR (from when ESPN introduced it) included judgement calls on drops, underthrows, and overthrows, among other things. These are clearly subjective to some degree, but also important to include. It's entirely unclear if they are still used in QBR since ESPN won't show us the recipe.

But what is the recipe used by the folks in the OP, and how do we know it was applied consistently?  If it's the details of the recipe you're after, are you truly comfortable ascribing more validity to their conclusions than you are to the conclusions arrived at via QBR and DVOA, given the knowledge we do have of how they're formulated?

[Dolfanalyst]

The other issue we have with the material in the original post is that, despite Tannehill's being ranked 10th overall among quarterbacks, he isn't a standard deviation better than the average of the top 32 QBs listed along any of the dimensions by which they were rated.

According to the website, those dimensions were the following:

•Acc: Accuracy (Graded out of 25)
•Arm: Arm Strength (Graded out of 25)
•Press: Pressure/run threat (Graded out of 20) (Pressure weighted at 15, run threat at 5)
•Dec: Decision-Making (Graded out of 20)
•Pos: Positional Value (Graded out of 10)
•Ovr: Top Possible Score of 100

Tannehill's Z-score (or in plain terms, the extent to which he deviated from the average of the top 32 QBs listed) for each of those dimensions was the following -- a value of 1.0 or more represents at least a standard deviation higher than the average:

Accuracy:  0.99
Arm Strength:  0.79
Pressure:  -0.02
Decision-Making:  0.06
Overall:  0.58

So despite being ranked 10th overall (and rankings can provide misleading information for just this reason), Tannehill is not significantly better (as defined by at least a standard deviation, in my view) than the average quarterback among the top 32 listed in any area in which they were rated by these supposed authorities.

(Granted he's very close when it comes to "Accuracy," and so close in fact that we'll go ahead and give him that.  He's also fairly close in the area of "Arm Strength."  He isn't close "Overall," however.)

This sort of finding is much more consistent than the 10th overall ranking would make him appear with regard to how he's been evaluated by the sorts of sophisticated statistics we've discussed in this thread.  Again, the 10th overall ranking can be misleading in this case.

In fact, the only QBs among the top 32 listed who were at least a standard deviation better than the average of the group with regard to the "Overall" rating were the following:

Aaron Rodgers
Cam Newton
Sam Bradford
Tom Brady
Philip Rivers
Drew Brees
Ben Roethlisberger
Andrew Luck

Everyone else among the top 32 was either non-significantly different from the league average, or significantly below it.

[fyo]

What stands out to me in that table is that you have an absolute stud with a QB rating of 101.8 and a YPA of roughly 8.  I know who that is, and I know he compiled those statistics after 2004.  Had he done it prior to 2004 it would be mind-boggling.

There are actually TWO guys that stand out in that table, significantly higher than the others in both passer rating and YPA. One is the one you're thinking of, of course, the other compiled his stats quite a while before 2004, making them frankly astonishing.

As for yards being a volume statistic, I'm well aware of that. What I was looking at was quarterbacks who were asked to (and managed) throw a whole heck of a lot in the beginning of their careers. Quarterbacks that were, one way or another, asked to deliver far more than your average NFL quarterback. In that company, Tannehill doesn't really stand out much, neither good nor bad. What can be said is that the rest of those guys either had great careers or look like they will have great careers. Not necessarily HOF careers, of course, but still pretty darn good careers. Looking at the guys who are done with theirs, there's really one that sticks out in a bad way and he's perhaps the "bottom" projection for Tannehill. As long as we get a guy like the one who followed that guy, I'm fine with that .

[MyGodWearsAHoodie]

Is number 11 Rodgers?  I would rank 11 the best than 13.  Even though 1 and 2 has a high TD and rate I would rate them lower than most because my subjective opinion is int% is the most important stat. I would prefer a player like 5 or 9 over 1 or 2

[fyo]

Is number 11 Rodgers?  I would rank 11 the best than 13.  Even though 1 and 2 has a high TD and rate I would rate them lower than most because my subjective opinion is int% is the most important stat. I would prefer a player like 5 or 9 over 1 or 2

Rodgers didn't make the list for lack of production. Only players with more than 13k yards in their first 4 years (5 for Brady) are on it.

If you WERE to take Rodgers and remove his pre-starter years, he'd be so far ahead of everyone else it's ridiculous. 17k yards, interception rate of 1.8%, touchdown rate of 6.4%, 105 passer rating.

Your emphasis on low interception rate made me think of Brett Favre. He had a first year like Brady where he didn't really play and so didn't make the list. Had I though of him, I'd have added him manually, even though he doesn't come with a pun like Brady did.

For reference, Favre would have slotted in at #6 with nearly identical stats to #5 across the board.

Considering my original goal was to look at rookie quarterbacks asked to throw insane amounts in the first part of their career, perhaps I shouldn't have added Brady (or any others who sat their first year). Certainly, spending 3 years developing behind Favre isn't equivalent to starting from day 1.

Oh, and #13 on the list is a bit of an odd player, since his first NFL season came when he was 29...