Some insight into why Philbin was fired and Gase was hired.

[Spider-Dan]

Can we answer any of those questions?  If we can't, then we can't determine the reliability of those subjective grades, and if we can't determine reliability, we can't determine validity.

When we rest on completely objective information, however, there is no need to assess reliability.  A yard is a yard is a yard.

But you aren't relying on objective information; you're citing "pass blocking grades" and "DVOA."

So which one is it?  Are we to rely only on objective stats, or are we to cherry-pick the subjective stats that fit our desired outcome?

[Dolfanalyst]

Reporters that have talked to the teams that use PFF.

I'm certainly willing to learn something.  Can you provide those news stories?

[Dolfanalyst]

But you aren't relying on objective information; you're citing "pass blocking grades" and "DVOA."

So which one is it?  Are we to rely only on objective stats, or are we to cherry-pick the subjective stats that fit our desired outcome?

I dislike subjective statistics altogether.  Again, someone else brought up PFF's pass-blocking grades, and I merely pointed out that they didn't correlate with quarterback measures.

DVOA is entirely objective.

[Rich]

DVOA is entirely objective.

This doesn't sound objective to me.

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run?

http://www.footballoutsiders.com/info/methods#dvoa

[Dolfanalyst]

Yep. Go to Google and do a search. That's about as far as I'll help ya. Which is farther than you go to provide your sources.

Here was the first one that came up:

PFF grades are not great when viewed by themselves, but can add value when combined with other statistics.  I see a lot of potential in the grades assigned by Pro Football Focus, but I think, for now, it’s a potential not fully realized.  Here’s why (and here’s why an analysis of 2008 data may be as applicable as an analysis of 2013 data): PFF normalizes its raw data (you can read more about it here).  But they use the same normalization factors that they have always used.  If you’ve read Nate Silver’s The Signal and the Noise, you know how important it is to review, test, and adjust your methods.  And repeat.  And then do it again.  It’s not clear to what extent PFF is doing this.  So just like navigating the steep grades on route 322, exercise a fair degree of caution when considering their grades.

http://www.bleedinggreennation.com/2014/6/14/5808148/how-intelligent-are-pro-football-focus-grades-NFL-Philadelphia-eagles

[Dolfanalyst]

This doesn't sound objective to me.

http://www.footballoutsiders.com/info/methods#dvoa

And now post the rest.

[Rich]

And now post the rest.

There is still a level of judgment involved in their analysis, just like PFF has their grading system and there is a level of judgment involved.

As long as a human being is watching film and trying to determine all of the variables in a play, there is going to be a level of judgment. Therefore, it is subjective.

[Rich]

ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 26 seasons, 1989 through 2014, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2014, Marshawn Lynch had a DVOA of 23.1%, what we are really saying is “In 2014, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the keeper when necessary, had a DVOA of 23.1%."

Hmm.........

[Spider-Dan]

DVOA is entirely objective.

From the link above:

"On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success."

Why is 45% of needed yards considered a "successful play" on first down?  Why not 60%?  Why not 80% for second down?

"A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point."

Why is a 10-yard play worth three points?  Why not two points, or five?
Why is -1 point assigned to a 3-yard loss and not a 5-yard loss?

DVOA is entirely subjective.

[Dolfanalyst]

From the link above:

"On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success."

Why is 45% of needed yards considered a "successful play" on first down?  Why not 60%?  Why not 80% for second down?

"A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point."

Why is a 10-yard play worth three points?  Why not two points, or five?
Why is -1 point assigned to a 3-yard loss and not a 5-yard loss?

DVOA is entirely subjective.

Pertaining to the part I bolded (above), it's because of the probability associated with getting a first down on the basis of those downs and yardages.

Show some openness here.  Christ, you just started reading about the thing five minutes ago.  Learn something.

[Rich]

The only objective stats are raw stats. They aren't trying to tell you something, they just are what they are.

Unfortunately, raw stats can and often times will, lie to you.

[Dolfanalyst]

Hmm.........

But the PFF guys can do that, subjectively, whoever they are....

[Rich]

But the PFF guys can do that, subjectively, whoever they are....

I'm not arguing that PFF is subjective, you're arguing that DVOA is objective.

Hint: you're losing the argument badly.

[Dolfanalyst]

The only objective stats are raw stats. They aren't trying to tell you something, they just are what they are.

Unfortunately, raw stats can and often times will, lie to you.

And that's why you have to know how to evaluate statistics, so you can know when that's happening, which is precisely what I'm doing here with regard to the PFF subjective data.

[Spider-Dan]

The assigning of the value 1 point to a gain of 45% of the needed yards on first down is subjective.

Yards are objective.
Points are objective.
Assigning weighted values to the "success" of a play (outside of the yards it gained/lost, or points it scored) is quite obviously subjective, and it's absurd to argue otherwise.