The Aaron Donald dilemma: Why stats and tape both matter

Is Aaron Donald really an average run defender? That might depend on what you are looking for.

The 2020 NFL season has yet to officially kick off, and I have already missed on a prediction for the year ahead.

In doing so I have brought shame upon my family and the staff here at Touchdown Wire.

You see, dear reader, in our season predictions I predicted that the next football debate was going to be “does the draft really matter.”

Instead, if Tuesday is any indication, the next football debate is “Is Aaron Donald a good run defender?”

The discussion was touched off by analysis from ESPN regarding the running game. Announcing the debut of their “Run Stop Win Rate,” Seth Walder sent this tweet:

Diving into the article, Brian Burke explained the metric as follows:

The running game is all about position and angles, and so are these metrics. First, each block is identified — that’s the easy part. Next, our system determines whether the defender was able to defeat the block. It does this by using a large set of rules based on relative distance to the runner, relative velocity to the runner and many other more complicated measures. A defender doesn’t have to make the tackle to win his block. He can penetrate the backfield to cause a disruption, contain the runner behind the line of scrimmage, or squarely fill his assigned gap to earn a win. If a blocker allows his defender to win, he is debited with a loss.

What they found was as follows. Donald had a Run Block Win Rate of 29% last year, just below the league average of 30%. Explaining the findings further, Burke stated:

But the more we watched Donald’s play, the more we came to the conclusion that the model was picking up on something. The plays marked as losses were losses, and Donald was not winning on run plays at an exceptional rate. According to the RSWR model, winning is possible by a defender:

– Beating his blocker such that it puts him in better position to stop the runner or …

– Disrupting the pocket/running lane by pushing his blocker significantly backwards or …

– Containing the runner, even if blocked, such that he must adjust his running lane or …

– Recording a tackle within three yards of the line of scrimmage.

Now, as you might expect, this article – and the notion that Donald is an average run defender – was not taken lightly. It reached the point when one of Donald’s former defensive coordinators, Wade Phillips, chimed in:

Now there certainly exists an almost natural tension between the two camps: Film Twitter and Analytics Twitter. In one came you have people that “grind the tape” and use film as the basis of their arguments, while in the other you have people that are looking at the game through a new lens, and offering various new methods of thinking about the sport. Some of the advancements offered from Analytics Twitter have been almost revolutionary in how we think about football.

But not everything is perfect.

One of the tensions between the two camps is the idea of context. This was first bandied about when Pro Football Focus began grading players on every single play. The common pushback was “how can you give them a grade if you do not have the playcall?” PFF has a staff of former players and coaches training their analysts, and everything gets checked and rechecked, so over time the product has become an informative data point. Yet the idea of context does still matter when trying to bridge the divide between the film, and the numbers.

Regarding Donald, it is this context of RSWR, on both the micro and the macro level, that might need some refinement.

In defense of the finding, Burke outlined on Twitter that Donald’s penetration was often ineffective against the run. Even though he was breaking through the line of scrimmage, he was not helping his team stop a running play, even with that penetration:

To be fair, not all penetration is equal against the run. If you’re flying upfield and getting five yards into the backfield, but the offense runs a toss play away from you, that’s not getting much done.

Take this play against the Carolina Panthers. Donald is aligned as a 3T, in the B-Gap between the right guard and the right tackle. Watch as he flies upfield, quickly penetrating into the backfield:

This could be exactly the kind of play that Burke is talking about. Donald explodes downhill here, knifing through his gap responsibility, putting pressure on the opposing backfield, and getting immediate penetration.

But Christian McCaffrey simply runs this outside zone play behind him – almost into the area Donald vacates – and picks up five yards.

Penetration? Yes. Effective? Well…

We’ll return to this idea in a minute, but this play illustrates exactly what Donald can do off the snap. He is, as I have described him in the past, a “destroyer of worlds:”

But one of the traits of his that stands out when watching him is this quickness off the snap. Donald can get upfield in the blink of an eye, and it is this ability that can be effective against the run. Watch this play against the New Orleans Saints:

Another 3T alignment, another outside zone play to his side, only this time Donald gets into the backfield immediately, disrupts the ball-carrier in the backfield, and while he does not finish the play for a loss, the cavalry does arrive and stop this behind the line of scrimmage.

Disruptive, and effective.

Which is exactly what the Rams want from him.

They’ll live with plays like the first one – which might get graded as ineffective – because they know he’ll come through on plays like the second. So instead of having him come off the line slower and perhaps hold his ground, putting him in position to stop the first run, they’ll let Donald be Donald, knowing plays like the second will follow.

Nate Tice, who covers the NFL for The Athletic’s “The Athletic Football Show,” had perhaps the ideal analogy for Donald as this debate unfolded:

This is a perfect description. Because you could say that on the first play, Donald swings and misses. But his coaches are find with him doing that, because they know when he connects, the ball isn’t staying in the yard.

Take this play against the San Francisco 49ers, which Burke used in his defense of the conclusion that Donald was an average run defender:

Donald immediately penetrates into the backfield, forces Matt Breida to make a cut behind the line of scrimmage, but fails to make the tackle. In Burke’s view this was ineffective. Others would point out that he handles his gap responsibility and forces the running back to adjust his path behind the line of scrimmage, and in the direction of more defensive help.

Let’s look at this play from the sideline view:

This is a play designed to go to the left side of the offense, but Donald’s penetration forces the RB to cut back to the middle of the line. I usually avoid “screenshot scouting,” but to play John Madden for a moment, look at how this play is set up when Donald forces the Breida to cut:

“You get a seal here, and a seal here, and a seal here.”

If Donald doesn’t penetrate like he does, this play has a chance to go for big yardage.

He might strike out, but when he connects…

In The Genius of Desperation, fellow Touchdown Wire writer Doug Farrar tells the story of the “Stunt 4-3,” and Mean Joe Greene. The venerable defensive lineman, sick of getting double-teamed, wanted to align with an angle towards the center and then split the A-Gap at the snap with speed and quickness. It made the Steel Curtain defense that much more formidable. Andy Russell wrote of this in The Games that Changed the Game:

He hated to get blocked and wanted to make plays. So it occurred to him that because he was so quick, he should just line up inside between the center and the guard, tip his shoulder sideways, and when the ball was snapped, he’d dart through the hole. He was supposed to line upon the guard. [But] he jumped in the gap between the guard and the center, tilted his body, and just blew through that gap, and it was devastating. It was beautiful to see.

Were there times that Greene probably took himself out of a play? Likely. But were the Steelers willing to live with it? Of course, because of what he could deliver in other moments.

Now let’s think of this on a more macro level.

What else has analytics taught us about the game? Passing is king, right? Through the lens of Expected Points Added we know that throwing the ball is usually more important that running it, at least in terms of scoring points. Josh Hermsmeyer, who analyzes the game rather well for FiveThirtyEight.com, wrote this recently, arguing that teams are not passing enough.

He examined Expected Points Added per Play (EPA/P) on first down situations, often viewed as traditional run situations. Specifically, he examined NFL teams’ EPA/P and success rate when running vs. dropping back for a pass on first and 10 facing seven to nine men in the box, 2017. What did he find? Only three teams had a positive EPA/P when running in this situation. The New Orleans Saints, the Green Bay Packers, and the New England Patriots. How successful were they? In terms of EPA/P, the Saints added a whopping +0.03, the Packers a stout +0.02, and the Patriots a league-high number of +0.04.

But throwing the football? Only six teams saw a negative EPA/P when throwing the football in this situation. Every other team was positive, with some teams adding as much as +0.37 (Tampa Bay and Atlanta) or even the Los Angeles Chargers, who added an EPA/P of +0.60 when throwing the football in this situation.

The whole point of the game is to score more points than your opponent, and through the lens of EPA/P we know that the passing game adds more value than the running game. Hermsmeyer’s study is not alone in this regard.

What does quick penetration from a defensive lineman do?

It disrupts a pocket, and throws a quarterback into life preservation mode.

Revisit that Twitter thread of mine linked above, and see how quickly Donald’s penetration leads to problems for opposing passers.

It is a passing league, and disruption is production from a defensive lineman. If passing is indeed king, then stopping the pass is kingly for a defense.

So the point is this. This new metric from ESPN may be completely correct in that Donald’s RSWR is below league average, and that might be completely fine from the Rams’ point of view. But it does not make him an average run defender. The statistic might not directly lead to the conclusion that was made, that Donald is an average run defender. This could be a situation where the topline conclusion might not directly mesh with the underlying evidence.

Furthermore, while the disruptive presence he brings might be ineffective on one play, it might be incredibly effective on the next. Ultimately, the Rams are also looking at him to be a disruptive force, and they’ll live with the times his quickness and penetration runs him out of a play, because there are other times when it puts him right in the thick of things in the opposing backfield.

Or in the lap of the opposing quarterback.

To continue with Tice’s baseball analogy, they aren’t paying him for his batting average, they’re paying him for his slugging percentage.

Aaron Donald has been an exception to a rule. He was viewed as an undersized defensive tackle coming out of college, which led to a bit of a slide down the draft board. But he makes up for that perceived lack of size with his quickness and skills as a penetrator. He might be an exception to this new metric as well, in that his RSWR might be below the league average, but for what the Rams want him to do, he’s anything but.