Shot Matrix IV: Initial Studies on the Persistence of Shot Quality

I have been writing here about expected goals rates from shots from different areas of the pitch, assisted by different kind of passes. I've demonstrated that shot quality differs wildly based on shot location and pass type. If I'm going to apply this information to my Premier League power rankings and projections, though, I need to know whether different teams actually take shots of different levels of quality, as a persistent trait. If all clubs, over time, average out to take shots of similar quality, then for projection purposes I should not consider shot quality. But that's not what I've found.

I took 68 club-pairs, looking at clubs that have remained in the Premier League over the last five seasons. For each club, I calculated their expected goals rate, that is, the average quality of each shot taken in a Premier League match. I compared a club's average shot quality in the first season to their average shot quality in the next season. For reference, an average shot has a goal expectation a little under 9%, and you see different clubs generally vary between 7% and 11% for team shot quality. Here's a graph of all 68 points. (Nerdery to follow.)

As you can see from the graph, the data don't align in a perfect straight line, but in general clubs with a shot quality of <.09 in year 1 will have a shot quality <.09 in year two, and vice versa. It is not absolutely determinative, but shot quality looks like a persistent club trait.

(See Appendix: Nerdery below for full analysis.)

Team Shot Quality: An Effect of Tactics

Ok, so statistically, Premier League clubs have a persistent tendency to take, on average, better or worse shots. But why? Well, obviously, it must be about tactics. As I showed in the article on through-balls, EPL clubs have strong tendencies to either attempts lots of shots of through-balls or very few through-balls. Arsenal have taken 15% of their shots in Zones 3, 4, and 5 from through-balls this year, and that's actually a very low number for Arsene Wenger's club. They have averaged a through-ball rate of over 20% from 2009-2013. Tony Pulis' Stoke City averaged a through-ball rate of around 5%. These are clear tactical effects.

And these tactics should have an effect on shot quality. Through-balls produce high-quality chances. A club with more through-balls is likely to be trying higher quality shots, and that's precisely what we see in the case of Arsenal. A whole bunch of those points in the top right corner of the graph are Arsenal. (Arsenal's persistently high shot quality is surely a real effect, over the sample size there is a less than 1% chance of that occurring randomly.)

You can also see this effect in the rate of shots off crosses. Before Paul Lambert took over Aston Villa last year, they had averaged a rate of slightly under 60% of shots from close/central areas (Zones 1, 2 and 3) assisted by crosses. Since Lambert took over, the Cross% has dropped to about 45%. I'll talk more about these descriptive tactical stats in a later post.

So, clubs have different rates of shot quality because of different tactics, and these tactics change when a new manager comes in. So in the graph above, some of the points where a club drops or jumps in average shot quality might happen when a new manager takes over. We can certainly see that in the case of Spurs. Under Harry Redknapp, Tottenham Hotspur averaged a shot quality of ~8.5%, slightly below league average but pretty normal. Under Andre Villas-Boas, that shot quality rate has dropped to 7.5%, among the worst in the league. AVB frees his clubs to shoot from distance more than any other manager in the Premier League, leading to high total numbers of shots but low rates of shot conversion.

For the next graph, then, I've removed from consideration all clubs that changed managers either between or during seasons. This leaves me with 34 matched pairs of season. They are displayed to the right here:

You can see the clustering along a horizontal line running from the bottom left to the top right quadrant. And you can see how empty the top left and bottom right quadrants are. Clubs that keep the same manager show a persistent tendency to take, on average, shots of the same rough average quality year-to-year.

Now, one of the peculiarities of shot quality is that there are lots of ways, tactically, to take good shots. Arsenal are the classic example, they pass and pass, looking especially for through-balls into dangerous areas, and eventually look to "pass it into the net." The traditional passing game is our primary model of high-quality shot-taking, but in fact it's not the only one.

The exact opposite tactical method, the "go 4-4-2 and put crosses in" strategy, also produces high-quality shots. If your primary strategy around the box is to get the ball wide and try to play crosses into dangerous areas, you're going to be taking relatively few low-value speculative long balls. And even though shots off crosses from close areas have lower expectation than normal shots from these areas, they're still a lot better than shooting from outside the box or from wide areas in the 18-yard box. A bunch of those dots in the top right are Arsene Wenger's clubs, but another good portion are Tony Pulis. These are their Shot Quality rates for years they managed, for Wenger 2009-present, for Pulis up through last season.

• Wenger: 9.9%, 10.3%, 10.7%, 9.6%, 9.0%
• Pulis: 9.7%, 10.8%, 9.5%, 9.5%

You can see a similar effect with West Ham before and after Sam Allardyce. West Ham averaged shot quality of about 8% in the two seasons under Gianfranco Zola and Avram Grant. They took about 47% of their shots from outside the box. With Allardyce in charge, the rate of shots outside the box has dropped to about 36%, and their average shot quality has jumped to over 10%.

So, shot quality rates are not a perfect proxy for tactics. Lots of different kinds of tactics can produce either high or low rates of average shot quality. But if a club maintains a particular tactical scheme, they will likely have a persistent level of average shot quality, either low or high. It's not determinative year-to-year, but it looks like a real effect to me.

Appendix: Nerdery

I laid out my statistical methods for testing persistence of a stat in perhaps over-elaborated form in my article on Spurs' shockingly low goal conversion rates. I'm going to be looking at both R-Squared correlation, for which numbers above as close to 1.0 as possible are best. G/SoT I found had an R-Squared correlation of 0.01, meaning basically no relationship. For the tables above, I got an R-Squared of 0.39 for year-to-year correlation of shot quality, and it increased to 0.49 when I looked at teams which maintained the same manager. Those are pretty good numbers.

The other way I'm testing the persistence of statistics year-to-year is with Root Mean Square Error and Mean Absolute Error. These statistical tools both test how well last year's shot quality numbers project next year's shot quality. Do I get larger errors when I project a club's average shot quality using their previous season's average, or when I just use league average. I found in the G/SoT article that league average G/SoT projects future club G/SoT much better than club G/SoT over 11 matches. Here, I find instead that I get smaller errors using a club's previous year shot quality. In the following table, if club shot quality is a good predictor of next year's shot quality, then "RMSE Club" will be lower than "RMSE League" and likewise for MAE. The units here are expected goals, an error of 4 means that the projection missed by about 4 expected goals in the next season.

The quality of shots a club takes in year 1, then, is a better predictor of the quality of shot they'll take in year 2 than a total regression to league average. When I look to project, I will use some regression (a rough 25% regression improves RMSE by a little bit), but I will consider team average shot quality an important figure in estimating expected goals from the shot data.