I kind of went to town on this one: I analyzed the different tactical options Klinsmann utilized without Donovan or Dempsey, and then evaluated some key players in the attack. Screenshots for days! https://andthenthehex.wordpress.com/2016/01/06/the-us-attack-after-donovan-and-dempsey-part-2/ Also, yet another data point that Donovan and Dempsey rocked. I discuss the Goalimpact scores in the article, but here are the charts:
Thanks for taking the time to put this together and share it! I'm honestly skeptical of this stat. Because of the way it is designed, it will automatically have the biased perceptions of those who created it. As the description you linked to points out... there are so many confounding variables that the signal you get from this could be measuring any of a number of things that may or may not be related to the performance of one particular player.
Thanks for the feedback! Regarding Goalimpact, the model is a top-down approach that looks at goal differential while the player is on the field (Daniel Altman at North Yard Analytics does similar work, but with expected goal difference.) Let's take MLS Cup as an example: after 20 minutes, Portland was up 2-1 against Columbus. At the 72nd minute, Columbus subbed on Jack McInerney for Tony Tchani. The game ended 2-1, so Tchani had a +/- of -1 and McInerney had a +/- of 0. However, Tchani and McInerney didn't face the SAME Portland team: Asprilla subbed on for Melano in the 59th min. Additionally, Tchani and McInerney didn't play with the same teammates, as McInerney never overlapped with Finlay (Cedrick subbed for him 10 minutes earlier.) The Goalimpact model accounts for all of these differences across tens of thousands of players (Jorg mentioned to me that he just started tracking NASL last season!) Goalimpact is an ongoing calculation of a player's entire career, so you can imagine how the relationships grow exponentially after a single season. Here is a more in-depth read of how a single game can change the score of thousands of different players. What's cool is that the model gains predictive power fairly quickly, too. For Donovan, it looked like the model and his performance were near-perfect hit until his first Everton loan. Dempsey started out-performing his initial projection mid-2009.
On a completely different note, this was the most interesting tactical wrinkle I found. In the last 10 minutes of the Holland friendly, we rolled with this unbalanced formation: This was the best screencap I could grab, but still doesn't do the formation complete justice. Morris played as the lead striker, but Wood played and advanced role in the left half-space. He'd track the outside right back, but then attacked on a diagonal to provide numbers forward. This was an intentional shift, as Wood initially subbed on for Johannsson as the lead striker. Diskerud played in advance of Bradley and Williams, but it wasn't a traditional triangle, either. He essentially occupied the space a left central midfielder would in a 4-1-2-3, but he didn't have a partner. Even more intriguing: Diskerud entered the game playing on the right side of central midfield, with Bradley in advance and Beckerman holding. Bradley and Williams were very much a line of 2 in the defensive shape, with Bradley surging forward in attack (not just in this screenshot). Yedlin played right midfield, but would align with Bradley and Williams vertically. Now, I'm fairly sure Klinsmann didn't tell the team to unveil their lopsided 4-3-2-1 in the last 10 minutes of a friendly. However, it was an interesting interpretation among this set of players within a tactical framework (which, if I had to guess, was a standard 4-2-3-1.)
Enjoyed your blog post and the articles referenced. And I tend to agree with your ultimate conclusion that a two-forward/striker tandem approach is probably best for USMNT going forward. Not sure I am as ready as you to anoint Wood as part of that but frankly haven't seen enough of him to say you're wrong either....
If you read the write-up on the GoalImpact website, what you, and the model author, call "top-down," is what is often referred to as a probabilistic model, as opposed to a mechanistic model. The weakness of all statistical models is that they assume that correlation=causation. As the model author rightfully points out, football is so complex that it's virtually impossible to construct a meaningful mechanical model... as such his approach is definitely interesting. However you can build millions of meaningless statistical models that seem to perform very well. (I'm sure you already know this... just trying to explain my skepticism). My bigger concern with this model, though, is the sort of Bayesian component where performances are weighted by league. I'm willing to bet that the "jump" in Donovan's performance based on his time at Everton correlates pretty closely with the arbitrary weighting system imposed on the model. So what you may actually be seeing there is a circular argument, where the assumed difference in quality between MLS and BPL creates the difference in player ratings. In addition, maybe it's simply a wording thing, but the idea that the actual performance can never exceed the expected performance (as seen in the graphs) is also problematic for me. Anyway, I know it's easy to sit back and take pot-shots at people who are doing interesting and creative work like this, so I don't want to be too negative. However, I do want other folks to recognize the potential weaknesses of this kind of approach. Thanks again for posting this... it get's me thinking about this sort of thing, which I enjoy a lot.
I read your entire blog post analyzing these formational shifts, and it's great. I hadn't thought specifically about a lot of those games in that kind of detail. It really makes me wonder what our players are learning from the coaching staff, and how they are trying to implement it in matches. It seems very ad-hoc when you break it down like you have.
Caution - you used the term "Bayesian"! Now we can go deeper (and your statistical expertise likely exceeds mine) I totally agree with you regarding the skepticism of the Donovan jump, but though it *might* be overstated, I don't think it's unreasonable. Even if there isn't a league weighting (though I think there probably is), Donovan played against players with higher Goalimpact scores while at Everton than he did with LA. Everton also performed well both times Donovan was there, and from a glance, disproportionately so when compared to the rest of the season. All of this would increase Donovan's score quickly. However, the model also catches Donovan's early impact on the Quakes within MLS. The Quakes were TERRIBLE just before Donovan arrived. San Jose went from 0.91 PPG in 2000 to 1.73 PPG the next year (side note: attending games in the summer of 2000 was not much fun). I don't have Donovan's +/- , but let's assume Ian Russell and Richard Mulrooney didn't improve massively from one year to the next. Donovan's score also increases sharply in the 2011 MLS season, when LA averaged 1.97 PPG. It's likely that Donovan' personal PPG was even higher. All of this occurred at a time when, based on an average career trajectory, Donovan should be slowing down slightly. If the pattern of Donovan's playing time at Everton - LA - Everton went peak - plateau - peak, I'd be worried about overweighting. As it is, I think the Everton loans were additional proof points that Donovan contributed greatly to club and country in that time period. Finally, I love that you can see the 2013 Gold Cup spike when Donovan returned to national team play.
Thanks for the kind words. Among the three strikers after Altidore - Wood, Johannsson, Zardes - I think Wood works best for virtually any two striker system. He's had the best production (which is always a positive), but I especially like his skillset alongside Altidore. He shoots/finishes well, runs into channels with intent, plays fairly quickly when he needs to, looks comfortable with his back to goal, and shows the ability to pressure high on defense. I do think all four will play a role with the US this year throughout qualifying.
Just one quick note on the league weighting. Here's the quote from the GoalImpact website: While this is pretty vague, basically what I'm getting from it is that there is some sort of human intervention here to determine the quality of the opponent. If there wasn't some sort of ranking used there, then (as they more or less say in the quote) top players in every league would have goalimpact scores (GiS) just as high as top players in the top leagues. As long as the distribution of goals for and against in two leagues are similar, the distribution of player GiS should be similar. In the case of Donovan going to Everton, the bias deliberately built into the model will automatically raise his goalimpact score even if he performs less well then he did in LA (to a point). When playing in MLS, there is an artificially imposed ceiling in the GiS he can achieve as compared to someone in BPL. So while observers may believe that the quality of his play while at Everton was markedly better than in LA, and his GiS reflects that... it's very difficult in this situation to differentiate the signal from the model bias. BTW... I too think it's cool that you can see the bump from his 2013 USMNT performances. The bottom line is that it's pretty much impossible to create a statistical model that objectively rates the quality of players across leagues... as if we didn't already know that.