A priceless look into traditional scouting

One Stan Ternent and the limits of expert judgment

As ever! October is a free preview month! If you want to subscribe ahead of November, please, well:

And if you like what you read, please:

When Stanno met Robbo

A light one today because it’s Friday, but I wanted to look at Richard Sutcliffe’s profile of former Hull scout Stan Ternent over on the Athletic.

Now, this would be easy territory for me to poke fun at traditional scouting and trash a person in his seventies who has spent decades in the sport as a player and a manager. Ternent seems comfortable and the article is a nice place for him to talk about some of his latter accomplishments, and I’m not here to pick on a man looking back on a successful career.

BUT—that’s not to say there aren’t a few things we might glean from it.

Sutcliffe gets Ternent to relay how he successfully pushed then-Hull manager Steve Bruce to sign Andy Robertson and Harry Maguire, two fairly major mainstays in the English game who have been sold on for a pretty penny.

First, I think in general there are too many sports profiles that don’t push back a little bit on their subjects. I’m not talking about being combative, I’m talking about putting some of the things subjects say under scrutiny and in context.

In Ternent’s case, we can’t really evaluate his insight into the future prospects of unproven or unknown footballers unless we have a better understanding of Ternent’s overall recruitment record. For every Andy and Harry, how many misses were there? A robust understanding by all sportswriters of survivorship bias might help encourage them to show a little skepticism on these things.

That’s about as tetchy as I’ll get.

What’s more priceless here to me are Ternent’s anecdotes on the informal rules scouts follow in assessing player ability.

Some background: I strongly believe that experience does matter in traditional scouting and that it’s natural that people who depend on their ability to recruit good players to make a living will learn to find ways to be more or less effective at it.

I think Stan Ternent is no different (but again, we don’t have a full record of his recommendations to judge his record, and that’s even before we talk about the likelihood of scouts picking decent players by chance when they recruit from a professional pool). However, there is a limit to what human beings can tell us about footballing ability.

And I’m sorry to say for those who know me well, this is where I’m going to talk about Paul Meehl and Robyn Dawes.

Algorithms always win

Before you read on, I HIGHLY recommend you read this great Farnam Street summary on the work of Meehl and Dawes, which has consistently shown that “simple algorithms using a few equally-weighted predictive variables” will out-predict even the most experienced expert. The article quotes from both Kahneman and Michael Lewis’ book The Great Undoing, on the work of Daniel Kahneman and Amos Tversky.

Beyond providing a good summary of an interesting topic that really ANYONE interested in football analytics should understand, the article cleverly sums up why algorithms beat expert judgment:

The fact that doctors (and psychiatrists, and wine experts, and so forth) cannot even agree with themselves is a problem called decision making “noise”: Given the same set of data twice, we make two different decisions. Noise. Internal contradiction.

Algorithms win, at least partly, because they don’t do this: The same inputs generate the same outputs every single time. They don’t get distracted, they don’t get bored, they don’t get mad, they don’t get annoyed. Basically, they don’t have off days. And they don’t fall prey to the litany of biases that humans do, like the representativeness heuristic.

Or survivorship bias.

Anyhoo, I would behoove you not to be thrown off by the word ‘algorithm.’ We’re really just talking about a few simple rules. I tried to put this into practice (and caught shit for it from at least one very smart analyst) over on my now-defunct site, frontoffice.report.

It’s in this context I want to look at Ternent’s commentary on his own scouting criteria. Here he is on Robertson:

“I had gone to Kilmarnock to watch another player,” he says about that 2013 trip north of the border. “But it was Robbo who was the one that caught my eye.

“It was a synthetic surface, bitterly cold and he just glided across the pitch. Quick as lightning. Left-sided, loved the ball at his feet and just a natural player.

“I told Steve all about him the next time we spoke and how I had to go back and watch him again. The second game was against Hibs. Robbo was exactly the same, just as impressive.

“His agent was there that night. Liam O’Donnell, a decent guy. We had a chat and it was clear Robbo was a good lad. He was also hungry after being released by Celtic (at 15). Just the sort of lad we wanted at Hull. Really level-headed, just like Harry turned out to be at Sheffield United.

And here he is on Maguire:

“With Harry, I watched Sheffield United a few times,” he says. “I had a few misgivings, namely about his pace. I also felt he could do a little bit better in the air. Don’t forget, we wanted to take him to the highest level, the Premier League, and he had to be right.

“But what Harry did do was read the game very well. To cut a long story short, I did my homework on him. Turned out he was a good lad. He had also played all these games at a young age and I felt he was worth pursuing.

Reading this, I want to know so much more. How many matches is the minimum Ternent needs to make a recommendation? Does he refer to any counting stats? How does he record his observations? What was the homework he did on Harry, exactly?

If you’re going to get some time to sit down with a renowned scout, getting this kind of detail to me is crucial, not having Bruce pat himself on the back for guessing right based on a performance at Wembley Stadium.

But anyway: pace, confidence on the ball, being good in the air, being a good lad, a natural player, loving the ball at your feet…it’s not bad, exactly, but you can see where issues of subjective judgment and internal contradiction might seep in.

Less wealthy clubs need analytics the most

One criticism I here from time to time is that the Hulls of the world can’t afford to make evidence-based decisions. Algorithms. analytics; they don’t have the staff for that.

But the consequences will cost the less wealthy clubs far more. Even a club as dysfunctional as Man United at the moment can coast on their rep for the next ten years and still win all the commercial deals. Clubs like Hull live and die on decent recruitment.

And I’m not even talking about analytics here in the sense of databases and AI-driven models or whatever—what I’m getting at is far simpler. Clubs just need rules and consistency, they need a way to recruit that isn’t dependent on the fox-like senses of any single legendary scout. You need to give guys like Ternent better tools to do an even better job.