When human judgment enters the picture, so too will errors in human judgment. Think of this as “noise,” just as you might think of a signal-to-noise ratio in an audio signal. And just as in listening to music, this noise is not a feature, but a flaw.

In the context of human action, management professor at HEC Paris and former McKinsey senior partner Olivier Sibony defines “noise” as the unwanted variability in human judgment. “When you look at how people make a professional judgment, there is an average error … and that is what has traditionally been called bias in statistics and in the study of judgment. But when you have already identified bias, there is something left, and that is the unshared error, the unwanted variability of errors, that is noise.”

In a new book, Noise: A Flaw in Human Judgment, by psychologist and Bites alumnus Daniel Kahneman, Sibony, and Harvard law professor Cass Sunstein (author of Nudge), the trio look at the lottery that noise creates in social outcomes, and discuss ways to practice better “decision hygiene” to prevent noise from infecting important outcomes.

Coinciding with the release of Noise, Sibony spoke with interviewer David Edmonds in this Social Science Bites podcast about noise as a concept, the types of noise, why acknowledging it matters, and a little on what we can do to avoid it. This is an area of great interest for Sibony, whose own research centers on reducing the impact of behavioral bias.

“Bias and noise,” Sibony explains, “are mathematically equivalent in the effect they have on error. Noise causes exactly as much error as bias does for the same quantity of noise or bias.

“And so, if you can reduce noise, you can reduce error.” Or put another way, make better decisions.

He gives the example of insurance claims adjusters. “When you look at how two of these people judge the same case, what price they set on the same insurance policy or the price they set on the same claim, and you ask them how much they expect to disagree, they say, ‘Of course we’re not going to be in perfect agreement; it’s a matter of judgment, after all. It’s still a calculation – we’re not just adding up numbers and saying, “The answer is X.” Otherwise our job would just be automated. That’s what makes the job interesting – it’s a matter of judgment. So we expect some disagreement between us. But hey! We are all highly qualified, competent people, so we are more or less interchangeable depending on who is available.’”

If you ask the adjusters, or their bosses, about how much variability they expect, the answers come back around 10 percent. And if you ask business executives in general what they would expect the difference to be – and Sibony talked to hundreds -- the answers came back at 10 to 15 percent.

But looking at the actual variability in real life, he reveals, the differences vary by as much as 55 percent.

This isn’t just some peculiarity of insurance. “This was,” Sibony said, “something we found everywhere we looked!” He offers many examples: assessments by financial professionals, x-rays read by skilled doctors, professors grading essays, and many more. What he terms “big differences” appeared repeatedly

“More worrisome, perhaps, if you look at how judges sentence people who have been found guilty of a crime … [W]hen the average sentence is seven years in prison, the average difference, the mean difference between two judges is three-and-a-half.” And so, as Sibony notes, when appearing before two judges, you’ve already been sentenced to five years, or to nine years, “just based on the luck of the draw.”

This variability, this happenstance in outcomes, matters for a trio of reasons: fairness (“when similarly located people are not treated similarly, it is unfair”); credibility of the underlying institutions; and because we’re routinely making bad (or at least not the best) decisions.

In addition to teaching strategy, decision making and problem solving at HEC Paris, Sibony is an associate fellow at Oxford University’s Saïd Business School. He writes often on strategy and decision making in the academic and popular press, and his book Vous Allez Commettre Une Terrible Erreur ! (published in English as You’re About to Make a Terrible Mistake!) received the was awarded the Manpower Foundation Grand Prize in 2019 for best management book. He is also a knight in the French Order of the Légion d’Honneur.

"That’s such a hard question,” Gina Neff, a sociologist at the Oxford Internet Institute, responds when asked what social science research or thinker most influenced her. “It’s like a busman’s holiday for an academic, because so many things have influenced my thinking.” Her answer, by the way, was Ulrich Beck’s concept of the risk society, as explained in this 1986 book.

In this montage drawn from the last two years of Social Science Bites podcasts, interviewer David Edmonds poses the same question to 25 other notable social scientists.

For many of the guests, the answer proves difficult to pin down to just one person or work (“That’s like asking who’s your favorite kid,” was David Halpern’s first response). For a few guests, the response is instant.

“A simple answer, really,” replies Rupert Brown, naming a fellow social psychologist, Turkish-American Muzafer Sherif, and his work in the 1950s. And sociologist Les Back, too, answers instantly: “Don’t even have to think about it: WEB DuBois’s The Souls of Black Folk. DuBois is the writer who captures both the heat and the passion of life and also the cool historical perspective and analysis in the most extraordinary compound of literary expression.”

Back, in turn, was mentioned by one scholar: Kayleigh Garthwaite as her great influencer.

A number of the guests cited titans from the early days of social science – Max Weber, Karl Marx, Pierre Bourdieu, Emile Durkheim, while others named modern-era titans like Stephen Pinker, Daniel Kahneman, Jonathan Haidt or Jean Piaget. And many named creators of the new canon – Jim Scott cited A.P. Thompson’s Making of the English Working Class, Alondra Nelson picked Troy Duster’s Backdoor to Eugenics, and Gurminder  Bhambra tabbed Danielle S. Allen’s Talking to Strangers: Anxieties of Citizenship since Brown v. Board of Education.

And no such list would be complete without a wild card, and for that we turn to Michelle Gelfand, who turned to Herodotus’ The Histories and the lessons a 2,500-year-old post-mortem of an ancient war can teach us today: “He was a brilliant cross-cultural psychologist … he also had a really interesting observation — that all humans are ethnocentric. They don’t just think that their culture is different, they think it’s better.”

This is the fourth collection in this series (and the 100^th Social Science Bites podcast).