Experience vs Evidence

A critical issue arises.  We convene an august group of experts to address the issue and recommend ways forward.  The expert panel recommends courses of action.  Those empowered to act embrace one or more recommendations.  They endorse and resource ways forward.

Does this work?  Sometimes experiences can preempt needs for evidence.  Vietnam, Gulf War, Opioids, Covid, and Climate Change provide compelling examples of where expert opinion has been ill-informed and often outright wrong.  We need agreed upon empirical evidence upon which to base our decisions.

A classic counterpoint is Haskins and Margolis in Show Me the Evidence (Brookings, 2014).  In conflict with their original intentions, they illustrate the impacts and strengths of an evidence-based approach.  The prestige and seniority of someone’s position provides credibility to positions that are not necessarily empirically sound.

My colleague Gary Klein has provided important findings on the value of intuitions of experts, especially when the time frame for acting is short.  However, these findings only hold when the new situation is comparable to situations for which people are expert.  Otherwise, the delusion of the ubiquity of expertise emerges.

People who may be expert in chemical physics often seem only too willing to pontificate about social science.  In fact, they seem willing to pontificate about anything.  Anyone who has won a Nobel Prize or a Super Bowl seems to be expert at everything.  We are all too willing to accept the prognostications of one who has accomplished anything.

I am not arguing against the value of experience.  Important and valid insights often emerge from immersive experiences.  Yet, having been in a position of authority does not imbue anyone with expertise on everything.  In fact, I am most impressed with people who, having been in such positions, now know how to marshal evidence to support their positions.  Such people are invaluable.

Leave a Reply