The influence of experts on crowds success and diversity

Inspired by the wonderful talks I am listening to in these days at ICCSS2016 and by the recent outcome of the UK referendum, I was wondering how the opinions of experts and news can influence the population.

We know the well established Wisdom of Crowds effect: the average opinion is better because uncorrelated noise (how wrong the opinions are) averages out over large numbers, thus enhancing the signal.

We also know that some people are better than others. We call these people experts. Experts’ opinion is better than the average individuals because it is less variable (more clustered) and closer to the true signal.

Question: what effects do news have when they broadcast experts opinions to the whole population?

  • Does the average error of the population reduce after knowing the expert’s opinion?
  • Does the average error of the population increase after knowing the expert’s opinion?

I created a Matlab simulation (that you can download here) that tries to answer this question. You can tweak different parameters like:

  • How gullible is the population, that is how much is swayed by the expert.
  • How many people there are in the population.
  • How closer to the true value the expert’s opinion is compared to the population opinion. This is the ratio between the variance of the expert’s error and the variance of the population error (in the graph this is called “expert’s error”, sorry for the approximation)
  • The number of observations that we average across

Below is the result for a population of 100 people. The colour represents the improvement in accuracy (that is the distance from the true value) of the population average from before to after the expert’s opinion is broadcast.

Improvement in average population accuracy

The contour line shows the areas of the parameter space where the expert improves the average opinion. Warmer colours indicate better improvement. The blue colour means bad news. I was surprised by two things:

  • the contrast between the tiny area of improvement (on the left of the contour line) and the huge area to the right.
  • the magnitude of the improvement (a small improvement) compared to the magnitude of the decrease in performance (a disaster).

Running the simulation will also output another image showing the decrease in diversity of the population opinion. If you don’t think diversity if important check this out.

Conclusions? No matter how an expert is accurate, there always will be some residual errors in their judgement. Broadcasting that opinion to the whole population has the effect of biasing it instead of helping it. The effect seems to be irrelevant at best, but disastrous in the worst case scenario, that is an expert that is not so much of an expert.

What do you think? Please leave me your feedback!