The influence of experts on crowds success and diversity

Inspired by the wonderful talks I am listening to in these days at ICCSS2016 and by the recent outcome of the UK referendum, I was wondering how the opinions of experts and news can influence the population.

We know the well established Wisdom of Crowds effect: the average opinion is better because uncorrelated noise (how wrong the opinions are) averages out over large numbers, thus enhancing the signal.

We also know that some people are better than others. We call these people experts. Experts’ opinion is better than the average individuals because it is less variable (more clustered) and closer to the true signal.

Question: what effects do news have when they broadcast experts opinions to the whole population?

  • Does the average error of the population reduce after knowing the expert’s opinion?
  • Does the average error of the population increase after knowing the expert’s opinion?

Simulation
I created a Matlab simulation (that you can download here) that tries to answer this question. You can tweak different parameters like:

  • How gullible is the population, that is how much is swayed by the expert.
  • How many people there are in the population.
  • How closer to the true value the expert’s opinion is compared to the population opinion. This is the ratio between the variance of the expert’s error and the variance of the population error (in the graph this is called “expert’s error”, sorry for the approximation)
  • The number of observations that we average across

Results
Below is the result for a population of 100 people. The colour represents the improvement in accuracy (that is the distance from the true value) of the population average from before to after the expert’s opinion is broadcast.

improvement_expert_sim
Improvement in average population accuracy

The contour line shows the areas of the parameter space where the expert improves the average opinion. Warmer colours indicate better improvement. The blue colour means bad news. I was surprised by two things:

  • the contrast between the tiny area of improvement (on the left of the contour line) and the huge area to the right.
  • the magnitude of the improvement (a small improvement) compared to the magnitude of the decrease in performance (a disaster).

Running the simulation will also output another image showing the decrease in diversity of the population opinion. If you don’t think diversity if important check this out.

Conclusions? No matter how an expert is accurate, there always will be some residual errors in their judgement. Broadcasting that opinion to the whole population has the effect of biasing it instead of helping it. The effect seems to be irrelevant at best, but disastrous in the worst case scenario, that is an expert that is not so much of an expert.

What do you think? Please leave me your feedback!

2nd Annual International Conference on Computational Social Science

This week the second Annual International Conference on Computational Social Science will kick off. I have been invited to give a presentation of my work during one of the parallel sessions. This is the first time I give an oral presentation to an international conference and the first time I present my results on my work on representing information manipulation during social interaction through the Opinion Space.

If you happen to be in Evanston, IL, on Friday 24th of June I hope you will ask many interesting and challenging questions!

Collective Intelligence Conference 2016

Today the Collective Intelligence Conference 2016 kicks off!

So excited to be in New York for the first time and for such a great line-up of speakers. Today Scott Page and Hugo Mercier together with others will open the panel on Political Decision Making. This first plenary panel will be led by Henry Farrell.

If you are around on Friday, come along to my poster “Real-time Uncertainty Updates in Interactive and Non-Interactive Social Exchanges”. I will show you how we managed to disentangle interactive and non-interactive communication between people performing perceptual decisions. How do people share their confidences in their answers? How do they affect each other? Do the mechanisms of opinion aggregation differ depending on the way information is communicated?

I would love to hear your feedback!