Groups often suffer from the hidden profile problem: information is unevenly scattered across members of the group and it is challenging to piece it together. AI can learn hidden representation spaces from observing correlations and patterns. Here I use variational auto-encoders (VAEs) to learn a low dimensional representation of the correlations existing between people sharing the same information. Once learned, I used this representation to deploy a bot to decorrelate people's judgments.
To make informed judgements, it has become second nature to search information online. But how good are groups at collectively retrieving information to make geo-political forecasts? In this study, I experimentally manipulated the group's demographic diversity and group modularity. I found that while people's forecasts were not correlated as a function of their demographic diversity, their judgments became correlated after they used common search engines to retrieve information. In other words, search engines turned demographic similarity into information similarity.
Accurate news are at the basis of an informed populace. But how information adds up in collectives is sometimes counter-intuitive. Is it more important to be individually accurate or collectively accurate? In this study, I show that sometimes being individually accurate is worse for the group. Under these circumstances, it is better to lose a bit in individual accuracy to win in collective accuracy. Individual vs collective incentives can stir the group in the right direction.
...
...
Every Brutal Choice Has Elegance
The HFC program seeks to develop and test hybrid geopolitical forecasting systems. These systems will integrate human and machine forecasting components to create maximally accurate, flexible, and scalable forecasting capabilities. Human-generated forecasts may be subject to cognitive biases and/or scalability limits. Machine-generated (i.e., statistical, computational) forecasting approaches may be more scalable and data-driven, but are often ill-suited to render forecasts for idiosyncratic or newly emerging geopolitical issues. Hybrid approaches hold promise for combining the strengths of these two approaches while mitigating their individual weaknesses. Performers will develop systems that will integrate human and machine forecasting contributions in novel ways. These systems will compete in a multi-year competition to identify approaches that may enable the Intelligence Community (IC) to radically improve the accuracy and timeliness of geopolitical forecasts.
Develop a system that naturally blends humans and machines to augment decision systems.
Forecasting is an interesting problem. The issue with forecasting is that it is hard. And geopolitical forecasting is even harder. It's not like, hey, what do you think the weight of *YOUR FAVOURITE BOVINE* is? It's more like, hey, Will Fidesz and KDNP win 133 or more seats in Hungary’s upcoming parliamentary election? You are trying to predict the probability of events of global relevance. You are trying to compress into a single number between 0 and 1, a lot of information.
Develop a system that naturally blends humans and machines to augment decision systems. Humans are good at human stuff, like jumping to conclusions from very little data. Machines are good at doing machine stuff, like using godzillions of observations before making any sense. How would you even design your system? Do you have machines presenting summarized data to the humans? Do you have humans organizing data for the machine? Do you have machines and humans making decisions together? Or in parallel? The possibilities are limitless.