08.09.2020

Confirmation Bias

European Projects Behind the Scenes (4)

Gregor Cerinšek, IRI UL

/kɒnfəˈmeɪʃ(ə)n ˈbʌɪəs/

Definition: The tendency to search for, interpret, favour, and recall information in a way that confirms or strengthens one’s prior personal beliefs or values. It is a type of cognitive bias. People display this bias when they gather or remember information selectively, or when they interpret it in a biased way. The effect is stronger for desired outcomes, for emotionally charged issues, and for deeply entrenched beliefs.

In my introductory post, I have already discussed that what I currently miss in European projects is a more open environment that would support and encourage researchers and project leaders to speak frankly and without reservation about their actual project experiences. This includes demonstrating good practices as well as bad practices, challenges, mistakes, troubles, conflicts, pitfalls, etc. In this manner, they would be able to openly expose real happenings, causes, and activities leading to (un)successful completion of a certain project. Reading project reports and looking at other project representations I would like to become familiar with the realities “behind the scenes” that considerably influence project results and impacts. Unfortunately, due to the nature of reporting and external assessment, these learnings are frequently being lost.

If we think about it more carefully, this principle is crucial for every creativity, innovation, and scientific discovery in general. Experimenting, which undoubtedly leads to mistakes and failures. Or, expressed in a more scientific way; a need to falsify our initial assumptions and hypothesis. Similarly, the famous philosopher Karl Popper described the open society as the one that considers skepticism being its key working principle, refusing and resisting definitive truths.

My ethnographic results show that European projects do not encourage this kind of modus operandi in a sufficient manner. Several of my informants have revealed that they were desperately searching for impacts that were potentially produced by project activities and their developed solutions. It is not an unusual practice that researchers implement diverse (scientific) methodologies in order to prove their hypothesis and convince the evaluators and broader public that the project can be considered as successful. And if creative enough, they also find these kinds of confirmations.

Still, this kind of corroboration could turn out to be at best irrelevant or false, or at worst viciously misleading. A tendency of naïve empiricism has been identified; i.e. searching for confirmations and positive examples that are furthermore treated as evidence. Evidence that corroborates and supports initial assumptions or promises as described in the project application. For instance, project teams search for and present students’ testimonials that positively address a new study programme; how it successfully contributed to skills development. They widely announce the positive results from evaluation questionnaires – involving close-ended questions that were defined in a (frequently unconscious) tendentious manner to lead to anticipated or desirable answers. Consciously (or unconsciously) they consider and report on selected examples that prove that project aims and objectives have been realized.

This is clearly a confirmation bias; and it is crucial that everyone planning and leading projects in this manner should be aware of that, including evaluators, supervisors, and project officers. They should be aware of the fact that a series of corroborative facts is not necessarily evidence and that we can and should get closer to the truth by negative instances, not by verification. And this is especially true for innovation projects. That it is misleading to build a general rule only from observed facts that are in line with a predefined agenda. My research demonstrates that the current system of organization, delivery, and assessment of European projects does not entirely enhance true reflection, meaning that key findings and learning experiences remain uncovered and are not always described in project reports.

Sometimes, a big amount of data presented in 150 pages’ long reports (that, by the way, almost no one reads in detail) can be entirely meaningless. At other times one single piece of information can be of key importance – and this is exactly the type of thick data that we should all search for.

Self-confident, unbiased, and open perspective towards research and development should become a key building block of European projects; primarily encouraged by European authorities. The ability to research and demonstrate the actual reality – without the urgent need to find signs and confirmations that stroke the initial objectives, hypothesis, and expectations as written in the application form. And this is the ultimate key to every discovery. In searching for a new way to India we discover something unexpected that we did not realize or know before. But even more crucial is to be allowed and encouraged to speak openly and publicly about it.

Gregor Cerinšek, IRI UL researcher and project manager, reflects on the background of European projects, i.e. all practices, events and activities that are not publicly discussed, but nevertheless significantly affect the acquisition of projects, their course and long-term success (or failure). He formulates findings and recommendations on the basis of conversations with project managers, coordinators, researchers, supervisors, consultants and evaluators – with all those who are in any way involved in the phenomenon of European projects.

Previous episodes: