Studies said that …………….

Everyday newspapers report astounding discoveries by scientists and researchers. Some of this research is done upon absolutely new topics. There are also tests done on existing topics where the new study either agrees or disagrees with existing studies done on the subject matter. The ones that get highlighted are usually (notice the word “usually”) done by much acclaimed people with a lot of credentials. So for laymen like us it is easier to agree with them rather than disagree. After all, who are we to agree or disagree with these experts? 

 Now this raises a couple of very interesting questions. At least they did for me. Firstly, how sound are these studies? Are they unbiased and done through rigorous research? Secondly, how do we react when we see these studies? 

  I got the answer of the first question when I listened to numerous Econtalk podcasts. For the unaware, Econtalk is a talk show hosted by Professor Russ Roberts of George Mason University. What I realized is that these research findings which frequently find the way to the newspaper headlines are not as sound or rigorous as we may have assumed them to be. The distinguished researchers, many from top notch universities of the world can be quite biased. They frequently manipulate the data (data fitting) to ensure that the outcome matches with what they want to show rather than what the truth is. 

 The newspapers only mention the summary of the findings. However, we rarely have access to the data set used to come to the conclusion. Even if we had access to all the data, how many of us would have to look at it to check for mistakes. If another unbiased scientist using the same methodology came to a similar conclusion then only we can call the study quite robust. 

 However, just because some people decided to be less truthful does not mean that there are no good researchers. There are people following rigorous methods to come to conclusions. There are also people who cite the weaknesses in their methodology while submitting the results. Thanks to these people human beings have advanced quite a bit.

 Now we come to the other interesting part. How do people react when they learn about the outcome of a new research finding? This is where things get even more interesting. I have seen the same person saying that there a new research on healthcare saying that XYZ food has harmful side effects and the same person (another layman like me) on a different occasion trashing a separate research on a different food item . What has happened here? “Confirmation bias” has happened. 

 How do we actually decide which study to believe and which not to? Like I said, it is all about confirmation bias. We just believe the one that we want to believe and ignore all others. Other times, we also do a one person study by ourselves where the only data set is us. Let me take an example of a study which concluded that exercising makes us healthier. Normally I would just quickly relate to myself and try to remember whether that applied to me. Maybe, I did exercise in the past, but I never lost any body fat. I would then quickly say that the research is rubbish because it never applied to me. 

 What did I wrong here? Practically everything. Firstly, there are many variables that can influence ones health. My one person study did not control for all the other variables. Secondly, statistics gives a viewpoint about a large sample. I could very easily be an outlier. But that would not necessarily mean that the original research was wrong. Thirdly, I may not have even done the exercises correctly and cheated on form. There could be many more.

 Biases are and will remain an integral part of human life. However, it would do all of us good to try to minimize these biases. The first step is to be aware of them. Only then can we try to minimize them while thinking or analyzing. 

 Easier said than done (saying from personal experience).