.

4 Ways to Become A Better Consumer of Research

We are inundated with research studies that are intended to make sense of the complex world we live in. But how do you know if the research you read about is valid?

We are inundated with research studies that are intended to make sense of the complex world we live in. Marketers use it to understand and influence our purchasing decisions, while social scientists use it to understand human behaviors. But how do you know if the research you read about is valid? How do you know if it can be trusted?

Know who paid for the research study: There is an inherent level of bias when conducting research. The ways you ask questions, the order questions are asked, how findings are interpreted, and more, rely on a human element. You want to know who paid for the study to understand what pressure they might have put on the researcher. Does the financier have anything to gain or lose by the study’s results? Although many researchers do their best to eliminate human bias, if a tobacco company financed a study that found cigarettes are good for your health, be cautious.

Know who is talking about the research: When you learn about research results, chances are you are hearing from someone other than the researcher. Most people do not actually read the researchers published paper, and therefore rely on a journalist, blogger, or other writer to make the research findings understandable. The thing is, many journalists have never taken a basic statistics class, much less have formal training in how to interpret research findings or critically evaluate research methodology. Be wary of anyone that interprets research findings without having learned how to properly do so.

Know the difference between causation and correlation: One of the hardest concepts students must grasp when learning how to conduct research is the difference between causation and correlation. Causation refers to one thing causing another, also called cause and effect. For example people, who smoke develop lung cancer – ie. smoking caused cancer. Correlation refers to two things happening at the same time, however if one caused the other is uncertain. For example, girls who watch soap operas are more likely to develop an eating disorder. Did watching soap operas cause the eating disorder, or did the eating disorder cause them to watch soap operas? Or was it that the girls had poor body images, and therefore watched soap operas and had an eating disorder? It is very difficult for a research study to show causation. Most research results show correlations. Anytime someone tells you that a recent research study showed one thing caused another, be critical.

Know that statements of fact are not research: Another difficult lesson students of research much learn is that the research process does not produce statements of fact. You may hear “The research study proved that…” when in reality, the study simply found support for the researcher’s theory. Research typically relies on averages, so research shouldn’t use words like: never, always, none, all. Look for “softer” language in research such as: most, many, some, few.

Let’s see what you have learned. Click this link to see a commercial by 5-hour ENERGY® where the actress discusses a research study they conducted. Pay attention to the four lessons you just learned. What do you see?

The main takeaway for me is that the commercial would lead you to believe that 73 percent of doctors would recommend 5-hour ENERGY®. If you listen closely (and read the fine print) you realize that only 56 percent would specifically recommend 5-hour ENERGY® to their healthy patients that are already taking energy supplements. How many doctors have healthy patients? And how many of those healthy patients are already taking an energy supplement? Most importantly, how did the researcher frame the question? The doctors might have simply responded that if their health patients were taking an energy supplement, they would recommend it be low calorie. And since 5-hour ENERGY® is low calorie, it would meet these criteria.

So what do you think? Do you have other examples of bad research? What about research that you felt was poorly interpreted by a writer? Share your thoughts in the comments!

This post is contributed by a community member. The views expressed in this blog are those of the author and do not necessarily reflect those of Patch Media Corporation. Everyone is welcome to submit a post to Patch. If you'd like to post a blog, go here to get started.

N. Peske September 01, 2012 at 08:09 PM
In my work as an editor and ghostwriter, I deal with research from published journals that is almost always available via Google Scholar. I read the abstract and trust my client, the expert, to have read the entire article--that's the expert's department. Also, when it comes to nutritional information and advice, it can be very confusing to sort out what's what. The media tends to jump on one study that's just been released because it is new. I do rely on certain experts, including my nutritionist, who goes to the big conferences. Thanks for pointing out the correlation/causation issue--that's a common mistake I see in reporting. You described the problem succinctly!
Alan Moore, MS, CFP® September 01, 2012 at 08:30 PM
N. Peske, your comment reminds me of a recent study about the link between the age of fathers and the rate of autism in their children. Some news articles were titled "Autism linked to older fathers" while others were titled "Older fathers cause autism." It is amazing how quickly correlation became causation. Thanks for the great comments. Keep them coming!
Jay Sykes September 02, 2012 at 11:53 AM
My favorite 'use' of statistics to sell a product: "Four out of five dentists recommend sugarless gum for their patients who chew gum." http://www.youtube.com/watch?v=Gf_Jk1zLisg
Larry Booth September 02, 2012 at 08:39 PM
My favorite comment about the use of data from surveys was a letter to the editor in the Wall St Journal in the early 80s. "Torture a statistic long enough and it will confess to anything". You see that in the political rhetoric today; both sides use the same data to justify their position.
Jon September 13, 2012 at 03:17 PM
It's funny that you provide an anecdote regarding unethical Progressives who you know, as though that's evidence of anything. As a "critical thinking Progressive" that right there is a big red flag to me that you generally have no critical thinking skills. That's HOW education works. I'm "looking" for "evidence" of something you're trying to "prove." In that sense I've proven your point that I like to promote those skills... but your railing some awkward bon-mot about critical thinking also implies that you simply believe everything you hear and read. That is actually hilarious. Also, what "studies" are you citing that confirm people who are critical thinking have a "pessimistic outlook on life?" Are these studies done in your living room, asking your relatives to believe sticking a fork in their eye will "feel good," and then triumphantly declaring them "pessimists" when they're critical of your assertion? You are the golden goose to the uneducated bumper sticker collective. And I appreciate you, because you make it easier for me to get ahead in this truth-based world.

Boards

More »
Got a question? Something on your mind? Talk to your community, directly.
Note Article
Just a short thought to get the word out quickly about anything in your neighborhood.
Share something with your neighbors.What's on your mind?What's on your mind?Make an announcement, speak your mind, or sell somethingPost something
See more »