The scientific evidence…is often not clear.
Not a day goes by when we’re not faced with another loud nutrition headline:
Red meat causes cancer!
Saturated fat causes heart attacks!
Eggs are as unhealthy as smoking!
It’s frustrating for an experienced nutrition coach like myself to see those headlines without any context or explanation. All of the above headlines are a flagrant misinterpretation of research, some of which wasn’t very good to begin with. Yet clients email me in a panic, wondering if they need to change their diet immediately.
Putting aside the issue of bad journalism (which is an entirely different can of worms), let’s take a look at some of the limitations of scientific research and why you can’t believe every study you hear about.
What is SCIENCE?
The intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment.
Essentially, theories are tested through the scientific process in laboratories around the world (primarily at universities) and hopefully someone comes to a convincing conclusion to share with the rest of us.
It’s a tough process, fraught with imperfections (lack of funding, poor judgement, bad research design, poor control of variables) that constantly threaten to tear the whole thing apart. That’s probably a little overdramatic, but you get the idea.
Who were the subjects?
The subjects of the study are important for interpreting whether the results are applicable to you. If the research was performed on elderly females, do the results really apply to you if you’re a 38-year-old man?
It depends though. Maybe the benefits of a particular food or nutrition intervention are essentially the same and the data can be extrapolated to men. Only further research really clears this up (this is why most studies finish with a statement to the effect that “further research is needed”).
Maybe you read somewhere that high protein diets are hard on the kidneys. Not so. The data that indicated that high protein diets cause kidney issues took place in patients with pre-existing kidney disease. Someone with perfectly healthy kidneys has nothing to fear from a high protein intake. Follow-up research has shown no ill effects from high protein diets in subjects with normal kidney function.
How many subjects were there?
You might be surprised to hear that many published studies have only a sample size of eight or nine people. That’s like trying to find conclusive proof of something from you and a few of the people at your office. Would you consider your office a balanced representation of the general population? I’m guessing “no”.
For many areas of research, even a hundred people won’t get you much interesting information. Many small studies are published nevertheless, and often quoted on websites that make outrageous clickbait claims about nutrition.
Many of the health studies you hear about are done on rats. Scientists say that rodents are used as models in medical testing because their genetic, biological and behaviour characteristics closely resemble those of humans, and many symptoms of human conditions can be replicated in mice and rats. Still, it doesn’t take a PhD to guess that rats and humans differ from each other in important ways.
That study about champagne potentially preventing Alzheimer’s you read about recently (it was actually published in 2013 and was recently rehashed by the media) was conducted ONLY on rats. There haven’t been human studies yet and if you’re clever you can probably imagine all the challenges that sort of study would create (How do you get people to drink only the required amount of champagne? What other components of their diet might affect the results? How long do we have to wait to see if they get Alzheimer’s or not?).
Cherry Picking Supporting Data
Researchers are generally smart, hard-working folk, but they’re also human. They have their own personal opinions and biases. Every research paper has to include pages of supporting references from previous studies that support any statements made in the paper.
If a study doesn’t match your conclusions? Don’t include it.
That’s one of the reasons you can find studies to support completely opposing arguments for just about any area of nutrition.
Do dairy products cause prostate cancer? Some studies say they do and some say they don’t. If I wanted to vilify dairy I could “cherry pick” the studies that showed that dairy caused cancer. The Dairy Board could come back and pull the studies that showed that there was no relationship between dairy and cancer.
Not all studies are equally useful, and have issues with sample size, lack of control for variables, and even poor interpretation of results.
Is your head starting to spin yet?
Meta-analysis is essentially a study about all the previous studies that have been done, and seems like a good bet for making overall conclusions about nutrition facts.
Certain studies are excluded because they’re too small, or the subjects used are not appropriate to the question being answered (excluding studies with elderly women if you’re looking at muscle building in young men). Some of the conclusions end up being vague and often cite the need for further research. They are also inherently biased because of the researchers’ personal beliefs.
What Kinds of Studies?
The double-blind, placebo-controlled study is the gold standard for scientific research. Neither the researcher nor the subject knows whether they are being treated with the drug or not (a sugar pill is given as a placebo). The effects, and side effects, of the drug are then evaluated.
It makes sense for drugs and nutritional supplements, but how exactly would that work for food? There are so many components to each person’s diet that you can’t adequately control for all the different variables of their nutrition and lifestyle. Most nutrition studies are epidemiological, meaning they track health patterns in populations, and they’re unlikely to take individual differences into account. Nutrition data often comes from subjects self-reporting their food intake, which is notoriously inaccurate (just how many drinks did you have last weekend?).
Association vs. Causation
Many studies don’t tweeze apart the difference between association and causation. Just because an event occurs in conjunction with another thing doesn’t mean that one thing causes the other. If you’re not sure what I mean, check out this list of bizarre things that are correlated to each other.
Red meat may be associated with cancer, but this may be because eating red meat (particularly poor-quality processed red meat) is often associated with a “cluster of unhealthy behaviours” like lack of vegetables, smoking, sedentary lifestyle, lots of processed food, and not getting regular medical care. The red meat itself does not necessarily cause the cancer.
What if we could get 1000 active, non-smoking, healthy (no disease, eating a wide variety of vegetables) vegetarians to add a few grassfed steaks a week to their diets for a year and then measured the change to their blood work and health status? Great experiment, right? Good luck finding subjects for that one! Even if you do, there are still so many factors that complicate the issue that it wouldn’t lead to a conclusive result.
The Scientific Mind
I like to think of myself as a scientist because I have a couple of undergraduate degrees in sciencey subjects (sports science and psychology), but I’m not a researcher and only did a few undergraduate research projects. I’m aware of the challenges faced in conducting unbiased scientific research.
I’ve recently begun reading “In Defense of Food: An Eater’s Manifesto” by Michael Pollan and while I don’t agree with all his conclusions about the best way of eating, he does a great job of explaining the issues that occur with nutrition research. There’s so much inaccuracy that it’s often hard to trust the conclusions made.
Your Personal Case Study
The most important study is the one that you’re conducting on yourself on a daily basis. Certain foods make you feel better and get you leaner while certain combinations of food make you feel unwell. Pay attention to what your body’s telling you.
What if a low-carb diet works for 80% of people, but you’re part of the 20%?
What if a vegan diet is dangerously unhealthy for 99% of the population, but you’re the 1%?
Do you ever wonder why large groups of people follow low-carb diets successfully and large groups of people (generally smaller) follow high-carb diets successfullly?
We’re all different!
There are certainly commonalities, but the truth for one person is not the same for another. Gender, race, age, and genetic background effect how you digest food and what benefits you obtain from it.
The Proof is Out There
Very little is actually “proven” by scientific research, so if someone tells you this it’s BS. Studies “suggest” and studies “indicate”, but to say that a study proved anything isn’t accurate.
The associations between smoking and cancer are extremely strong, but not everyone who smokes gets cancer and some people who’ve never smoked get lung cancer. So not only does smoking not cause cancer in everyone, but not smoking doesn’t necessarily prevent you from getting lung cancer.
Playing the Odds
I hope you didn’t take that previous paragraph to say that you can safely smoke.
In this case, the extremely high potential for harm far outweighs the potential benefits of smoking (I really don’t understand what they might be, but smokers say there are some).
The potential benefits of vegetables are very high, and although they aren’t proven to prevent or cure any disease 100% of the time, the body of evidence points to vegetables to being healthy.
So if you eat more vegetables and don’t smoke, science says that you’ll likely be healthier and less likely to develop cancer or heart disease. Also, you’ll smell better.
What does this all MEAN?
Most of us look things up on the Internet or scroll our Facebook feeds and share information if it aligns with our pre-conceived ideas about the truth. If something contradicts our current belief, we tend to dismiss it or actively try to disprove it (hello trolls!).
We need to use our common sense and best judgement because we’ll never really know “the truth” in nutrition. Some things aren’t confirmed by research (yet), but are you really going to wait 20 years for all the research to be more conclusive? We can only weigh the balance of the evidence against our own judgement and experience.
Where do I stand?
I love science and I respect scientific research, but I’m aware of it’s limitations. A single study, whether it tells me that brussel sprouts reduce cancer rate or caffeine helps with fat loss isn’t going to change my behaviour in a significant way (I may contemplate a few more brussel sprouts in my diet, but maybe not).
John Oliver did a great job of presenting the issues with scientific research on his show Last Week Tonight. I highly recommend you watch it to get some perspective on the nutrition studies that you hear about.
When I share information in my articles, I sometimes refer to scientific research and what the current thinking is with respect to a particular topic (fat loss, diets, etc.). What I tell you is also supported by over twenty years of experience in the fitness industry, both as an athlete and as a coach. I enjoy reading some of the studies related to nutrition and fat loss, maximum muscle gain, and anti-aging processes because those are my specialties. I just don’t get hung up on keeping up with everything.
Scientific research is a good general guideline for nutrition recommendations. Part of optimum nutrition is finding the right diet for the individual. Appreciate science and what it seems to be telling us. Just realize that nutrition research hasn’t got all the answers.