Posted on

Why Epidemiology is Crap

Epidemiology is crap.  Epidemiology Graphic

Epidemiology refers to a type of science that is essentially observational and examines group health.  It looks at a group of people, some aspect or aspect of their diet or lifestyle and some outcome such as disease.  Then it tries to determine what variable (or variables) of diet or lifestyle correlate with some outcome.  And epidemiology is crap science so far as I’m concerned.  Now let me tell you why.

The First Epidemiological Study

In college I took a history of medicine class and learned about what I think the first epidemiological study was.  This was hundreds of years ago where it was observed that people in one part of a city (in France or England maybe?) got sick and those in another didn’t.

So people wanted to find out why.  As it turned out, each part of the city got all of its water from a different well and this was the big variable between the two areas.  So it was logical to look at their water supply for initial clues.

And it turned out that one part of the city was mixing sewage with the water supply.  Shockingly (or not), those people were getting sick.  Voila, epidemiology is born along with this impressive practical conclusion:

Don’t shit in your water supply.

But that’s kind of where it started.  That brings us to today.  Where every new epidemiological study seems to contradict the last.  And where single nutrients or aspects of lifestyle get blamed for some disease.  And where nutritional policy is set based on these observations.

Observations from a science that is not just weak.  It’s crap.

The Egg Study

The epidemiological study that kicked this off was a long-term study of 30,000 people over a span of 13 to 30 years.  That’s a lot of people for a long time which should make it valid, right?

It concluded that a little under 2 eggs per day raised the risk of heart disease by 17%.  And that each additional half egg raised it more.

Case closed, right?  Eggs are bad again.  Or are they? 

Because for some reason I came across an article published on that addressed this.  And it spurred this piece.  Since I have zero clue how valid or not the site is, I’ll be focusing on the researchers and studies they referenced rather than the Undark piece per se.

You will notice similarities in their and my writing simply because I’m quoting the same primary references as they did.  But I will be adding many of my own thoughts to that.

Memory Based Food Recall in Epidemiology

Most epidemiological studies used what is memory based food recall as part of its “method”.  This means that subjects are asked to fill out a food diary based on memory.   What did you eat today?  A week ago?  A month ago?  Last year?  10 years ago?

Can you see the problem here?

Outside of athletes who eat the same thing every day, who the hell can remember what they ate a week ago, much less a month or decade ago.  Even if they could remember, would that brief memory represent the past 10 years of food intake?  Of course it wouldn’t.

And yet that is what this “research” is usually based on: asking people what they ate over some time frame or some time ago and hoping it’s accurate.

So many studies will ask the subjects what they ate at the beginning, part way through, at the end or some combination. Many don’t do that.

In the case of the egg study, the subjects did a single food diary at the start of the study and never again.  The researchers seem to have assumed their diet didn’t change for the next ONE to THREE DECADES.

They couldn’t even bother to ask again at the end for any sort of food recall.  One food recall diary at the beginning of the study.  That’s it.   Do you eat now what you ate 30 years ago?  I doubt it.  It’s pitiful.

But it gets even worse than that.

Memory Based Food Recall is Terrible

The simple fact is that memory based food recall is a terrible method in and of itself.  We’ve known for decades that people are terrible at reporting their food intake.  Most underreport ranging from 20-50% of total calories.

Related: Why Am I Not Losing Weight?

This is why people will report eating only 800 calories and not losing weight.  They are actually eating double that.  Lean people do it, obese people do it, active people do it and inactive people do it.  Even dieticians suck at it.  Everybody is terrible at doing it from memory.

In addressing this, the researchers Archer and Pavla have shown that self-reported intakes are physiologically implausible.  On average, they are nearly 50% below the calories needed for survival.  The data can’t possibly be correct because the people would all be dead if they truly ate that little every day.

But this has another implication.

If self-reported calories are 50% lower than reality, the reported intakes of protein, carbohydrate, fat or any given micronutrient will be as well.  It will all be wrong.   You can’t even assume that they are all 50% low because people mis-report some nutrients more than others.  The whole data set is garbage because everybody sucks at doing food recall diaries.

Memory Based Food Recall is Terrible Part 2

In a detailed paper on the topic, they state the following

Unlike most research findings (7) the lack of credibility of M-BM [Memory-Based Methods] data has been replicated consistently over the past 3 decades. (3, 8, 9) We know of no other data collection tool (in any field of actual science) that demonstrates a significant decrement in performance each time it is used, yet that is precisely what was found in the Energetics study.(10)

Now, reference 10 is a paper by Arab et al.  It gave the same subjects 8 dietary recalls over the span of 2 months.  Some were in lab, some were online, I think some were mailed.  Now, they were looking at the tolerability of people doing these things more than the numbers.  Basically, will people do these damn things a bunch of times?  The answer apparently is yes.

But when Archer and Pavela analyzed the data, they found something interesting, again quoting from the above reference

Over the administration of eight 24-hour recalls, Arab et al(10) reported a statistically significant decreasing trend for energy (−535 kcal/d; P<.001) that varied by macronutrient (protein: −22 g/d, −88 kcal/d; fat: −27 g/d, −243 kcal/d; carbohydrate: −54 g/d, −216 kcal/d)

So every time the people did the food recall for the day, they kept reporting lower and lower intakes.  Even if the first report was accurate, the more that people do it, the worse they get.  And there’s nothing to indicate that the first one was accurate.

Researchers in actual sciences will try to validate their methods, taking the same measurement twice a few days apart to see how close they are.  If the method is good, it’s in the usually in high 90% range with relatively low variation.   It has to be otherwise measuring before and after the study is useless.

In the case of epidemiology, relying on memory based recall, not only are the self-reports awful, they get worse with every use.  Even by the same person.

But that’s not the only problem.

Everything is Multi-Factorial

I say quite frequently that the word multi-factorial should be tattooed on everybody’s eyelids.  Because everything is.  No single food or nutrient causes a disease or makes you more or less healthy.  No single aspect of lifestyle causes health problems.  It’s the combination of all of it.

It’s Always Multi-Factorial

Because here’s what epidemiological studies usually try to do: link some single aspect (occasionally more) to the outcome they are examining.  So boom, food recall, eggs correlated with an increased risk of heart disease.  The rest of the diet?  Irrelevant so far as they are concerned.  It was the eggs and only the eggs.

Yes, I know that studies attempt to separate these factors out to one degree or another.  It doesn’t change anything.

And you can’t do that, conclude that a single aspect of an overall lifestyle causes a single outcome.  Well you can but it’s a crap method of doing “science”.  And that’s epidemiology.

In the context of diet, the fact is that you have to consider the whole diet.    As a researcher named Michael Blaha wrote

it is impossible to disentangle the effect of one particular food or one macronutrient from the accompanying foods and macronutrients that characterize a typical dietary pattern.

Because not only do we eat a mixture of foods on a daily basis, certain food patterns tend to cluster.  So people who eat a lot of fatty charred meats often have an overall bad diet.  They don’t eat many fruits and vegetables.  They usually eat a lot of refined carbohydrates.  Maybe drink sugary sodas.  Yet somehow one thing is to blame.

By the same token, people who eat leaner cuts of meat show common patterns.  More fruits and vegetables, less refined carbohydrates, you get the idea.  Their micronutrient intake is assuredly different and better as well.  It all contributes.

 No matter how hard you try, you can’t separate out a single
food from the overall pattern.

But wait, there’s more.

Overall Lifestyle Patterns

It goes deeper than that to lifestyle.  Because you see common lifestyle patterns associated with certain intake patterns.  The person eating the fatty charred meats, etc. is likely to be inactive, carrying excess bodyfat, may drink, smoke, be under a lot of stress.

In contrast, the person eating lean proteins and fruits and vegetables is probably active, leaner, doesn’t smoke, doesn’t drink (or not to excess), etc.  Their whole lifestyle is better because people who do certain things in their life usually do other related things.

It’s moronic to blame a single variable.

A good example of this was the whole thing about meat intake and colon cancer.  You might have seen recent articles asking “Did we draw poor conclusions?”  Yup.

As one paper states

We argue that claims about the health dangers of red meat are not only improbable in the light of our evolutionary history, they are far from being supported by robust scientific evidence.

First off idiots conflated all types of meats.  So super fatty steak or processed deli meat was considered the same as lean chicken breast or tuna.  You can get very lean cuts of red meat as well.  And you can’t group them all as “meat”.  Well you can but you’re an idiot if you do.

I’ve seen people literally synonymize meat with fat.  As if you can’t find low- or non-fat dairy, lean red meat, skinless chicken breast, tuna or eat egg whites. And even when you point this out to them, they won’t budge. It’s baffling that people can be that dumb.

Intervention studies, real science where they take two groups and do a specific intervention shows that lean red meat is extremely healthy.  It’s fatty charred meat with a background of an overall shitty diet and lifestyle that is the problem.

I’ve written about this on the website and in The Protein Book.

The Protein Book: A Complete Guide for Coaches and Athletes

Honestly, the bigger the issue with colon cancer is more about the lack of fiber intake than the presence of meat.  It’s simply that people who eat lots of fatty meat typically don’t eat lots of fruits and veg.  And you can’t separate the two.

It’s Multi-Factorial.

Please get that tattoo or at least internalize it before you start thinking ONE thing causes ONE other thing.  It almost never does.  Yes, falling out of a plane without a parachute is pretty predictive of death.  But when you’re talking about diet and lifestyle….

Bias in Epidemiology

It’s interesting to read epidemiological studies.  Because it seems that, more often than not, the conclusions reached tend to reflect the belief of the researchers.  This happens in all sciences to one degree or another but the topic today is this.

The China Study is probably the best example I can think of.  Done by a rather militant vegan/vegetarian, it purported to link dairy intake to cancer.  It did this by looking at cancer rates and dietary intake in various provinces of China.  It correlated everything with everything and drew the conclusions the researcher already believed: animal proteins are bad.

Adding to the hilarity, the researcher then tried to provide a mechanism for the cancer increased based on dairy impacting IGF-1 and IGF-1 in cancer prone rats.   There was no direct link, just this attempt at algebraic inference.

Animal research is also generally crap relative to humans but that’s a different video.

And it was total bullshit.   It left out contradictory data and basically just correlated everything with everything until it got the results it wanted. Denise Minger took the study apart in detail and I’ll point you to her piece if you’re interested.  

And I’m not saying that pro-meat epidemiological studies are any less crap.  I think it’s all crap and don’t pick and choose which to believe based on my own biases.  I ignore it all.  Epidemiology is crap.

That’s why we do intervention studies, actual science.  Where you take two groups and split them and have one do one thing and the other another thing and see what happens.  Because that’s actual science where you can determine things a little more clearly.

And invariably when you do that, the results often contradict the epidemiological stuff. Though it’s funny when nutritional ideologists will ignore intervention studies and only rely on observational ones when it should be exactly the opposite.

Epidemiology is Crap

And that’s why I think epidemiology is crap.  First, it’s based on a shit method of determining food intake or lifestyle.  Second, it assumes that not only is that recall accurate, which it isn’t, but doesn’t change over time. Third, it tries to separate out a single variable from the multi-factorial lifestyle issue.  And then of course the real problem hits: clickbait news media over-reports it.

As Archer and Lavie put it in a letter to the National Academies referring specifically to epidemiology:

Nutrition is now a degenerating research paradigm in which scientifically illiterate methods, meaningless data, and consensus-driven censorship dominate the empirical landscape

And as Dr. Anthony Pearson put it in a blog on Medpage today

Rather than drastically cutting egg consumption, I propose that there be a drastic cut in the production of weak observational nutrition studies and a moratorium on inflammatory media coverage of meaningless nutritional studies.

Which are just very scientifically sounding ways of saying what I’ve been saying for years.

Epidemiology is crap.

Facebook Comments