So I had originally said I would leave this be, that this wasn’t a rap battle, after writing my last detailed criticism of the recent Brad Schoenfeld study. Well clearly that’s not the case.
More on the Statistics
First let me point readers to a thorough analysis of the statistics used in Brad’s paper by Brian Bucher. Basically he takes them apart and shows that none of the THREE metrics supports their strongly worded conclusions.
None of them.
In this vein, here’s something interesting.
Brad and his group have NEVER used Bayesian statistics until this paper. I searched on my folder of his papers and the term Bayesian shows up 4 times. Three are papers that Menno Henselmans was on and it’s his email address. The fourth is one of James Krieger’s meta analyses. At best James has used them before.
Now I find this interesting because there is no way to know if Brad and James had planned to use this approach ahead of time. James has asserted that they did but this cannot be proven. Here’s why: in research it is common to register trials before doing them. This is required in medical research by the Declaration of Helsinki (note that not all journal choose to follow this).
Basically you outline your goal, hypothesis, methodology, what statistical methods you intend to use. This prevents researchers from gaming it after the fact, using different statistical methods to try and make an outcome happen. This is common in research, you just keep throwing different statistical methods at your data until something says it’s significant. Registration is just another way to reduce bias or, in scientific terms, shenanigans.
Brad Schoenfeld appears to have never registered a single trial of his. Again, apparently he is above scientific standards. But this allow for the following potential to occur.
They gather their data, colored by every issue I already described. Now they run standard P value stats and find that there’s no difference between moderate and high volume. That’s what Brian’s analysis showed as the subscript on the 3 and 5 set groups is identical. They were NOT different from each other by that method. Both were better than low volume but moderate and high were IDENTICAL statistically.
In the absence of registration, there is NOTHING to stop them from applying another method, say Bayesian to try and make their desired outcome happen. Which they did. Even here, the Bayesian factors were weak, approaching not worth mentioning by standard interpretations. This didn’t prevent them from making a strong conclusion in the paper or online. Brad is still crowing about it despite the simple fact that the statistics do not support the conclusion.
But there is no way to prove one way or another that they did or did not do this. But without registration of the trial, they can’t prove that they didn’t. And the onus is on THEM to do so. Somehow I doubt that they will provide said proof. But registering the trial would have prevented yet another criticism of their paper. More below.
More on the Individuals Involved
Before I get into anything else, I want to examine how three individuals involved in this responded to both my original and more recent criticisms.
Brad Schoenfeld: Brad ducked them completely. He said he wouldn’t respond because I insulted him. You know who else told me that? Layne Norton when I took him to task over reverse dieting years ago. Then Brad left my FB group and blocked me on Facebook. Because it’s always easiest to win an argument when you don’t allow for dissenting opinions. Gary Taubes, Noakes, Fung and Layne do the same thing. It’s standard guru operating procedure. Just make any criticism you can’t address disappear. If anyone had done it to Brad you’d have never heard the end of it. But he’s above the law.
I’d note that I never removed Brad from my group. He left voluntarily, presumably because he got tired of seeing people ask him to address my criticism when he couldn’t. So he just punked out completely. I’d also note that he played the “you don’t even science” card on a couple of critics. That’s a Layne Norton tactic too. Typical guru approach.
I’d add that Brad also blocked Lucas Tufur (who wrote an excellent article on their paper) as well for the mere act of “suggesting Brad was misinterpreting a study somewhat”. Typical guru behavior and don’t pretend it’s anything else.
Science is based on discourse and debate and that is how it progresses. Honest scientists embrace debate because it gives them the opportunity to defend their work (flatly: if you can’t address criticism, perhaps your work is not as strong as you think).
When Brad writes letters to the editor about papers he doesn’t like, he expects a response. But just as with blinding and randomization and Cochrane guidelines, Brad is clearly above the scientific method. He gets to guru out. Others do not. The behavior he would never allow anyone else to engage in is acceptable for him and him alone. Well and other gurus like Layne Norton who built himself up as the “anti-guru” until he became one himself. The standards he held ALL OTHERS to stopped mattering when he was selling reverse dieting (seminars on which bought him a mansion).
Eric Helms: Now in a sense Eric Helms has no dog in this fight in that he wasn’t involved with the paper. Except that he does because of how he dealt with this issue. In email I had asked him about it and told him my issues and he debated back and forth before telling me he hadn’t even read the paper. But he was already defending Brad.
In my group (and note that I tagged him, forcing him to get involved) I asked him about Brad’s misrepresentation of the Ostrowski data. His response, a total deflection was “But you’ve done it too.” I asked him when. And now get this: he referred me to a NINE YEAR OLD blog post I did on FFMI. NINE YEARS OLD.
In it, I looked at some data from the Pope paper on previous Mr. America winners and had stated that only one or two exceeded the FFMI cutoff. The real number was closer to 6. But it’s of no relevance. My mistyping didn’t change my conclusion: While there are exceptions to the FFMI cuttoff, overall it is a good cutoff in 99% of cases. But it was an error, yes. I admitted it and changed it immediately (because intellectually honest individual admit a mistake and fix it, something more in this field should try). I’d have changed it sooner had I known before a week or so ago.
Regardless, what Eric did was to compare a nine-year old blog post (where the error changed nothing) to a scientist lying about data (in such a way as to change its conclusion) in a published peer-reviewed journal. Said scientist using that lie to support a paper’s conclusion and increase his own visibility (and presumably seminar visits at $5k per appearance).
I’m sorry but does Eric really think a NINE YEAR OLD blog post can or should be held to the same standard as a published scientific paper? Apparently so but only because it allowed him to completely avoid addressing my issue with Brad’s paper. It was a guru deflection and nothing more. He has never since addressed a single criticism I have levied against Brad’s paper. NOT ONE.
More than that it was a blindside. Eric has been a colleague and I guess friend for many years. I edited his books, he contributed a lot to The Women’s Book including an appendix on peak week and making weight. He could have told me about this error any time in the last half decade. Instead, he apparently saved it up for ammunition against me in case he ever needed it. It would be like me leaving one of the myriad errors in his books in when I edited it to use against him if need arose. But I didn’t do that. Because I have intellectual integrity.
Eric has since blocked me on FB and left my group. Again it’s easy to win an argument when someone can’t defend themselves. He has apparently claimed it was due to me ‘impugning his integrity’. Sorry, Eric I can’t impugn something that doesn’t exist. He has acted unprofessionally and, in an effort to defend Brad (with whom he also does seminars with) blindsided a different colleague entirely.
A man with integrity would not deflect a real criticism with a blindside. A man with integrity doesn’t have to crow online about how he has integrity. A man with integrity shows that he has integrity by his actions. By being intellectually honest and not applying a pathetic double standard when it suits him.
Eric has not shown integrity in this matter.
Whatever, I will show him the meaning of true integrity shortly.
James Krieger: And finally James himself. First let me point readers to a FB thread where James is getting kicked around by Lucas Tufur over his recent post. You can watch James mis-reference and mis-represent studies front, back and sideways while Lucas points out his errors and he just moves the goalposts. Maybe that will tell you what’s going on. It’s just desperation at this point, he can’t admit that the study was methodologically unsound, the statistics didn’t support the conclusion or say they were wrong. So it’s pure guru behavior.
Now I will continue to give James credit in that he was the only one with the balls to even attempt a defense. Brad punked out and deflected and Eric did too which is simply pathetic. James at least tried even if he used the same guru tactics, deflections and obfuscations in doing so. He still doesn’t understand and can’t defend Bucher’s analysis of his stats but even then I’d question why someone with an MS in nutrition and ex. phys is doing the stats in the first place. That’s what mathematicians are for. I have a cousin with an MS in applied mathematics and she runs the stats on big medical trials. Why is James doing it?
Well I think it’s simple: he’s good enough at it (mind you computer programs do most of the work at this point), knows how to use Bayesian statistics to obfuscate stuff and, most importantly, shares Brad’s bias about volume. An unbiased statistician wouldn’t play silly buggers like James did. Note again my comment above: in an unregistered trial there is NO evidence that James didn’t run the frequentist methods and then, when it didn’t support the conclusion they wanted, use other stats in a feeble attempt to make them happen. This happens a LOT in science. That’s why you register papers, to eliminate the potential or accusation that happens. It’s why you randomize and blind to reduce the RISK of bias.
After writing my last criticism, James first attempted to defend to some degree the criticisms before writing HIS final last response. You can find it here. Note that the intellectually honest individual shows both sides of the story something I doubt he has done. Now I’m sure he made some good points. But he also made some gross misrepresentations, ones most won’t catch. Some highlights.
Lyle Doesn’t Even Science
James asserts that I simply don’t have enough experience doing science (there it is again, “Lyle doesn’t even science”) to understand the realities of doing it. And yet I know that proper randomization, blinding, trial registration, data reporting etc. are good practices to reduce the risk of bias. Maybe Brad and James should do some remedial work since I seem to know a lot more about good scientific practices than they do. Seriously, if a bunch of randos on the Internet are having to educate ‘professional’ researchers about basic methodology….
Cost and Funding
He also blathers about the cost and funding involved and how many of the methodological issues aren’t realistic financially. I never said science was easy and I know it’s expensive so there’s his strawman. According to Google Scholar, Brad has published about forty seven papers already this year. That’s 5 per month since it’s only September, most researchers do maybe 1 a year because that is how long data gathering and analysis usually takes. Funding is clearly not an issue and perhaps Brad should do one GOOD study per year instead of putting his name on 4 per month. Or maybe that’s why they are hiring a non-mathetmaticist to do the stats: he can’t afford an actual statistician because he’s using his funding on too many poor quality papers.
Let me add: Describing the randomization of a study is free. Registering a trial is free. Blinding might increase the costs for technical reasons but, as above, rather than doing endless sub-par studies, why not put the funding towards fewer QUALITY STUDIES? The same fancy computer that James uses to run his stats can randomize subjects to the different groups at no cost. Certainly getting a second Ultrasound tech might cost money, perhaps Brad is the only one on campus trained in it. He could still be blinded to who he is measuring which, as per Cochrane, reduces the risk of bias from high to low.
Blinding of Ostrowski
He also babbles something about whether or not Ostrowski was blinded or why I didn’t mention it above. This is a pure deflection. Essentially he’s arguing that since other studies might be methodologically unsound, it’s ok for theirs to be. This is like arguing in court that “Yes, this man may have murdered someone. But how do we know YOU haven’t murdered someone.” to deflect attention from the issue at hand The methodology of Ostrowski is not at question here, the methodology and discussion of Brad’s paper is. Regardless, it’s irrelevant.
Whether or not the Ostrowski is blinded or not doesn’t matter because I’m not the one holding it up as providing evidence. I agree that there is a trend as claimed by Brad but I’m not using the data per se as evidence. If James is saying it should be dismissed for not being blinded, then Brad can’t use it in the discussion to support his conclusions. And in using it in his discussion, Brad is saying it’s valid. Which means that how it was represented is all that is at issue here. And it was misrepresented completely, a point that James finally acknowledged himself.
Basically, either the data is valid or it’s not and James can’t have it both ways. And my actions don’t impact on that. Only Brad’s does.
And that is still just a deflection from the fact that, whether the Ostrowski data is good or not, BRAD LIED ABOUT WHAT IT SAID to change it’s conclusions from contradicting to agreeing with him. Of course, James has still failed to address that so far as I can tell at least not directly. He even said it was a misrepresentation. Ok, so why is it still not an issue that has to be addressed?
Put differently, why does Brad Schoenfeld get to lie about data in a published paper and nobody blinks? I make an inconsequential error in a 9 year old BLOG post and I’m at fault.
James also argues that the studies on edema timing aren’t relevant sine it was a new stimulus to the trainees. The Ogasawara study cited in Brad’s paper was in beginners while the Ahtiainen I cited in my last piece was a long-term training study in strength-trained men. So James is not only trying to have it both ways but he’s factually wrong. The study THEY used is in untrained individuals, the study showing edema is in trained individuals. So this is just more of his endless deflection. I refer you back to the link above where Lucas Tufur is kicking James around on this topic and you can see James continue to defend what is indefensible.
Oh yeah, James Krieger has now blocked me on FB as well, right after publishing his article. But it’s always easier to win an argument when the person you’re arguing with or attacking can’t argue back isn’t it? I’d note again that I left all three of them in my FB group to give them the opportunity to address criticisms and all three voluntarily left. I did not and would not have blocked or booted them so that I could win by default. They left by choice. And then blocked me so that I don’t even have the opportunity to respond to them.
On his blog, James has asserted that he blocked me due to me sending them nasty emails and calling them mean names. All true. And? First and foremost, Layne Norton played the same card. Said he didn’t have to address my criticisms since I made his wife cry (I wonder if she cried more when he left her for an Australian bikini chick). Second, this is just disingenuous posturing. I’ve been emailing Brad, Eric and James for a couple of weeks calling them guru shitbags and more (my creativity for insults is quite well developed after so many years).
And it wasn’t until the day he posted his last response that he blocked me (or that I noticed, he had been in my group for that previous week). He waited to do it to ensure I couldn’t respond to him. So just another, well let’s call it what it is: a lie. He’s lied in his articles about studies and data, and he’s lying now.
Finally, what is with this industry and otherwise big muscular dudes being such insecure children? Do words on a screen really hurt them that much? Or is this just more pathetic guru behavior to avoid my criticisms? Hint: it’s the latter. Seriously, these guys need to eat less tilapia: their skin is too thin. But I digress.
Oh yeah, if you look at Jame’s article he put up a picture of one of his clients who does the high volume bicep work as supposed proof of concept. But the last time I looked, anecdote (i.e. one individual) doesn’t count as science. It never has an never will.
This is like the cancer quack holding up a SINGLE survivor from their program and ignoring everybody who died. This is like Layne Norton, who pretends to be the anti-guru, saying he’s got 100’s of emails so science doesn’t matter. So James, either pretend to be a scientists or don’t. Don’t play silly buggers where it’s science until it’s not. Even if it’s bad science in this case which it is.
If I were a different person, I’d do the same and put up a picture of someone who got big using moderate volumes. But that’s not an argument that has any validity so I won’t. I’ll leave such nonsense to gurus and pay attention to the scientific facts. If you want to science, stick with the science. If you want to use anecdote, that’s fine. But don’t pretend it’s science. James wants it both ways, just like he does in most of his discussion above.
The Guru Crew
So add James Krieger to the guru group of Brad Schoenfeld and Eric Helms. Their actions are no different than endless others before them: Tim Noakes, Dr. Fung, Gary Taubes, Layne Norton, individuals that most reading this piece see as stupid gurus while they give Brad, James and the rest a pass for identical behaviors.
Individuals who just block and ignore criticism rather than address it, usually after endless deflections and obfuscations. Brad ducked every criticism, Eric deflected it and blindsided me and James used a mix of deflection and obfuscation. Standard guru operating manual.
If that doesn’t tell you everything you need to know about the conclusions of this study, no amount of in-depth analysis by me or Lucas Tufur or Brian Bucher will help. They have played nothing but guru games from the get go. They shouldn’t get a pass when others do not. And yet here we are.
Others at Play
I am told that others have joined the circle jerk. Greg Knuckols of course but he writes for MASS with Eric Helms and of course he has to agree. Also, that pesky seminar circuit he has to keep himself on to make the big bucks. And all are, of course, saying my criticisms have no weight or that I have no weight.
Nevermind that Brad routinely shared my previous research reviews, Eric thought I was good enough to contribute to my book, James and I have done a webinar together. All I have done is ask them to address specific criticisms which, for the most part they have not. So like many before them, they resort to simple ad hominem attacks against me. Now I don’t even science, now I don’t have any weight. They didn’t mind when they were making money off of me. But that’s par for this course isn’t it?
This is every guru in the history of ever. Food babe, the Snake diet guy, more than I can think to name. The true guru ignores criticism, deflect, obfuscate, block critics and then attack them in a forum they can’t defend themselves. Don’t tell yourself it’s anything else. Don’t let the fact that you like them and think I’m a foul-mouthed asshole color it. Simply ask yourself why this group of individuals is being allowed to engage in behavior that, if it were anybody else, would be criticized and destroyed? It’s that simple.
Ask yourself why they get a pass.
A Challenge to the Whole Crew
And now my challenge. All of the individuals involved claim to blend science and practice in terms of their training recommendations. I have done the same for nearly 25 years (and note that most of these people came up reading MY books to begin with). I’ve been in the weight room since I was 15, I’ve been involved in the science of performance since college. It’s 35 years of training and nearly 30 years of being a training and nutrition nerd. I know that science is good but limited, so is anecdotal evidence. So you have to see how they fit together. Sometimes science supports what the bros knew (protein is good), sometimes it contradicts it (meal frequency doesn’t matter). It’s useful to compare.
But if the crew is going to say that they blend science and practice, well that has an implications which is that new data, such as Brad’s volume data, should be incorporated into their training advice or how they train themselves. But will it be? Somehow I doubt it.
I doubt that any are going to move their trainees to the workout that Brad’s data would suggest as optimal. And that tells you all of what you want to know. If they believe in this study’s results, they should all adapt their own personal training and that of their trainees to it. When/if they do not, then they either don’t believe it or are being disingenuous in saying they blend science and practice. It’s that simple.
They can’t have it both ways.
Ignore the study, ignore the fundamentally flawed methodology, ignore Brad’s lie about the Ostrowski data, ignore the back and forth between the nerds and everything else and just ask yourself this question:
If Brad et. al. think this data is valid, why aren’t they implementing it in their trainees? Yes, I know, Brad has prattled on about using high volume overreaching cycles with folks. Huge volumes for a couple of weeks. Fine, I’ve no issue with that. But this study was 8 weeks straight of volumes no human, juiced or otherwise, has done with good result. If they’ve done it at all.
Now, if I saw data that I found applicable, I would implement it into the advice I give. If I found data I didn’t agree with, I wouldn’t. I don’t think this study is worth a damn and won’t change my approach to training or my recommendations based on it. I’m finishing up an analysis of the 7 extant volume papers to present in a week or two when it’s done. And the gross data supports not only what I’ve always recommended but will continue to recommend.
But I bet that neither will they. Yet they are defending a piece of data that I can almost assure readers they will not apply. Why? Because they know it’s bullshit. They know it’s not right. Because if it were, they’d have moved their trainees to that style of training 2 weeks ago. I even asked them via email when they would be changing their own or their client’s training.
They know this finding doesn’t mean shit. It also goes against 5 of the 7 total papers on the topic. One is garbage 5 say moderate volumes beats out high and Brad’s paper is the outlier that’s left. In science, you build models of the total of the data, not the single study. Again note that James stated this explicitly, how you can’t draw a conclusion from a single paper, implying that I was doing that. Except that I wasn’t, Brad et. al. are the ONLY ones drawing strong conclusions and it was yet another deflection by James, attempting to project THEIR behaviors onto me. I love it when other people make my argument for me. I will be examining all 7 current studies on the topic in a week or two and you’ll see what falls out of the model. And it’s not Brad’s conclusions.
But whatever, I’m back into nerd mode.
Do the Workout
So back to the challenge, either to them or anybody who thinks they are more right than I am (which is fine, I never said everybody had to agree with me, I just asked that they address my criticisms honestly which has not been done).
Do the workout. If you think the results are valid then DO THE WORKOUT.
And because I am a helper, I have drawn one one up based on THEIR findings. Recall that it suggested 30 sets per muscle group per week for upper body and 45 for lower body as providing optimal growth. This was in individuals with a minimum of 1 year of training and their strength levels were advanced noob at best. So if you’re not a rank beginner, and believe their data, it applies to you. One year or more of regular training and at least a 1.2bodyweight squat and 1.1 bodyweight bench and you can DO THE WORKOUT.
But also keep in mind that it only used compound movements (ok, leg extension for quads) and looked at quads, biceps and triceps. We have no data on pecs or back or delts or glutes or hamstrings. So those have to exist independently of the other muscle groups, especially for lower body. And the below is what that implies in terms of a practical workout scheme based around the performance of 30 sets/week for each upper body muscle (all sets to failure) and 45 sets/week for each lower body muscle.
I’ve provided two options below. The first is a non-split routine training either upper or lower body on each of 3 days/week to get the total volume in. All sets are on 90 seconds rest and taken to concentric failure. If you get more than 15 reps on any given set, add 3-5%.
Option 1: Non split routine
Mon/Wed/Fri: Lower body
Leg press: 5X8-12RM
Leg extension: 5X8-12RM
Lying leg curl: 5X8-12RM
Seated leg curl: 5X8-12RM
Standing calf raise: 5X8-12RM
Leg press calf raies: 5X8-12RM
Seated calf raise: 5X8-12RM
So that’s a 45 set workout for just legs. If I wanted to get pedantic, I’d suggest another 15 sets for glutes to make Bret Contreras happy*. I leave that to the individual trainee but that would take it to a 60 set workout three times weekly. Have fun.
Tue/Thu/Fri: Upper Body
Flat bench: 5X8-12RM
Incline bench: 5X8-12RM
Cable row: 5X8-12RM
Undergrip pulldown: 5X8-12RM
Shoulder press: 5X8-12RM
Lateral raise: 5X8-12RM
Rear delt on pec deck: 5X8-12RM
Face pull: 5X8-12RM
Barbell curl: 5X8-12RM
Preacher or incline DB curl: 5X8-12RM
Close grip bench: 5X8-12RM
Triceps pushdown: 5X8-12RM
Now we might quibble over the above, Brad’s study did 30 sets/week of compound pushing and pulling. But they measured biceps and triceps. Should we take out the direct arm work or is it 30 sets of compound pushing and pulling and 30 more sets of direct arm work. I leave it to you to decide and I’ll address the odd way of counting sets (which is fundamentally wrong in my opinion) in a future article.
This is a 6 day/week split routine with each muscle hit once/week. That means that all 30 sets for upper or 45 for lower muscle groups have to be done on that single day. Note that even Arnold and his ilk did 20 sets per muscle once a week and not all sets were remotely close to failure. With drugs. Brad is suggesting 1.5 times that for upper body and a little over double for legs. Enjoy.
Leg Press: 15X8-12RM
Leg extension: 15X8-12RM
You could technically pick more movements but this is what they used in their paper so I’m using it too. If you want to do 5 sets of 9 different movements that target quads, go to town. But it’s 45 sets of 8-12 to positive failure with 90 seconds rest for quads no matter what.
Flat bench press: 5X8-12RM
Flat DB press: 5X8-12RM
Cable crossover/pec deck: 5X8-12RM
Incline bench: 5X8-12RM
Incline DB press: 5X8-12RM
Incline flye: 5X8-12RM
Narrow grip Cable row: 5X8-12RM
DB row: 5X8-12RM
Undergrip lat pulldown: 5X8-12RM
Medium grip lat pulldown: 5X8-12RM
Cable pullover: 5X8-12RM
Standing calf raise: 15X8-12RM
Leg press calf raise: 15X8-12RM
Seated calf raise: 15X8-12RM
DB overhead press: 5X8-12RM
Barbell overhead press: 5X8-12RM
Hammer overhead press: 5X8-12RM
DB Lateral raise: 5X8-12RM
Cable lateral raise: 5X8-12RM
Machine lateral raise: 5X8-12RM
Pec delt rear delt: 10X8-12RM
DB bent over rear delt: 10X8-12RM
Face pull: 10X8-12RM
I can’t think of more rear delt movements.
Lying leg curl: 15X8-12RM
Seated leg curl: 15X8-12RM
Barbell curl: 5X8-12RM
DB curl: 5X8-12RM
Preacher curl: 5X8-12RM
1-arm preacher curl: 5X8-12RM
Incline DB curl: 5X8-12RM
Cable curl: 5X8-12RM
Close grip bench: 5X8-12RM
Triceps pushdown: 5X8-12RM
1-arm triceps pushdown: 5X8-12RM
Barbell nosebreaker: 5X8-12RM
French press: 5X8-12RM
Cable French press: 5X8-12RM
Again we might quibble over the set count on arms. Does it count separately or does the compound pushing and pulling get it done? Another topic for another day.
But there ya’ go. The applied Schoenfeld et. al workout routine.
Believe their data? Think I’m full of shit? Then do the workout. When you get overuse injuries and overtrain and get tendinitis, you can buy my injury nutrition recovery book. Regardless, if you think they are in the right it’s simple:
DO THE FUCKING WORKOUT.
But make up your own decision: If you believe them, do the workout above and report back on your results.
I love to be proven wrong and will always say I was wrong when that happens.
I also love to tell people “I told you so” and have them tell me “You were right.”
Time will tell which will occur.
See you in 8 weeks.
* I mentioned Bret Contreras above and wanted to add this comment. Bret is listed as like the third author on the paper, right. But what did he actually do on the paper? Mind you, this is another issue: in most papers, the contributions of each individual author is commonly listed. Is Bret even in the same location as Brad? What did he contribute to the study? As importantly, why has he remained completely quiet on the issue (so far as I can tell)? He’s not defending it nor promoting it and one has to wonder why.