How to check the study behind a clinically studied claim

You don't need a science degree to check a supplement claim. You need about 15 minutes, a free database and a handful of questions. The goal isn't to become a researcher. It's to find out whether the research a company points to actually describes the product you're about to buy.

Here's how to do it, one step at a time.

Step 1: Find the actual study

Look for a reference on the product page, the company's science page or the packaging. If there isn't one, email the company and ask for the full citation.

Don't be surprised if the trail goes cold. When researchers analyzed advertising for 104 sports products, they found 431 performance claims. More than half (52.8%) of the websites making claims gave no references at all. Of the studies the researchers could appraise, 84% were at high risk of bias, and only three of 74 were judged high quality with a low risk of bias (Heneghan et al., 2012). Sports products aren't supplements for digestion, but the lesson carries over: a claim and its evidence are two separate things.

Step 2: Look it up yourself

Go to PubMed, the free database of the US National Library of Medicine, and search for the study title or the ingredient plus the condition. Open the abstract, the short summary at the top. Note the PMID number so you can find it again.

Read the abstract, not the company's summary of it. Look for the results section, which usually says what changed, by how much and compared with what.

Step 3: Check that the study matches the product

This is where most "clinically studied" claims fall apart. Compare the study and the label on six points:

• Ingredient or strain. A systematic review of 228 probiotic trials found strong evidence that effects are specific to both the strain and the condition. Some Lactobacillus strains helped prevent antibiotic-associated diarrhea in adults, while other Lactobacillus strains didn't (McFarland et al., 2018). Same species name, different result.
• Dose. Is the product's amount per serving the same as the study's?
• Form. Capsule, powder, food or extract can matter.
• Duration. A 12-week result doesn't promise anything at week 2.
• Population. Healthy young men, people with IBS and women in midlife aren't interchangeable.
• Outcome. A change in gut bacteria isn't the same as less bloating.

Dose deserves a closer look, because more isn't automatically better. In a 12-week trial in 44 people with IBS, a galactooligosaccharide prebiotic at 3.5 g a day improved stool consistency, flatulence, bloating and a composite symptom score. At 7 g a day, it improved overall assessment and anxiety scores, but the abstract doesn't report the same improvements in flatulence and bloating (Silk et al., 2009). A product using a different amount can't simply borrow those results.

Step 4: Check how the study was done

• Was it in humans? Lab dishes and mice are starting points, not proof.
• Was there a control group, ideally a placebo?
• Was it randomized and blinded, so neither participants nor researchers knew who got what?
• How many people took part? A study of 12 is a hint. A study of several hundred is more convincing.

Step 5: See who paid for it

Funding is usually listed near the end of the paper. Industry funding doesn't make a study wrong, and many good trials are paid for by the companies that make an ingredient. But it's worth knowing. In an analysis of 206 nutrition articles about soft drinks, juice and milk, none of the interventional studies funded entirely by industry reached an unfavorable conclusion, compared with 37% of those with no industry funding (Lesser et al., 2007).

Step 6: Zoom out

One trial is a beginning. Search for a systematic review or meta-analysis on the same ingredient and symptom, and see whether independent groups found similar results. If the only evidence is a single small study run by the seller, the honest label would say "promising," not "proven."

Your five-question scorecard

1. Can I find the study?
2. Same ingredient or strain, dose and form as the product?
3. Done in people like me, measuring the symptom I care about?
4. Randomized, placebo-controlled and reasonably large?
5. Replicated by anyone independent?

And one step before all of this: if your digestive symptoms are new, getting worse or come with blood in your stool, weight loss or night waking, get a diagnosis first. Research on a supplement can't tell you what's causing your symptoms.

Sources

1. Heneghan C, Howick J, O'Neill B, et al. The evidence underpinning sports performance products: a systematic assessment. BMJ Open. 2012;2(4):e001702. PMID 22815461
2. McFarland LV, Evans CT, Goldstein EJC. Strain-specificity and disease-specificity of probiotic efficacy: a systematic review and meta-analysis. Front Med (Lausanne). 2018;5:124. PMID 29868585
3. Silk DB, Davis A, Vulevic J, Tzortzis G, Gibson GR. Clinical trial: the effects of a trans-galactooligosaccharide prebiotic on faecal microbiota and symptoms in irritable bowel syndrome. Aliment Pharmacol Ther. 2009;29(5):508-18. PMID 19053980
4. Lesser LI, Ebbeling CB, Goozner M, Wypij D, Ludwig DS. Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Med. 2007;4(1):e5. PMID 17214504

This article is for general information and isn't medical advice. Talk to your healthcare provider about whether a supplement is appropriate for your symptoms and health history.