Pediatrics published an RCT of a program encouraging school children to drink more water (e.g., in lieu of sweetened drinks), to prevent overweight. Quick take: The study's claims of effectiveness notwithstanding, it found no impact on the prespecified primary outcomes: Body Mass Index (BMI) z-score & BMI percentile. (Continued below.)

Program & Study Design:

  • The study randomized 18 low-income elementary schools in CA, with a sample of 1,544 4th graders, to treatment (the Water First program) vs control. Water First promotes water drinking (e.g., in lieu of sweetened juices/milk) and increases drinking water access in schools.

Findings:

  • The study says it found an improvement in its primary outcome - overweight prevalence (i.e., % of kids overweight) - at the 15-month mark. Per the abstract: “The primary outcome was overweight prevalence .… From baseline to 15-months, increases in overweight prevalence were significantly greater in control schools (3.7%) compared to intervention schools (0.5%).”

  • But overweight prevalence wasn't the outcome the study had *prespecified* as primary. The 2/2021 study protocol & the 10/2021 clinicaltrials.gov registration prespecified BMI z-score & percentile as primary.

  • The study found no impact on BMI z-score or percentile (primary outcomes), as shown in the impact table above. Also, from just eyeballing the table you can see all weight outcomes are pretty darn similar for Treatment vs Control at the 15-month mark.

Comment:

  • Note: In January 2022, the study changed its clinicaltrials.gov registration to make overweight prevalence the primary outcome, but this was nearly 2y after the study's last data collection, so isn't a *pre*specification. This is like betting on a horse after watching the race.

  • Such misreporting of primary impact findings - at least for the social program RCTs I focus on - is unfortunately all too common, even in top, peer-reviewed journals like Pediatrics. The findings also received uncritical press coverage.

We use cookies to improve your experience and to help us understand how you use our site. Please refer to our cookie notice and privacy statement for more information regarding cookies and other third-party tracking that may be enabled.

Intuit Mailchimp logo