@JPAL published an RCT of three STEM summer programs for underrepresented high schoolers. Quick take: The main findings — including large impacts on college STEM degree receipt — are quasi-experimental, not RCT; the RCT impacts are mostly small & not statistically significant.

  • The headline finding that 1 of the programs (the 6-week version) increased college STEM degree receipt by 33% is a quasi-experimental (non-RCT) comparison of higher-ranked vs lower-ranked students, with statistical adjustments to try to equate the two groups.

  • The study abstract should make this clear (not just say it's "RCT"), as such quasi-experimental methods are not as credible as high-quality RCT designs.

  • The paper does report RCT results for 1 of the programs (the online version). The effects found on STEM degree receipt, college graduation, & other outcomes at the 4 year mark are small and not statistically significant.

  • Here’s the study abstract and report.

We use cookies to improve your experience and to help us understand how you use our site. Please refer to our cookie notice and privacy statement for more information regarding cookies and other third-party tracking that may be enabled.

Intuit Mailchimp logo