American Economic Review published RCT findings for Saga Education's high-dosage math tutoring of low-income 9th & 10th graders in Chicago. Quick take: High-quality RCTs find sizable effect on math achievement that persists 1-2 years after tutoring ends.

Program:

  • Saga provided math tutoring to 9th & 10th graders in low-income high schools - 50 min/day, 5 days/week for 1 schoolyear.  Tutors were recent college grads, trained & paid a modest stipend. Tutors worked with 2 students at a time. Cost was approximately $4800/tutored student.

Study Design:

  • 2 RCTs evaluated Saga at scale in 15 high-poverty high schools, with a pooled sample of 5,343 students randomly assigned to treatment vs control (>95% Black or Hispanic, 87% low-income).

  • Only about 40% of treatment group (T) students received Saga tutoring due to scheduling & related issues, so the studies estimated impacts for both the full T group (“intent to treat”) & those T students who received at least 1 tutoring session (“treatment on treated”).

Pooled Findings of the 2 RCTs:

  • The RCTs found sizable effects on math achievement at end of the tutoring year, but what's remarkable is that effects persisted. At end of 11th grade — 1-2 yrs after tutoring ended — the effect on district test scores was 0.23 for tutored students (treatment on treated) & 0.10 for the full T group including untutored students (intent to treat).

  • The gain of 0.23 for tutored students represents a 2.6-fold improvement over the annual gain in math otherwise expected for 11th graders. Saga also increased tutored students' 11th grade math GPA by a quarter point. These effects were statistically significant, p<0.01.

  • The RCTs found no discernible effects on behavior, absenteeism or high school graduation rates. The first RCT also tested a cognitive-behavioral program for males (Becoming a Man) in 2 treatment arms, & found no discernible effect on academic outcomes, suspensions, or arrests.

Comment:

  • Based on careful review, this was a well-conducted RCT (e.g., excellent baseline balance, moderate sample attrition but not differential, valid analyses). Overall, I think the findings are very encouraging.

We use cookies to improve your experience and to help us understand how you use our site. Please refer to our cookie notice and privacy statement for more information regarding cookies and other third-party tracking that may be enabled.

Intuit Mailchimp logo