WHY A CONTROL GROUP IS NEEDED TO DETERMINE WHETHER A SOCIAL PROGRAM IS EFFECTIVE, in 4 simple charts. The example I’ll use is HHS’s Comprehensive Child Development Program (CCDP), which provided intensive case management services to low-income families with young children (continued below).

  • As shown in chart 1 (upper left), mothers in CCDP saw their employment rate more than double over the 5y after program entry. But CCDP was evaluated in a rigorous study that had a randomly-assigned control group of families, & mothers in the control group saw an almost identical gain (chart 2, upper right).

  • So CCDP had no effect on mothers' employment, versus an equivalent group of mothers who didn't participate. If CCDP had been evaluated in the usual non-rigorous way (examining employment outcomes without reference to a control group), it would've been deemed highly effective.

  • Same story with child outcomes. Chart 3 (lower left) shows that, among children in CCDP, the percent scoring "at risk" in cognitive development & behavior fell sharply from ages 2&3 to age 5, during their time in the program.

  • But control-group children saw almost identical gains (chart 4, lower right). Conclusion: Simply looking at before & after improvement in participant outcomes, without respect to an equivalent group of nonparticipants (controls), often yields the wrong conclusion about program effectiveness.

  • Here's a link to the HHS (@OPRE_ACF @abtassociates) study report.

We use cookies to improve your experience and to help us understand how you use our site. Please refer to our cookie notice and privacy statement for more information regarding cookies and other third-party tracking that may be enabled.

Intuit Mailchimp logo