Program:

• The RTW program, operating from 2015-2019, awarded grants to local agencies to provide un- or under-employed workers with customized services including staff guidance on career planning, occupational & work-based training, employment readiness courses, & job search assistance.

Study Design:

• DOL chose 4 local RTW programs for the RCTs: (1) Maryland Tech Connection (MTC); (2) JVS programs (Skills to Work in Technology & Job Search Accelerator) in San Francisco; (3) Finger Lakes Hired (FLH) in Monroe County NY; and (4) Reboot Northwest (Reboot NW) in Portland OR & Vancouver WA. Each program had flexibility to develop/customize services to meet local needs.

• The RCTs had samples of 1,029 workers (MTC), 993 (JVS programs), 610 (FLH), & 980 (Reboot NW), and were well-conducted based on careful review (e.g., baseline equivalence, negligible attrition).

Findings:

• Unfortunately, none of the RCTs found discernible positive earnings impacts over follow-up periods of 3-4 years, & one program - MTC - had a statistically significant adverse impact over the pre-specified primary follow-up period (quarters 5-10). The graphs above tell the story.

Comment:

• I believe the disappointing results reflect a flaw in how federal agencies often select programs for large RCTs: They underestimate the challenge of finding programs that are truly effective (many plausible-sounding ideas just don't work in practice).

• If the goal is - as I think it should be - to build the body of proven-effective programs which, if scaled-up, could help tackle major social problems, the feds would do better to focus large RCTs on programs that don't just *sound* like good ideas (as with RTW) but are backed by highly promising prior evidence - e.g., from smaller RCTs or quasi-experiments. Doing so could achieve a much higher success rate, as discussed here.