Capacity building initiative learns to enhance design and sharpen impact via evaluation

The Carl & Roberta Deutsch Foundation launched its HALO Award in 2010, bringing funding, strategic guidance, and technical support together, to help build nonprofit capacity through better volunteer management. To ensure continuous learning and improvement,  they integrated evaluation into the initiative design…

As a learning organization, the Foundation wanted evaluation to:

Since Cohort 2, Blue Garnet has served dual roles, in partnership with the Foundation, as technical assistance provider and learning partner. In support of the Foundation’s evaluation objectives, Blue Garnet worked with Foundation staff to design and implement an evaluation approach that included a few key components:

• Real time assessment on participant experience and curriculum feedback: Quick polls at end of each group session during the HALO year
• Longitudinal evaluation to assess HALO’s support and impact: combination of quantitative and qualitative data collection across 3 points in time (baseline, immediate post-HALO, and one-year-post)
• Learning conversations with Foundation staff and Trustees to discuss findings, recommendations, and implications for refining the initiative and/or future evaluation

After 9 years of evaluation across 10 cohorts, the HALO Award has demonstrated impact on volunteer program capacity. HALO Award participants report an average 16% increase in program capacity during the HALO year, plus an average 2% increase in the year immediately following HALO. Preliminary findings also point to impact beyond, showing elements of stronger overall organizational capacity one year out.

Assessment learnings also drove refinements to the HALO Award design, from making Technical Assistance (individual organizational coaching and peer cohort learning) a requirement and integral part of the Award experience, to strengthening curricula, tools, and templates to support volunteer management, e.g., volunteer management plan, Spotlight Cases, and alumni panels.

For every cohort, data and insights have been generated and used to inform program design and implementation. As findings are solidified, the focus of learning has shifted from process/experience toward generating impact. With this, evaluation design evolved to reflect latest thinking and streamlined to minimize burden on awardees.

In 2021, the HALO Award’s 10^th Cohort began their capacity-building work. In anticipation of this milestone, the Foundation seized the opportunity to inspire greater investments in volunteer programs—even beyond HALO—through impact storytelling. With Blue Garnet’s support, awardees were invited to “share their HALO Story” in the multimedia HALO Award Stories project. Volunteers, staff, and leaders from across LA County responded to the call, painting a colorful and varied picture of impact over the past decade.

For Way-Ting Chen, Blue Garnet lead on this work since the beginning,  the HALO Award is a stellar example of how strategy and evaluation are two sides of the same coin. “It’s tremendously exciting to work with a client who is a true learning organization. The Foundation understands the value of evaluation as a tool for continuous learning, for refining strategy and design, and for ultimately being accountability to its community. And, after all these years, it’s still a lot of fun!”

For more reflections, watch Way-Ting’s testimonial for HALO Award Stories project.