Evaluation has played a crucial role in supporting AMP’s growth and efforts in uplifting the mentoring profile in Alberta since its inception. In 2014, after completing their second organization-wide evaluation, AMP recognized the benefits of evaluation and saw the potential insights evaluation could provide to their partnership. As a result, AMP staff started to encourage and invite their partners to also learn about and engage in evaluation.

Regarding mentoring agencies specifically, AMP shares evaluation resources that can support the improvement of Alberta’s mentoring programs while considering the changing mentoring needs and contexts across time. This was of particular importance during the COVID-19 pandemic, where AMP and mentoring agencies were faced with the question of how to ensure high-quality mentoring in a time of physical distancing.

Illuminating Gaps in AMP’s Evaluation Capacities

The COVID-19 pandemic brought several barriers to the mentoring landscape in Alberta. Not only did the needs of mentors and mentees change, but how mentoring was delivered and what mentoring could look like became uncertain. In response to these changes and policies regarding physical distancing, mentoring agencies shifted to online platforms for service delivery. However, as a “new” form of mentoring, both AMP and many of its partnering agencies questioned the effectiveness and impact of virtual mentoring programming. AMP recognized the need for more specialized knowledge, skills, and resources for program evaluation - capacities that both AMP and mentoring agencies could benefit from increasing. To increase their program evaluation capacities, AMP turned to the community and found the Evaluation Capacity Network (ECN).

A Community-Based Partnership for Evaluation Capacity-Building

Housed at the Community-University Partnership at the University of Alberta, the ECN is an interdisciplinary network of community partners, practitioners, researchers and funders who are devoted to supporting the social serving community through community-driven, culturally-responsive, action-oriented and equity-based evaluation capacity building initiatives and resources.

Of particular interest to AMP was the ECN’s experiential postsecondary course called U-Eval, a week-long intensive course where graduate students partner with community-based initiatives and programs to co-create a community-based evaluation framework. One of AMP’s mentoring agencies, BGCBigs, participated as community partners in this course in 2020, and from this an evaluation plan was created for virtual mentoring. After hearing of this experience, AMP wanted to bring this expertise and learning opportunity to other mentoring organizations across Alberta. To do so, AMP approached the ECN with the proposition of a partnership. AMP hoped for other and more targeted learning opportunities for themselves and their partner mentoring agencies to increase their program evaluation capacities.

The partnership between ECN and AMP officially started in the summer of 2020 with several online evaluation capacity-building (ECB) workshops hosted by the ECN, where basic evaluation knowledge was introduced to AMP partners. This led to the ECN designing a 6-month intensive and experiential ECB training called E-Eval. Separated into two parts, E-Eval was a collaborative and flexible online training where AMP’s partnering organizations first learned the basics of evaluation and how to develop their own evaluation plan. In the second half of E-Eval, the mentoring organizations were supported in developing their skills in data collection, analysis, and how to use and share evaluation findings while simultaneously engaging in their evaluation. While didactic sessions were synchronous and online, all participating agencies had the opportunity for individual coaching sessions beyond the synchronous portion to further support the capacity-building processes.

What did AMP and its Partners Learn?

From their participation in these three evaluation initiatives, AMP and its partners truly recognized just how much time and resources are required to engage in program evaluation. However, they also learned that external influences can significantly impact an organization’s or an individual’s evaluation capacities. In fact, AMP and some partners experienced unexpected challenges in meaningfully engaging in program evaluations due to structural and staff changes and pressures from funders. During these times, the ECN supported AMP in their evaluation needs, responding with reflexivity and inclusivity. Check out next month's edition of AMP’s Evaluation Journey blog to find out!