Let’s Review E-Eval

After the partnership between AMP and the Evaluation Capacity Network (ECN) was made official in the summer of 2020, no time was wasted in planning and developing an evaluation capacity-building training specific to the evaluation needs of mentoring agencies across Alberta. In recognizing the need for a collaborative and flexible approach to learning, the ECN created E-Eval training. Starting in the fall of 2020, E-Eval was an online training that used didactic sessions and experiential activities to support six mentoring agencies from across Alberta as they learned about program evaluation, and how to use it in their organizations.

Spanning six months, E-Eval was separated into two parts to foster a gradual yet comprehensive increase in the evaluative understandings and skills of participating agency staff. The first half was focused on educating the agencies on the basics of evaluation, such as partner maps, logic models, and how to come up with purpose statements and evaluation questions. In the second half of E-Eval, the agencies focused on developing their data collection, analysis, and knowledge mobilization skills while developing and implementing their own evaluation plans. To provide further support, participating agencies had the opportunity for individual coaching sessions beyond the synchronous portion to further support the capacity-building processes.

Adapting E-Eval

In 2022, AMP recognized the need for consistent and an increased quality of mentor programming for youth and young adults transitioning from care, and also understanding whether a new mentor program would be impactful. Because a disproportionate amount of youth and young adults in Alberta who access mentoring services identify as Indigenous, AMP also had the goal of taking an Indigenizing and decolonizing approach to program development and evaluation. AMP approached the ECN for their support because of their ongoing partnership success and the flexible and collaborative approach E-Eval offered.

What Made E-Eval 2.0 Different

The second iteration of E-Eval held several similarities and differences to the first E-Eval training. With the same 6-month online structure, the second iteration of E-Eval supported learners from three Alberta mentoring organizations who were preparing to launch a new mentoring program for youth and young adults transitioning from provincial care systems. The ECN and AMP integrated Indigenous and decolonizing approaches to the structure and content of E-Eval and evaluation framework. This included collaboration with an Indigenous consultant as well as Indigenous evaluation experts from the community. Adding further changes to the learning environment, as funders, several AMP staff members were involved and participated as learners as well!

Similar to the first E-Eval training, the three agencies were initially going to make their own evaluation framework for this new program. However, through group discussions, AMP and the agencies realized it would better fit their educational and practical needs to have a single evaluation framework for the new program, which could then be adapted to fit the context of each agency. Beyond this joint evaluation framework, the unique learning environment of funders and learners working together led to several insightful outcomes. The agency staff experienced an increase in their individual and organizational evaluation capacities, AMP (the funder) became an advocate for the mentoring agencies, especially in negotiating funding requirements and in learning about agency needs and capacities.

What is next for AMP’s Evaluation Journey?

AMP has taken on a leadership role regarding evaluation, taking its learnings and sharing its knowledge with its partners through evaluation resources and community events. AMP also continues to conduct summative evaluations at the end of each funding period (every three years)! Evaluation will always be a fundamental part of AMP’s story, both in terms of evaluating itself and also, bringing evaluation to its partnership. Check out next month’s AMP Evaluation Journey blog to learn more about AMP’s next evaluation events and news!

Check out these links for more information about E-Eval and the AMPxECN Partnership!

• Evalbyte: Co-Creating Evaluation Frameworks with Nonprofits and Funders
  (Evaluation blog post from the Evaluation Capacity Network)
• Evalbyte: Building Evaluation Capacity with Diverse Needs in Alebrta’s Mentoring Sector
  (Evaluation blog post from the Evaluation Capacity Network)
• Evalbyte: Weaving Indigenizing and Decolonizing Approaches Into Community-Based Evaluations
  (Evaluation blog post from the Evaluation Capacity Network)
• Evaluation Capacity Building: Experiential Learning Through Community–University Collaboratives
  (Published manuscript authored by several ECN researchers and evaluators)
• E-Eval: Learnings from a Community-Based Evaluation Capacity Building Initiative
  (Published manuscript authored by several ECN researchers and evaluators)