Education

PME 802 – Program Theory-Driven Evaluation Science

In Module 2, we have been tasked with constructing a program theory- theory of action and theory of change-and rationalizing it. It has been a bit of daunting task as there seem to be so many theories and templates, but I remain passionate about my evaluation project. To clarify previous posts, my program evaluation is of a placement/internship program that is already in existence. It is part of a college-level program, that hosts students in class for four months and then the students are required to complete a four-month unpaid placement.

Following the program theory template of Donaldson, S. I., and Lipsey, M. W. (2006). This is what I came up with – EH Program Theory.

To my understanding, a program theory should outline the basics of the evaluation; the “if and when” of the program implementation. I followed the template of Donaldson and Lipsey because I found that my program evaluation fell into the category of a program theory-driven evaluation science. While it is highly constructed from a program theory model, there is also an element of social science theory that could be drawn upon. A lot of research exists on internships and the impact they can have. I would explore whether there are reliable resources to reference on optimal program length, and then use direct program experience and resources to explore whether the current students are getting jobs.

I believe that my program theory follows the plausibility and sensibility process outlined by Donaldson and Lipsey:

  1. Organizational plan – How?
  2. Service utilization plan – Students receiving the intentions.
  3. Impact theory – Is it bringing about intended benefits.

In terms of a logic model for the program, I used the template provided by The W. K. Kellogg Foundation, and this is what I came up with: EH Logic Model.

I liked this template because it is simplistic and is easy to outline the resources, activities, outputs, short – and long-term outcomes, and impact. While the general idea of my evaluation is to determine if the program is following through on its claims of students getting jobs from placement, I also added in some evaluation elements of whether the students are obtaining jobs in their field because of the experience of placement. I believe this would give administrators of the program a better idea on how to frame marketing materials to also not give students false expectations.

Based on these models, our class notes, readings, and recommendations from Dr. Lam, I have revisited my evaluation questions posted on September 22nd:

  1. What percentage of students have obtained jobs directly from the placement organization?
  2. To what extent is students’ success in obtaining a job attributed to the placement program?
  3. Is there a difference between students that have obtained a job from placement versus students who have not?
  4. In what ways are activities working for the program to run efficiently?
  5. How are students satisfied or dissatisfied with program execution?
  6. To what extent are students meeting the objectives of the internship program?
  7. What are the retention rates of the program?
  8. Has the program seen an increase in enrollment and applications?

I feel like I still have so many questions to ask and to be answered, but I thought I’d pause for now and turn to my classmates/peers/experts for feedback.

Thank you for reading!

 

Leave a Reply

Your email address will not be published. Required fields are marked *