Abstract

In traditional approaches to evaluation, emphasis is often placed on determining the ultimate outcomes or end products of a program. Consequently, little regard is given to those mediating outcomes that define the actual operations or fidelity of the program. This purpose of the article is to describe how a logic model can be used to evaluate the fidelity of youth development programming.  In describing the model several things are included. First, basic assumptions underlying program evaluation are described. These assist in understanding how and why programs are assessed.  Along with this, basic evaluation designs that have typically been used in evaluation efforts are presented. Next, a logic model which embraces the idea of looking at both process as well as product aspects of a program is presented.  A theory of change approach is used here (Connell  & Kibrich, 1998; Izzo, et al., 2004) where basic principles and strategies to the evaluation process are applied.  Finally, an example of how this approach is applied to a youth development sport program is provided. Immediate and intermediate outcomes are illustrated in the example and show how they create a pathway for impacting ultimate program outcomes. The expected end product from this portrayal is to offer a broader lens for effectively evaluating the efficacy of youth development programs.

Resumen

En los enfoques tradicionales de evaluación, a menudo énfasis en determinar el último resultados o productos finales de un programa. Por lo tanto, poca atención se da a los resultados que definen las operaciones reales o fidelidad del programa de mediación. Este propósito del artículo es describir cómo puede utilizarse un modelo lógico para evaluar la fidelidad de la programación de desarrollo de la juventud.  En la descripción del modelo se incluyen varias cosas. Evaluación del programa de supuestos básicos, primer se describen. Estos ayudan a comprender cómo y por qué se evalúan los programas.  Junto con esto, se presentan diseños de evaluación básica que se han utilizado típicamente en los esfuerzos de evaluación. A continuación, se presenta un modelo de lógica que plantea la idea de mirar tanto de proceso como de producto aspectos de un programa.  Una teoría del cambio de enfoque se utiliza aquí (Connell y Kibrich, 1998; Izzo, et al., 2004) donde se aplican los principios básicos y estrategias para el proceso de evaluación.  Por último, un examp... de cómo este enfoque se aplica a un programa deportivo es proporcionada. Los resultados inmediatos e intermedios se muestra en el ejemplo y mostrar cómo crean una vía para impactar los resultados del último programa. El producto final esperado de este retrato es ofrecer un lente más amplio para evaluar con eficacia la eficacia de los programas de desarrollo juvenil.

https://doi.org/10.5232/ricyde2017.04908

References/referencias

Chafee, L.D. (2013). Expanded learning opportunities are key to student learning.  In T. Peterson (Ed.), Expanding minds and opportunities: The power of afterschool and summer learning for student success. (pp.17-20). Washington, DC: Collaborative Communications Group.

Connell, J., & Kubisch, A. (1998).  Applying a theory of change approach to evaluation of comprehensive community initiatives: Progress, prospects, and problems. In K. Fulbright-Anderson, A. Kubisch, & J. Connell (Eds.), New approaches to evaluating community initiatives: Vol. 2 theory, measurement, and analysis (pp. 66-79) Washington, DC: Aspen Institute. 

Deci, E.L., & Ryan, R.M. (2000). The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227-268.
https://doi.org/10.1207/S15327965PLI1104_01

Escarti, A.; Wright, P.; Paschall, C., & Guitierrez, M. (2015). Tool for assessing responsibility-based education (TARE) 2.0: Instrument revisions, inter-rater reliability and correlations between observed teaching strategies and student behaviors. Universal Journal of Psychology, 3(2), 55-63.
https://doi.org/10.13189/ujp.2015.030205

Gordon, B., & Doyle, S. (2015) Teaching personal and social responsibility and transfer of learning: Opportunities and challenges for teachers and coaches. Journal of Teaching Physical Education, 34, 152-161.
https://doi.org/10.1123/jtpe.2013-0184

Gordon, B.; Jacobs, J.M., & Wright, P.M. (2016). Social and emotional learning through ateaching personal and social responsibility based after-school program for disengaged middle-school boys. Journal of Teaching Physical Education, 35(4), 358-369.
https://doi.org/10.1123/jtpe.2016-0106

Granger, R.C. (2002).  Creating the conditions linked to positive youth development. In R.M. Lerner, C.S. Taylor, & A. von Eye (Eds.), New directions for youth development: Pathways to positive development among diverse youth. (pp.149-164). San Francisco: Jossey-Bass.
https://doi.org/10.1002/yd.20

Halpern, R. (2003). Making Play Work: The Promise of After-School Programs for Low Income Youth.  New York: Teachers College Press..

Heckman, P.E., & Sanger, C. (2013). How quality afterschool programs help motivate and engage more young people in learning, schooling, and life. In T. Peterson (Ed.), Expanding minds and opportunities: The power of afterschool and summer learning for student success. (pp. 31-33). Washington, DC: Collaborative Communications Group.

Hellison, D. (2011) Teaching Personal and Social Responsibility Through Physical Activity (3^rd Ed.). Champaign: IL: Human Kinetics.

Hemphill, M. (2015). Inhibitors to responsibility-based professional development with in-service teachers. Physical Educator, 72, 288-306.
https://doi.org/10.18666/tpe-2015-v72-i5-5756

Hirsch, B. (2005).  A Place to Call Home: After School Programs for Urban Youth. New York: Teachers College Press.
https://doi.org/10.1037/11087-000

Intractor, S., & Siegal, D. (2008). Project coach: Youth development and academic achievement through sport. Journal of Physical Education, Recreation, and Dance, 79(7), 17-23.
https://doi.org/10.1080/07303084.2008.10598210

Intrator, S.M., & Siegel, D. (2014). The Quest for Mastery: Positive Youth Development Through Out-of-School Programs. Cambridge, MA: Harvard Educational Press.

Izzo, C. V.; Connell, J. P.; Gambone, M. A., & Bradshaw, C. P. (2004). Understanding andunderstanding and improving youth development initiatives through evaluation.  In S.F. Hamilton and M.A. Hamilton (Eds.), The youth development handbook: Coming of\age in American communities (pp. 301-325). Thousand Oaks, CA: Sage.
https://doi.org/10.4135/9781452232560.n13

MacDonald, D. J., & McIsaac, T. (2016). Quantitative assessment of positive youth development in sport. In N. Holt (Ed.). Positive youth development through sport (pp. 83-96). London: Routledge.

MacDonald, G. B., & Valdivisso, R. (2000).  Measuring deficits and assets: How we track youth development now and how we should track it. In Public/Private Ventures (Ed.), Youth development: issues, challenges and directions (pp. 149-184). Philadelphia: Public/Private Ventures.

Madaus, G. F.; Scriven, M., & Stufflebeam, D. L. (1986). Evaluation Models: Viewpoints on Educational and Human Services Evaluation. Boston: Kluwer-Nijhoff.

Martinek, T. (2000).  Program evaluation.  In Hellison, D. Cutforth, N., Kallusky, Martinek, T. Parker, M., & J. Stiehl (Eds.), Serving underserved youth through physical        activity: Toward a model for school-university collaboration (pp. 211-228), Champaign, IL: Human Kinetics. 

Martinek, T. (2016). Project Effort: Creating responsible leadership through values based learning. Active and Health Magazine. 23(2/3), 43-46.

Martinek, T., & Hellison, D. (2016). Learning responsibility through sport and physical activity. In N. Holt (Ed.). Positive youth development through sport (pp. 49-60).  London: Routledge.

Martinek, T.; McLaughlin, D., & Schilling, T. (1999). Project effort: Teaching responsibility beyond the gym. Journal of Physical Education, Recreation, and Dance, 70, 12-25.
https://doi.org/10.1080/07303084.1999.10605954

Melendez, A., & Martinek, T. J. (2015).  Life after project effort: Applying values in a responsibility-based physical activity program. RICYDE. Revista internacional de ciencias del deporte, 41(11), 258-280.
https://doi.org/10.5232/ricyde2015.04105

National Research Council and Institute of Medicine (2002). Community Programs to Promote Youth Development. Washington, DC: National Academy Press.

Patton, M. (1997). Utilization-Focused Evaluation (3rd Edition). Thousand Oaks: Sage.

Ryan, R.M., Williams, G.C., Patrick, H., & Deci, E.L. (2009). Self-determination theoryand physical activity: The dynamics of motivation in development and wellness. Hellenic Journal of Psychology, 6, 107-124.

Schilling, T.; Martinek, T., & Carson, S. (2007). Developmental processes among youth leaders in an after-school, responsibility-based sport program: Antecedents andbarriers to commitment.  Research Quarterly for Exercise and Sport. 78, 48-60.
https://doi.org/10.1080/02701367.2007.10599403

Shadish, H.W.; Cook, T., & Campbell, D.T. (2002). Experimental and quasi-experimental design for generalized causal inference. Belmont, CA: Wadsworth

Simonton, S. (2016). Keeping adolescents engaged: What can after-school programs do? Youth Today. 25(2), 1, 23-24.

Wright, P. (2012). Offering a TPSR physical activity club to adolescent boys labeled "at risk" in partnership with a community-based youth serving program. Agora for Physical Education and Sport, 14, 94-114.

Wright, P., & Craig, M. (2011). Tool for assessing responsibility-based education (TARE). Instrument development, content validity, and inter-rater reliability. Measurement in Physical Education and Exercise Science. 15(3), 204-219.
https://doi.org/10.1080/1091367X.2011.590084

Wright, P.M.; Ding, S., & Pickering, M. (2010). Integrating a personal and social responsibility program into a lifetime wellness course for urban high school students: Assessing implementation and educational outcomes. Sport, Education, and Society, 15, 277-296.
https://doi.org/10.1080/13573322.2010.493309