Don A. Klinger Professor

Academic resilience has taken on renewed significance in the aftermath of the pandemic — a generational event that disrupted the education of more than 1.5 billion students across the globe. Unsurprisingly, cross-national research has adequately documented the profound impact of school closures on traditional cognitive domains such as reading, mathematics, and science achievement. At the same time, COVID-19 has also brought into sharper focus the important role schools play in supporting skills such as academic resilience, among others, that are commonly classified as “non-cognitive” skills, as well as their role in enhancing academic success. Yet academic resilience is often viewed as a narrow construct. Louis Volante and Don A. Klinger analyze how academic resilience has traditionally been conceptualized, along with the nature and scope of policy interventions to support students in the aftermath of the pandemic.

This study expands on our previous research focusing on provincial educational policies to support students’ academic resilience during the pandemic, with the current focus being on the “recovery phase” of the pandemic (January 2022 to December 2023). Our analysis identified 46 provincial documents that addressed one or more of the three dimensions of academic resilience during this recovery period. Similar to our previous findings, a greater emphasis was placed on academic outcomes. There was an increased focus on mental health, while much less attention was paid to physical health and well-being. While we identified examples of provinces that dedicated resources and funding to support these dimensions, we argue the policies implemented during the recovery phase will require more conviction to address the negative long-term impacts of the pandemic, particularly for disadvantaged students, and that such efforts will likely need to continue beyond public schooling.

The promotion and measurement of standards in compulsory education systems has been a prominent feature of Western education systems for centuries. But the COVID-19 pandemic and the rise of artificial intelligence (AI) have made the limits of current standards-based approaches to assessment more evident. Louis Volante, Don A. Klinger, and Christopher DeLuca discuss the importance of both cognitive and non-cognitive skills for the future success of schools. More authentic assessments, some of which leverage AI, can assess students’ performance in these essential domains.

Background: Evaluation “is a task in the best tradition of the most abstract theoretical science as well as the most practical applied science” (Scriven, 1968, p .9). The Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation (JCSEE) operationalize the theoretical aspects of evaluation and, when used, facilitate sound evaluation methods in applied settings. Between the publications of the first and second editions of The Program Evaluation Standards, the Center for Research on Educational Accountability and Teacher Evaluation (CREATE) was funded in 1990 at Western Michigan University with federal monies of $5.2 million, and between 1990 and 1995 by the United States Department of Education, Office of Educational Research and Improvement (OERI). CREATE was established for the betterment of evaluation within the educational context (Stufflebeam, 1991; Stufflebeam & Shinkfield, 1994). CREATE’s mandate and subsequent mission furthered the work of the Program Evaluation Standards and the JCSEE by using the standards in applied settings. Keeping to Scriven’s notion of evaluation as the best tradition, the collaborative work between CREATE and JCSEE is a well-established tradition that furthers the development of theoretical aspects of evaluation and the application of the evaluation standards. Purpose: Examine CREATE’s impact on the Program Evaluation Standards’ theoretical development and applied use. Setting: Not applicable. Intervention: Not applicable Research Design: Not applicable. Data Collection and Analysis: Systematic review of the theoretical development and applied use of the Program Evaluation Standards in the books, journal articles, monographs, special papers, meeting minutes, conference programs, and presentations associated with CREATE. Findings: CREATE has contributed to the operationalization of the theoretical aspects of evaluation with the Program Evaluation Standards and facilitated their use in applied settings. CREATE has also furthered the work of the Personnel Evaluation Standards and the Classroom Assessment Standards (formerly the Student Evaluation Standards). Leading scholars from CREATE and the JCSEE have contributed to the standards since the 1990s. Members of CREATE have published a notable range of books, journal articles, monographs, special papers and conference presentations related to the Program Evaluation Standards. Organizational capacity and shared goals of both the JCSEE and CREATE guided the practical application and theoretical development of the Program Evaluation Standards. Keywords: Program Evaluation Standards; Consortium for Research on Educational Assessment and Teaching Effectiveness; Joint Committee for Standards on Educational Evaluation

This study employed a qualitative content analysis of provincial policy documents, following deductive methods, to examine academic resilience and education policy developments across Canada during the COVID-19 pandemic. More specifically, the study explicates the nature and scope of provincial policy responses to the global pandemic that address academic issues, physical health and well-being, and mental health issues for K-12 students. The pan-Canadian analysis revealed a total of 62 documents were issued between January 2020 and December 2021 that addressed one or more of the triarchic dimensions of academic resilience. The findings suggested greater attention was devoted to academic issues and there was a general lack of policy differentiation in terms of how specific resources and supports were to be directed within provincial educational jurisdictions.

Self-regulation is positively associated with better academic, and life, outcomes. Consequently, many school systems aim to develop self-regulation, or related constructs. Thus, many teachers are asked to assess and report upon students’ self-regulation (or related constructs). How secondary teachers in Ontario, Canada accomplish this task was investigated using mixed methods research. Phase 1 involved semi-structured interviews with 26 secondary teachers. The second phase of the study involved the analysis of report card data to examine large scale trends in self-regulation grades. The third phase employed an online survey whose development was informed by the interviews of the first phase. The overall findings are that Ontario secondary teachers vary in their definitions of self-regulation, but the strongest influences on teachers' self-regulation assessments are negative student behaviours. Report card data suggest that teachers struggle to assess self-regulation independently from other constructs such as initiative or collaboration. Implications for practice are discussed.

The Programme in International Student Assessment (PISA) has become the prominent method of international comparison of the achievement of 15-year-old children in reading, mathematics, and science. Recently, the OECD, which administers PISA, has devoted a great deal of energy promoting the notion of "academic resilience"-which refers to the capacity of individuals to prosper despite encountering adverse circumstances. Countries are compared and contrasted in relation to the relative share of disadvantaged students that are able to achieve at higher achievement levels on PISA, with associations drawn from school-level factors and resulting implications drawn for policy reform. This paper offers a number of cautions with the growing influence of cross-national comparisons of academic resilience. Our discussion underscores how the OECD's notion of "academic resilience," which has come to dominate transnational policy debates, is quite narrow and limited by the measures it uses to assess student competencies.

Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school’s control. This study examined year to year changes in school level results on standardized tests delivered in Ontario, Canada. G-theory analyses found that test scores are not stable enough for meaningful conclusions to be made based on year to year changes in school level results. For small and medium sized schools, years of data need to be collected before defensible decisions can be made about trends in test scores. The authors introduce a ‘bounce’ statistic that provides a simple, easy to interpret measure of test score stability.

The present analysis examined the relationship between PISA results and their influence on policy development within a select group of European nations which included Estonia, Italy, France, and Finland. These countries reflect four distinct outcomes in relation to PISA results: (1) high achievement and high equity (Estonia); (2) stagnant performance (Italy); (3) low achievement and low equity (France); and (4) a negative equity trend (Finland). The discussion argues these outcomes can be associated with cases of policy inertia, avoidance, and refraction.