Abstract
In his book, Signposts of Success: Interpreting Ontario's Elementary School Test Scores, [David Johnson] (2005) employs regression procedures to separate out socio-economic factors from other factors that explain student performance and to control for socio-economic status when ranking schools. The first two chapters lay the foundation for the study Johnson completed. The first chapter addresses the need to ensure that comparisons among schools are valid and not open to misinterpretation due to the presence of unaccounted for socio-economic differences among schools. The factors included in the study are then identified. Unfortunately, he does not acknowledge that the use of SES alone may in itself be a limitation as other relevant contextual variables may also be important. Chapter 2 contains (a) a brief history of "jurisdiction-wide" assessment programs in Canada and the United States followed by a brief, but informative description of the activities, costs, and the public face of and reactions to the province-wide assessments conducted in Ontario by the Education Quality and Accountability Office (EQAO), and (b) a brief review of previously completed studies of the variance accounted for among schools on the Ontario assessments and similar studies conducted in other jurisdictions. The next four chapters contain the methodology and the results and a discussion of the results. Chapter 3 describes how the values of the socio-economic variables, identified in Chapter 1, were determined for each school in the samples using census data and the postal codes of the students who attended the school and correctly identifies the limitations of using the census-postal code approach. Chapter 4, the centre-piece of the book, contains a description of the regression analyses (presented in an Appendix to the chapter) and the application of these analyses to determine which of the factors described in Chapter 3 explains the variation in the proportion of students at Level 3 and Level 4 (The Ontario Curriculum sets Level 3 [student meets curriculum standards] as the provincial standard). In brief, Johnson suggests that approximately 25 per cent of the variation in the school proportions of students at Levels 3 and 4 is accounted for by the factors considered (pp. 73; 107). Unexpectedly and serving as a distraction, the influence of the same factors on the rate of exemptions is reported in Chapter 5. Chapter 6 (with its Technical Appendix) returns to the proportions of students who meet the curriculum standards and compares the Fraser Institute school rankings with school rankings based on Johnson's method in which schools with a similar socio-economic standing are compared. Interestingly, Chapter 7 describes a study in which principals, teachers, and parents in 13 schools that outperformed other schools with a similar socio-economic status were interviewed to identify factors other than socio-economic status that led to increased performance. Lastly, Chapter 8 contains the conclusions reached on the basis of the findings presented in Chapters 4, 6, and 7, and a personal assessment of the effectiveness of the province-wide assessment in Ontario and recommendations for improving the process.