Maintaining a common arbitrary unit in social measurement

Stephen Humphry

In educational assessment, it is common to equate test forms in order to draw comparisons between different populations of students. The process of test equating presents a number of challenges, many of which relate inherently to the problem of maintaining a common unit and origin. In order to develop a general theoretical approach to maintaining a common unit and origin in the measurement of quantitative attributes, the role of the unit is carefully examined. Classical physics is explicitly adopted as the guiding paradigm during the investigations throughout the dissertation. Accordingly, the central objective is to develop a theoretical foundation for maintaining a common unit and origin which meets two criteria: (i) it must be congruent with the definition of measurement in physics captured in the classical theory of measurement (Michell, 1999); and (ii) it must meet a key requirement of measurement in the physical sciences identified by Rasch (1960/1980). Rasch identified the relevant requirement, that of invariant comparison, based on analysis of Newton's second law and showed that the Principle of Invariant Comparison is formally embodied in his measuring function for dichotomous data (Rasch, 1960/1980). This model provides the basis for the development and exposition of general concepts and principles in the dissertation. In order to achieve the central objective, the unit is made formally explicit and specified in relation to the experimental frame of reference. Rasch (1977) defined a Specified Frame of Reference (SFR) in terms of a collection of objects, a collection of agents, and outcomes of the interaction between these. Drawing on a fundamental distinction introduced by Andrich (2003), the unit of a SFR is referred to as a natural unit and is distinguished from an arbitrary unit, the magnitude of which is theoretically independent of any particular SFR and instrument contained within. From this distinction, a definition of discrimination arises naturally; a definition that is also congruent with classical physics. The distinction and related definitions provide the basis for derivation of a general form of Rasch's measuring function for dichotomous data, referred to as the Extended Frame of Reference Model (EFRM). It is shown that the EFRM provides a rational basis for maintaining a common unit and origin in assessment contexts involving two or more Specified Frames of Reference. Simulation and empirical studies are employed to illustrate application of the EFRM. These studies also serve to illustrate that quantitative hypotheses entailed by the EFRM are open to empirical tests by providing a context for the use of graphical methods and statistical tests of fit. Empirical investigations are used to illustrate consequences of differences between natural units in the context of applied educational assessment. The studies also provide a context in which to characterise the model, and the structure of data that it entails. Although the simulation studies demonstrate the basic efficacy of the model, they also indicate scope for improvement in terms of the precision of estimates. To explore possible approaches to refining the estimation process, Maximum Likelihood (ML) equations are derived and examined. Firstly, Joint Maximum Likelihood (JML) equations are presented. Following this, Conditional Maximum Likelihood (CML) equations are derived. It is shown that while the CML equations permit separation of the person and item parameters, item locations are expressed in terms of natural, rather than arbitrary, units. A particular approach is proposed, emphasising links to the classical theory and the Principle of Invariant Comparison. In considering the proposed approach, a distinguishing feature of the definition of discrimination is highlighted: specifically, the nature of its definition represents the importance of relationships between quantitative attributes, and the specific structure of these relationships, to the measurement of any particular attribute. Although it is not possible to fully study this feature given the scope of the work, it is a key to the implications of the general theoretical framework embodied in the EFRM. Accordingly, these implications are touched on before concluding the dissertation.

Maintaining a common arbitrary unit in social measurement

Files and links (2)

Abstract

Details

Metrics