ABSTRACT
The history of science is a history of measurement in that measurement quality sets an upper bound on science quality. Just as a test cannot consistently be more valid than it is reliable, research findings cannot be more solid than the measurements on which they are based. The main purpose of this chapter is to help investigators in clinical and counseling research better understand fundamental measurement issues so that they can improve the quality of their research. The first major section entitled the Fundamentals of Measurement Theory reviews basic concepts inmeasurement theory, includingmeasurement error, classical reliability theory, reliability coefficients, standard error of measurement, parallel tests, domain sampling, coefficient alpha, and alternatives to alpha. The principle of aggregation is introduced and leads to the development of a scale for determining the number of repeated measurements needed to achieve a predetermined level of reliability. This is analogous to designing a study so that it has a predetermined level of statistical power. The next section discusses the impact of reliability on validity. Increasing the former predictably increases the latter. The next major section, entitled Developing Operational Definitions, discusses both the univariate and multivariate case. The following section entitled, Methods of Collecting Data, covers interviews, questionnaires, behavioral observation, psychological tests, and instruments. A subsequent section discusses how instruments can and have driven the construction of scientific theory. Reasons are given for why instruments can make such contributions. The next section, entitled Types of Psychological Scales,
covers nominal, ordinal, interval, and ratio scales. The importance of measurement units is raised and considered in further detail in a subsequent fifth section, entitled Units of Measure. Measurement units in psychology are discussed. An example is presented showing how the absence of units can lead to measurement that is highly reliable and valid but inaccurate. A method for evaluating the reliability of instruments is presented. The following section, entitled Reliability of Measurement: Generalizability Theory, extends the material on reliability presented in the Fundamentals of Measurement Theory section to present an introduction to and overview of generalizability theory. The Validity of Measurements section reviews construct, convergent, discriminant, content, and criterion-related validity. The issue of phantom measurement is discussed. The final section, entitled Measuring Outcomes, discusses the evaluation of change and the unreliability of change scores, among other topics.
