Refers to consistency in measurement; Something that produces similar results— not necessarily consistently good or bad, but simply consistent.

Is a statistic that quantifies reliability, ranging from 0 (not at all reliable) to 1 (perfectly reliable).

refers to the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes (Taylor, 1997, p. 3

tied to the measurement instrument used. Reliable tests give scores that closely approximately help us understand and calculate reliability, and without reliability a test cannot be valid.

a person’s standing on a theoretical variable independent of any particular measurement.

are measurement processes that alter what is measured

A statistic useful in describing sources of test score variability is the variance (σ²)—the standard deviation squared.

Variance from irrelevant, random sources is

Variance from true differences

Refers to the proportion of the total variance attributed to true variance. The greater the proportion of the total variance attributed to true variance, the more reliable the test.

Consists of unpredictable fluctuations and inconsistencies of other variables in the measurement process. This source of error fluctuates from one testing situation to another with no discernible pattern that would systematically raise or lower scores.

do not cancel each other out because they influence test scores in a consistent direction. They either consistently inflate scores or consistently deflate scores.

refers to the degree to which systematic error influences the measurement.

One source of variance during test construction is_______. terms that refer to variation among items within a test as well as to variation among items between tests.

are potential sources of error variance. The examiner’s physical appearance and demeanor—even the presence or absence of an examiner—are some factors for consideration here.

Other Source of Errors

An Estimate Of Reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test

When the interval between testing is greater than six months, the estimate of test-retest reliability is often referred to as the

an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error.

Is obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once.

allows a test developer or user to estimate internal consistency reliability from a correlation between two halves of a test

refers to the degree of correlation among all the items on a scale. A measure of inter-item consistency is calculated from a single administration of a single form of a test.

may be thought of as the mean of all possible split-half correlations, corrected by the Spearman–Brown formula.

Variously referred to as scorer reliability, judge reliability, observer reliability, and interrater reliability is the degree of agreement or consistency between two or more scorers (or judges or raters) with regard to a particular measure

the simplest way of determining the degree of consistency among scorers in the scoring of a test is to calculate a coefficient of correlation. This is referred to as a

The Nature of the Test

also referred to as the true score (or classical) model of measurement.

seek to estimate the extent to which specific sources of variation under defined conditions are contributing to the test score.

is based on the idea that a person’s test scores vary from testing to testing because of variables in the testing situation.

provides a way to model the probability that a person with X ability will be able to perform at a level of Y.

is the tool used to estimate or infer the extent to which an observed score deviates from a true score.

a range or band of test scores that is likely to contain the true score.

a statistical measure that can aid a test user in determining how large a difference should be before it is considered statistically significant.

Pressing emotional problems, physical discomfort, lack of sleep, and the effects of drugs or medication can all be sources of error variance

Potential sources of error variance. The examiner's physival appearance and demeanor-even the presence or abscence of an examiner-are some factors for consideration here.

the simplest way of determining the degree of consistency among scorers in the scoring of a test is to calculate a coefficient of correlation. This is referred to as a