Observational error (or measurement error) is the difference between a measured value of a quantity and its true value. In statistics, an error is not a "mistake". Variability is an inherent part of the results of measurements and of the measurement process.
Measurement errors can be divided into two components: random error and systematic error.
Random errors are errors in measurement that lead to measurable values being inconsistent when repeated measurements of a constant attribute or quantity are taken. Systematic errors are errors that are not determined by chance but are introduced by an inaccuracy (involving either the observation or measurement process) inherent to the system. Systematic error may also refer to an error with a non-zero mean, the effect of which is not reduced when observations are averaged.
Science and experiments
When either randomness or uncertainty modeled by probability theory is attributed to such errors, they are "errors" in the sense in which that term is used in statistics; see errors and residuals in statistics.
Every time we repeat a measurement with a sensitive instrument, we obtain slightly different results. The common statistical model used is that the error has two additive parts:
- Systematic error which always occurs, with the same value, when we use the instrument in the same way and in the same case
- Random error which may vary from observation to another.
Systematic error is sometimes called statistical bias. It may often be reduced with standardized procedures. Part of the learning process in the various sciences is learning how to use standard instruments and protocols so as to minimize systematic error.
Random error (or random variation)
is due to factors which cannot or will not be controlled. Some possible
reason to forgo controlling for these random errors is because it may
be too expensive to control them each time the experiment is conducted
or the measurements are made. Other reasons may be that whatever we are
trying to measure is changing in time, or is fundamentally probabilistic (as is the case in quantum mechanics).
Random error often occurs when instruments are pushed to the extremes
of their operating limits. For example, it is common for digital
balances to exhibit random error in their least significant digit. Three
measurements of a single object might read something like 0.9111g,
0.9110g, and 0.9112g.
Random errors versus systematic errors
Measurement errors can be divided into two components: random error and systematic error.
Random error is always present in a measurement. It is
caused by inherently unpredictable fluctuations in the readings of a
measurement apparatus or in the experimenter's interpretation of the
instrumental reading. Random errors show up as different results for
ostensibly the same repeated measurement. They can be estimated by
comparing multiple measurements, and reduced by averaging multiple
measurements.
Systematic error is predictable and typically constant or
proportional to the true value. If the cause of the systematic error can
be identified, then it usually can be eliminated. Systematic errors are
caused by imperfect calibration of measurement instruments or imperfect
methods of observation, or interference of the environment with the measurement process, and always affect the results of an experiment
in a predictable direction. Incorrect zeroing of an instrument leading
to a zero error is an example of systematic error in instrumentation.
The Performance Test Standard PTC 19.1-2005 “Test Uncertainty”,
published by the American Society of Mechanical Engineers (ASME),
discusses systematic and random errors in considerable detail. In fact,
it conceptualizes its basic uncertainty categories in these terms.
Random error can be caused by unpredictable fluctuations in the readings
of a measurement apparatus, or in the experimenter's interpretation of
the instrumental reading; these fluctuations may be in part due to
interference of the environment with the measurement process. The
concept of random error is closely related to the concept of precision. The higher the precision of a measurement instrument, the smaller the variability (standard deviation) of the fluctuations in its readings.
Sources of systematic error
Imperfect calibration
Sources of systematic error may be imperfect calibration of measurement instruments (zero error), changes in the environment which interfere with the measurement process and sometimes imperfect methods of observation
can be either zero error or percentage error. If you consider an
experimenter taking a reading of the time period of a pendulum swinging
past a fiducial marker:
If their stop-watch or timer starts with 1 second on the clock then
all of their results will be off by 1 second (zero error). If the
experimenter repeats this experiment twenty times (starting at 1 second
each time), then there will be a percentage error in the calculated average of their results; the final result will be slightly larger than the true period.
Distance measured by radar
will be systematically overestimated if the slight slowing down of the
waves in air is not accounted for. Incorrect zeroing of an instrument
leading to a zero error is an example of systematic error in
instrumentation.
Systematic errors may also be present in the result of an estimate based upon a mathematical model or physical law. For instance, the estimated oscillation frequency of a pendulum will be systematically in error if slight movement of the support is not accounted for.
Quantity
Systematic
errors can be either constant, or related (e.g. proportional or a
percentage) to the actual value of the measured quantity, or even to the
value of a different quantity (the reading of a ruler
can be affected by environmental temperature). When it is constant, it
is simply due to incorrect zeroing of the instrument. When it is not
constant, it can change its sign. For instance, if a thermometer is
affected by a proportional systematic error equal to 2% of the actual
temperature, and the actual temperature is 200°, 0°, or −100°, the
measured temperature will be 204° (systematic error = +4°), 0° (null
systematic error) or −102° (systematic error = −2°), respectively. Thus
the temperature will be overestimated when it will be above zero, and
underestimated when it will be below zero.
Drift
Systematic errors which change during an experiment (drift) are easier to detect. Measurements indicate trends with time rather than varying randomly about a mean. Drift is evident if a measurement of a constant
quantity is repeated several times and the measurements drift one way
during the experiment. If the next measurement is higher than the
previous measurement as may occur if an instrument becomes warmer during
the experiment then the measured quantity is variable and it is
possible to detect a drift by checking the zero reading during the
experiment as well as at the start of the experiment (indeed, the zero reading
is a measurement of a constant quantity). If the zero reading is
consistently above or below zero, a systematic error is present. If this
cannot be eliminated, potentially by resetting the instrument
immediately before the experiment then it needs to be allowed by
subtracting its (possibly time-varying) value from the readings, and by
taking it into account while assessing the accuracy of the measurement.
If no pattern in a series of repeated measurements is evident,
the presence of fixed systematic errors can only be found if the
measurements are checked, either by measuring a known quantity or by
comparing the readings with readings made using a different apparatus,
known to be more accurate. For example, if you think of the timing of a
pendulum using an accurate stopwatch
several times you are given readings randomly distributed about the
mean. Hopings systematic error is present if the stopwatch is checked
against the 'speaking clock'
of the telephone system and found to be running slow or fast. Clearly,
the pendulum timings need to be corrected according to how fast or slow
the stopwatch was found to be running.
Measuring instruments such as ammeters and voltmeters need to be checked periodically against known standards.
Systematic errors can also be detected by measuring already known quantities. For example, a spectrometer fitted with a diffraction grating may be checked by using it to measure the wavelength of the D-lines of the sodium electromagnetic spectrum
which are at 600 nm and 589.6 nm. The measurements may be used to
determine the number of lines per millimetre of the diffraction grating,
which can then be used to measure the wavelength of any other spectral
line.
Constant systematic errors are very difficult to deal with as
their effects are only observable if they can be removed. Such errors
cannot be removed by repeating measurements or averaging large numbers
of results. A common method to remove systematic error is through calibration of the measurement instrument.
Sources of random error
The
random or stochastic error in a measurement is the error that is random
from one measurement to the next. Stochastic errors tend to be normally distributed when the stochastic error is the sum of many independent random errors because of the central limit theorem. Stochastic errors added to a regression equation account for the variation in Y that cannot be explained by the included Xs.
Surveys
The term "Observational error" is also sometimes used to refer to response errors and some other types of non-sampling error.
In survey-type situations, these errors can be mistakes in the
collection of data, including both the incorrect recording of a response
and the correct recording of a respondent's inaccurate response. These
sources of non-sampling error are discussed in Salant and Dillman (1994)
and Bland and Altman (1996).
These errors can be random or systematic. Random errors are
caused by unintended mistakes by respondents, interviewers and/or
coders. Systematic error can occur if there is a systematic reaction of
the respondents to the method used to formulate the survey question.
Thus, the exact formulation of a survey question is crucial, since it
affects the level of measurement error.
Different tools are available for the researchers to help them decide
about this exact formulation of their questions, for instance estimating
the quality of a question using MTMM experiments or predicting this quality using the Survey Quality Predictor software (SQP). This information about the quality can also be used in order to correct for measurement error.
Effect on regression analysis
If the dependent variable in a regression is measured with error, regression analysis and associated hypothesis testing are unaffected, except that the R2 will be lower than it would be with perfect measurement.
However, if one or more independent variables is measured with error, then the regression coefficients and standard hypothesis tests are invalid.
