How should one understand probability distributions?

An experiment providing information about the probability distribution of a random variable is, in the terminology of EEEQ, an experiment ascribing the value v(f(x)) to the expectation value of all sufficiently simple functions f(x). A knowledge of all such expectation values is equivalent to a knowledge of the probability distribution.

In practice, however, what one measures is many instances of x, and one can, for a random variable with a continuous spectrum, construct arbitrarily nasty, irregular functions, which, for example, have the value 1 for all measurements to date, and the value 0 for all measurements to be made next week. Any statistical methods assume that such irregular functions are not taken into account.

Arnold Neumaier (
A theoretical physics FAQ