Lecture Notes for Simulation

14 February 2005 - Probability Distributions and Statistics

Review

• An experiment produces outcomes drawn from a sample space.

• A subset of the sample space is known as an event.

• Given a sequence of N trials, event e_i occurs n_i times.

• The relative frequency of event e_i is RF(e_i) = n_i/N.

• The probability of event e_i is P(e_i) = RF(e_i) as N goes to infinity.

• A random variable maps outcomes to real numbers.

Probability Density Graphs

• Mapping sample spaces to numbers enables various operations.

• Plot the values returned by a random variable vs. the probability of the random variable being that value.

  The x axis gives random-variable values xi. The y axis gives the P(X = xi).

• Eyeballing probability density graphs is a useful start to matching data sets to probability distributions.

• These graphs are vaguely related to probability density functions.

Probability Distributions

The Bernoulli Distribution

• Consider any experiment with a sample space of size two.

  • Or any random variable mapping into a range of size two.

  • Coin flips, pass-fail tests, boy-girl results.

• One outcome occurs with probability p, arbitrarily called the success probability, the other with probability q = 1 - p.

• A trial with two possible outcomes is a Bernoulli trial; a collection of Bernoulli trials form a Bernoulli distribution.

  A random variable that maps one outcome to 0 and the other to 1 is convenient.

The Binomial Distribution

• What's the probability of buying a dozen uncracked eggs?

  • Each egg is a Bernoulli trial.

• Sequences of N independent Bernoulli trials form a binomial distribution.

• The random variable X = the number of heads in O.

• The probability P(X) = i) = C(i, n)pⁱq^{n - 1}.

The Geometric Distribution

• My sister has four kids; only the youngest is a boy. What's the probability of that?

• More generally, given a binomial trial of size n, what's the probability it's n - 1 x₁s followed by a x₂?

• Sequences of such binomial trials form a geometric distribution.

• The random variable X = the number of heads preceding the first tails.

• The probability P(X = i) = pⁱq.

The Poisson Distribution

• A binomial trial is like an arrival trial.

  • How many heads in N flips?

  • How many new customers in the next N ticks?

• The Poisson distribution generalizes the binomial distribution to deal with arrivals (or other similar outcomes).

• The random variable X = number of events occurring during a time interval.

• The probability P(X = k) = e^-a(a^k/k!).

  • a is the expected number of events per time interval.

  

The Uniform Distribution

• The uniform distribution is the distribution of minimal knowledge.

  • The only thing known are largest and smallest possible outcomes.

• The random variable is X_i = i, 0 <= i < N.

  • The random variable X_i = i/(N - 1) maps the outcomes into the unit interval [0, 1], forming the standard uniform distribution.

• The probability is P(X = i) = 1/(max - min).

  Although simple, the uniform distribution is important because it models randomness.

The Normal Distribution

• The normal distribution is the distribution of central tendency.

  • Phenomena that symmetrically clumps around a value.

  • Repeated value measurements, human lifespans.

• The density function is f(x) = exp(-(x - m)²/2s²)/(sqrt(2pi)s).

  • m (mu) is the mean and represents the central tendency.

  • s (sigma) is standard deviation and represents clumping strength.

  The standard normal distribution has m = 0 and s = 1.

The Exponential Distribution

• The exponential distribution is the distribution of things that should occur real soon now.

  • "Constant rate" customer interarrival or service times.

  • Sudden or catastrophic failure.

• The pdf f(x) is exp(-x/m)/m, m is the mean.

The Chi-Square Distribution

• The chi-square random variable Y is the sum of n independent standard normal random variables squared:
  Y = X₁² + X₂² + ... + X_n²

• The parameter n is the distribution's degrees of freedom.

• The chi-square pdf is f(x) = (x^{n/2 - 1}e^-x/2)/(2^n/2 G(n/2))

  Because measurement errors follow a normal disribution, the chi-square distribution is the basis for several techniques that quantify measurement quality.

The Student's t-Distribution

• The Student's t-distribution is the ratio of normals distribution.

• If N is a standard normal random variable and C is a chi-square random variable with k degrees of freedom, then
  T = N/(C/k)^1/2

  is a t-distributed random variable with k degrees of freedom.

  k is a parameter to the distribution. The pdf f(x) is a/b, where a = G((k + 1)/2)/((k*pi)1/2*G(k/2)) and b = 1/(1 + x2/k)(k + 1)/2

The F Distribution

• The F distribution is the ratio of variance distribution.

• If C₁ and C₂ are chi-squared random variable with k₁ and k₂ degrees of freedom respectively, then
  F = (C₂/k₂)/(C₁/k₁)

  is an F distributed random variable with k₁ and k₂ degrees of freedom.

• The pdf with parameters k₁ and k₂ is a/b, where
  a = G((k₁ + k₂)/2) k₁^k₁/2 k₂^k₂/2 x^{k₂/2 - 1}
  b = G(k₁/2) G(k₁/2) (k₁ + k₂x)^{(k₁ + k₂)/2}

F-Distribution Examples

Statistics

• A statistic is an equation over random variables with no unknowns.

• Statistics are another useful way of characterizing distributions.

• Several statistical values are used as parameters to distributions.

  • These parameters can be fined tuned to improve the distribution's fit to the data being modeled.

This page last modified on 28 March 2005.