Input Variability as stochasticity
Deterministic and stochastic models
- when a dynamic system can experience random disturbances, these have to be mirrored by a stochastic model
- a model needs to include stochastic elements if these significantly influence the measured indicators
- one of the most frequent mistakes in considering stochastic models is evaluating the results of mean values only
- > average doesn’t tell you anything if you have no idea about the variance
Input Variability as stochasticity
Simulating variability
Fundamental part of most simulations are stochastic, since we cannot predict precisely what will happen in the future
For example:
- customer arrivals
- type of customer
- serving times
- machine cycle times
- machine breakdowns
- defective parts
- travel times
Input Variability as stochasticity
Simulating Variability
General approach
- specify overall behavior using probability distribution
- for each simulation run, select values at random from the probability distribution
- this calls for a way of making random selections - we usually start with random values from the Uniform(0,1) distribution
Input Variability as stochasticity
Simulating variability
Where do past probabilities come from?
- past data (therefore you need to have the data available)
- expert judgement (e.g. “one out of ten products” judged by an expert)
- nature of situation (e.g. throwing a coin will be either heads or tails)
-> as probabilities are a type of input data, the parameter values and their effect on simulation output have to be validated
Computing Randomness
What is randomness?
Processes:
- coin, dice, lottery, radioactive decay
Properties:
- equal chance, full range, independent (no correlation)
Computing Randomness
General approach
We need a way of making random selections
- The usual approach is to start with a random value from Uniform(0,1) distribution
- we transfer the resulting value to fit the given probability distribution of the simulation parameter
Computing Randomness
Example for a generator
Congruential random number generators
- Produces sequence of integers using equation: I=(a*I+c)mod m
- integers are all between 0 and m-1, so random number returned = I/m
- intialise with a seed (gives control)
- multiplicative version has c=0
Computing randomness
Congruential number generators
Drawbacks and how to compensate them?
- only possible values are 0/m, 1/m, 2/m, …, (m-1)/m
- repeating cycle
- maximum period before it repeats is m (m-1 if c=0)
Compensate drawbacks:
- vital to choose good values of constants
- large m and maximal cycle
-> congruential generators can work well but there are various other algorithms
Computing randomness
Testing random number generators
- test that it covers the whole range of values (see whether each number shows up with equal probability)
- test correlation
- look at values, pairs of values, gaps
- plot chart (look at the scatter plot)
- statistical tests (e.g. chi-square test)
Computing randomness
Importance of random number generator
- is a key part of most simulations
- unreliable random number generation can lead to misleading results
- the source of randomness produces input random variables with known distributions that we specify
- the input random variables are accepted by the simulation model and an output is produced
- the properties of the output random variables is what we want to estimate
Computing randomness
Using random number generators in a simulation
- we control which random numbers are used through seeds or streams - restarting a sampling process given the same seed or stream recreates the same random numbers
- using the same seeds will repeat the run exactly
- to run the model several times in experiments we need to change the seeds each time
- we can get more efficient experimentation using “common random numbers”: use different random number streams for different sources of variability: then different strategies can be compared under essentially the same conditions
Probability distributions
Continuous distributions
Various distributions are frequently used in simulations, e.g.:
- negative exponential - arrivals
- lognormal - service times
- Erlang - service times
- Normal - finance
- Weibull - times between breakdowns
- Uniform - equal probability
- Triangular - when know min, max, mode
- empirical - like a histogram and sample each bar as discrete category; use when no standard distribution fits
Probability distributions
Discrete distributions
- empirical - type of part or customer, routes
- standard discrete distributions such as binomial or Poisson, e.g. for customer arrival probabilities
Probability distributions
Negative exponential distribution
Position process:
- Given random arrivals at a constant rate, inter arrival times will follow the negative exponential distribution
Analyzing result variability
Simulation runs and random numbers
A single simulation run’s results are strongly influenced by random variations. Therefore a single run cannot lead to informative conclusions.
Processing multiple runs increases the statistical significance of results. These have to be analyzed using appropriate statistic methods:
- repeat n independent simulation runs
- calculate indicators and statistical measures (such as mean, variance) after every k and k+1 runs
- calculate confidence intervals
Analyzing result variability
Sampling
Sampling aims to draw realistic conclusions about the whole population from a mere sample
- > every simulation run is a sample
- > statistical tests to find out if the results from the sample are also applicable to the system
Analyzing result variability
Confidence intervals
A confidence interval describes the interval, in which a particular real indicator will be expected with a certain probability for a particular level of significance. The probability is determined by the level of significance, e.g. 90%, 95%, 99%
Example:
Does the simulation indicate a mean to be at least 10% greater than 100 with a 95% probability?
Analyzing result variability
Confidence intervals
Additional info
- CI is only realistic if the model correctly mirrors real variations and distributions
- large CI’s indicate that the number of simulation runs was not sufficient
- a simulation converges if the desired CI is reached based on the processed runs
- When CI’s overlap for two scenarios, the difference in the indicators in not significant
Analyzing result variability
Why student rather than normal distribution?
- every simulation run is a random draw of mostly normally distributed parameters
- a limited number of draws (sample) from a normally distributed population follows the Student (t) distribution
-
1 - Introduction16
-
2 - Forecasting and Estimation37
-
3 - Data Warehouses and OLAP44
-
4 - Unsupervised learning: clustering and association rules37
-
5 - Supervised Learning, Classification26
-
6 - Supervised learning, classification13
-
7 - Validating and Evaluating Data Models19
-
8 - Data Visualization26
-
9 - Simulation Modelling45
-
10 - Stochastic Simulation Modelling19
-
11 - Validating simulation models28
