What are 4 methods used to evaluate investment strategies?
- backtesting: how strategy performed if implemented in past
- historical (scenario analysis) stress testing: how strategy performs under certain conditions
- simulation: how strategy would perform in hypothetical scenarios rather than limiting it to actual historical scenarios
- sensitivity analysis: quantify impact of changing assumptions (variables) in a model
What is the objective of backtesting?
- use backtesting to determine how their real life investment strategies would have performed using historical data, allows for managers to gain insights & improve their skills
What 3 steps does backtesting process involve?
- strategy design
- historical investment simulation
- analysis of backtesting
What are 7 key activities in the strategy design step of the backtesting process?
- specify investment hypothesis & goals
- determine investment rules & processes
- decide key parameters (ratios)
- investment universe
- return definition
- rebalancing frequency & transaction costs
- start & end date
Describe the second step (historical investment simulation) of the backtesting process.
- build portfolio based on rules established in step 1 & simulate how it would have performed during historical period
What is rolling window backtesting?
- take orginal sample, then see how orginal sample performs in different time frame and it becomes out of sample data, take out of sample data and test again for a new time frame and it becomes the in sample data (process is repeated over & over)
Describe step 3 of the backtesting process (analysis of backtesting output).
- analyze return data and consider key metrics
What are 3 common problems in backtesting?
- survivorship bias: not accounting for stocks dropped from indexes will have impact on backtesting
- look-ahead bias: information was unknown or unknowable during backtesting timeline
- data snooping bias: when data is overanalyzed which gives rise to statistically irrelevant & occasionally nonexistent patterns
When performing historical scenario analysis what are 2 significant structural breaks you must consider? What are structural breaks in economic conditions?
- expansions vs recessions
- periods of high volatility vs periods of low volatility
- structural breaks: unexpected change over time in economic conditions
What is sensitivity analysis?
- sensitivity analysis: quantify impact of changing assumptions (variables) in a model
What are 2 types of simulations that seek to avoid backtesting drawbacks?
- historical simulation
- Monte Carlo simulation
What are the 8 steps to a properly designed simulation?
- determine target variable (usually portfolios return)
- specify key decision variables (returns on individual assets & weights within portfolio)
- specify number of trials
- define distributional properties (eg. normal, log normal, binomial, etc)
- draw N random numbers for each key decision variable using random number generator
- compute value of target variable for each set of simulate decision variables
- repeat straps 5&6 until N trials have been complete
- with N returns you can calculate mean return, volatility, sharpe ratio, etc)
