Research

Quantitative Consulting has many interesting research publications. Some of them were completed for the company purposes and others were outcomes of the academic research of our employees. Some of the content is accessible only after registration of your email address, rest of the content is downloadable from this site.

We hope you will find what you are looking for!

Publications

filter by

order by

alphabet:
date:
A Bayesian Approach to Backtest Overfitting, Jiří Witzany, 2017

A Bayesian Approach to Backtest Overfitting, Jiří Witzany, 2017

Backtest, Bayesian Probability, Bootstrapping, Cross-Validation, Investment Strategy, MCMC, Multiple Testing, optimization, Probability of Backtest Overfitting, Sharpe Ratio

Quantitative investment strategies are often selected from a broad class of candidate models estimated and tested on historical data. Standard statistical technique to prevent model overfitting such as out-sample back-testing turns out to be unreliable in the situation when selection is based on results of too many models tested on the holdout sample. There is an ongoing discussion how to estimate the probability of back-test overfitting and adjust the expected performance indicators like Sharpe ratio in order to reflect properly the effect of multiple testing. We propose a consistent Bayesian approach that consistently yields the desired robust estimates based on an MCMC simulation. The approach is tested on a class of technical trading strategies where a seemingly profitable strategy can be selected in the naïve approach.

04.08.2017