Backtest Overfitting | Translated in R

TimelyPortfolio

Download

Original Paper


Pseudo-Mathematics and Financial Charlatanism:

The Effects of Backtest Overfitting on Out-of-Sample Performance

Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim
October 7, 2013
Available at SSRN: //ssrn.com/abstract=2308659

Constants - R Equivalents

# these are the constants referenced
# Euler–Mascheroni's constant
-digamma(1)

## [1] 0.5772

# euler constant
exp(1)

## [1] 2.718

Expected Maximum Sharpe \(E[max_N]\)

\[ \begin{aligned} (1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right] \end{aligned} \]

R Translation

### Eq. 4
emax <- function(N) {
  ( ( 1 + digamma(1) ) * qnorm( 1 - 1/N ) ) +  
      (-digamma(1) * qnorm( 1 - (1/N) * exp(-1)))
}

emax( N = 10 )  # should be about = 1.57 to match paper

## [1] 1.575

Plot \(E[max_N]\)

#implement upper bound
upbound <- function(N) {sqrt(2*log(N))}
#make a ugly plot for reasonableness check
curve(upbound, from = 1, to = 1000, col = "red", lty = 2)
curve(emax, from = 1, to = 1000, add=TRUE)
grid()

plot of chunk unnamed-chunk-4

rChart \(E[max_N]\)

require(rCharts)
df <- data.frame(
  list(x=c(1,2:1000), y=c(0,emax(2:1000))))
d1 <- dPlot( y ~ x, groups = "x", data = df, type = "line", height = 270, width = 800)
d1$xAxis(type = "addMeasureAxis",orderBy = "x",outputFormat = ",0.0f")
d1$yAxis( outputFormat = ".2f")
d1

Check My Math with Eq. 6

### Try next example for Eq. 6
# if y = 5
# so solve for annualized Sharpe of 1
# says no more than 45 N should be tried

# first just do this to make sure I understand
N = 45
y = 5
emax( N ) * y^-0.5  #seems like on the right path

## [1] 0.9998

Minimum Backtest Length \(MinBTL\)

\[ \begin{aligned} \left(\frac{(1-\gamma)Z^{-1}\left[1-\frac{1}{N}\right]+\gamma Z^{-1}\left[1-\frac{1}{N}e^{-1}\right]}{\overline{E[max_N]}}\right)^2 \end{aligned} \]

R Translation 

#use emax from earlier for numerator
minBTL <- function( N, eMaxSharpe = 1 ) {
  (emax(N) / eMaxSharpe) ^ 2
}
#then this should equal 5 if correct
minBTL( N = 45, eMaxSharpe = 1 )

## [1] 4.998

Plot \(MinBTL\)

#make ugly plot for a reasonableness check
curve( minBTL, from = 1, to = 1000)

plot of chunk unnamed-chunk-8

rChart \(MinBTL\)

df <- data.frame(
  list(x=c(1,2:1000), minBTL=c(0,minBTL(2:1000))))
n1 <- nPlot( minBTL ~ x, data = df, type = "lineChart", height = 270, width = 800)
n1$yAxis( tickFormat = "#!d3.format(',.2f')!#")
n1$chart( useInteractiveGuideline = TRUE )
n1

Thanks

  • Ramnath Vaidyanathan
  • Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim