The setup

Quant Arena runs 72 systematic strategies on up to 10 markets — US stocks (SPY), gold, silver, oil, natural gas, agriculture, broad commodities, EUR/USD, bitcoin and ether — with the same rules, the same walk-forward optimization and the same metrics. Over the last five years of data that gives 576 strategy-and-market results.

For each one we compute two numbers that try to answer the question "is this edge real or luck?":

  • The probabilistic Sharpe ratio (PSR): the probability that the true Sharpe ratio is above zero, given how long and how lumpy the return history is.
  • The deflated Sharpe ratio (DSR): the same question, but against the best Sharpe ratio you would expect from luck alone when you test many strategies at once.

What the numbers say

Looked at one by one, 104 of the 576 results have a PSR above 95%. That is the kind of number that ends up in a marketing deck.

The DSR tells a different story. Not a single result clears 95%. The best is SAR-01 Parabolic SAR on EUR/USD at 91%, followed by REG-14 Volatility Regime Switch on gold (85%), MACRO-01 Cross-Asset Momentum on SPY (84%), TREND-20 Vortex Indicator on EUR/USD (83%), and MR-03 Kalman Filter Mean Reversion on broad commodities and VOL-13 Volatility Regime Strategy on gold (82% each). The median DSR across all 576 results is 23%.

In plain words: if you pick the best-looking strategy after trying hundreds, a five-year Sharpe ratio of 1.2–1.5 is roughly what chance alone would hand you.

Why this happens

Every extra strategy you test is another lottery ticket. With 60 strategies on the same market, the luckiest one will usually show a Sharpe ratio well above its true value. The DSR, introduced by Bailey and López de Prado, estimates how high that "luckiest ticket" should be from the spread of all the results, and only gives credit for the part above it.

Two details in our setup make the bar higher, not lower. First, the same strategy appears on many correlated markets. Second, walk-forward optimization picks parameters from a small grid every six months, which is itself a form of testing. We count only the strategies, not the parameter sets, so if anything our DSR is still generous.

Buy-and-hold is a hard benchmark

The same five years show how high the bar was. Buy-and-hold SPY returned about 13.2% a year with a Sharpe ratio of 0.81. Only 1 of the 62 strategies tested on SPY beat it on annual return, and 14 beat it on Sharpe ratio. Most timing strategies spend part of the time in cash, so they give up return in exchange for smaller drawdowns — a trade-off that can be worth it, but is not "beating the market".

How to use this when you read a leaderboard

  • Treat a high Sharpe ratio as a reason to look closer, not as a conclusion. Check the DSR on the strategy page.
  • Prefer strategies that hold up across markets and time windows. The Arena Score's robustness component uses the worst Sharpe across the 1-, 3-, 5-year and full windows for exactly this reason.
  • Look at what a strategy was designed to do. A trend filter that halves the drawdown of buy-and-hold is doing its job even if its DSR is modest.
  • Remember the costs. Our backtests do not subtract commissions or slippage; strategy pages show an estimate after 10 basis points.

What we changed

The deflated Sharpe ratio now appears on every strategy page next to the probabilistic Sharpe ratio, and the Arena Score uses PSR for its significance component. We would rather show an honest "probably luck" than a confident number that will not survive the next five years.