01 - The Backtest Is Not The Track Record

Historical Results Can Justify Investigation

Historical results can justify investigation. They cannot establish an operating record.

Family offices and fund-of-funds teams routinely encounter managers whose research is compelling but whose live history is too brief — or nonexistent — to support a conventional allocation decision. The standard institutional response is often to wait for a longer track record, pass on the opportunity entirely, or rely heavily on narrative, references, and pedigree in place of verified performance data.

Each response has a cost:

• Waiting can delay access to differentiated alpha.

• Passing may forfeit the opportunity altogether.

• Relying on narrative substitutes qualitative confidence for quantitative evidence.

  
  

There is another path for allocators willing to structure diligence differently: a defined, controlled forward-market program that observes a strategy in real time before it receives a larger mandate.

What Live Markets Reveal

A backtest can show what a strategy would have done under selected historical assumptions: chosen data windows, modeled transaction costs, assumed fill quality, and survivorship-adjusted universes.

What it cannot show is how the strategy behaves when those assumptions encounter live markets.

A live, controlled sandbox can establish evidence around:

• Signal integrity under current market regimes and real data-feed conditions

• Executable fills at the prices and sizes assumed by the model

• Realized trading costs, including slippage, financing, commissions, and market impact

• Liquidity behavior during volatility spikes, illiquidity, or regime transitions

• Exception handling when the system encounters unmodeled conditions

• Decision discipline, including whether the manager follows the documented process or applies overrides

• Operating reliability across an uninterrupted live trading cycle

  
  

A backtest cannot establish these items conclusively because they depend on real-time market conditions, actual broker and venue behavior, live infrastructure, and the manager’s actions under pressure.

Reframing the Diligence Question

A live sandbox changes the central diligence question.

Instead of asking, “Is this historical result attractive?” — a question that a backtest alone can never fully answer — it asks:

Does this process behave as expected under current market conditions?

That is a forward-looking and falsifiable test. It allows an allocator to observe whether research assumptions, execution procedures, operational controls, and manager discipline remain intact when the strategy is exposed to real market conditions.

This approach allows allocators to begin investigating earlier in a manager’s lifecycle without treating a backtest as a substitute for the operating record it was never designed to provide.

Diligence Approaches

A controlled forward-market sandbox does not replace statistical rigor. It supplements it with observable, real-time evidence that a backtest cannot structurally produce.

The Objective

The objective is not to replace diligence with a short-term test.

It is to give diligence a live operating record to examine: a documented period in which the allocator can evaluate the strategy’s signals, execution, controls, exceptions, and decision-making in real time.

That record creates a more informed bridge between promising research and a larger capital allocation.