Choose the split beforehand
For time-dependent data, preserve chronology and define the training, selection and final evaluation periods in advance. Fit preprocessing only on the permitted training observations. Overlapping labels may need gaps between periods.
Keep the final period held out
If you repeatedly change a strategy after reading its hold-out result, that period is influencing selection. Record those changes and seek a new independent evaluation. A renamed “test set” does not undo information already used.
Report limitations with results
Compare periods using the same costs and measurement conventions. Explain limited sample size and unusual market conditions. Forward simulation can add later observations, but simulated fills still differ from actual execution.
Sources & references
- scikit-learn — Cross-validation and time series evaluation ↗
- Bailey et al. — The Probability of Backtest Overfitting (2015) ↗
References support specific definitions or methodological context. Examples and research checklists are explanatory; no performance result is claimed.