In-Sample vs Out-of-Sample: The Data Split That Exposes Overfitting
The fastest way to fool yourself in trading is to test a strategy only on the data you used to build it. The in-sample / out-of-sample split exists to stop that.
- 01 In-sample data is for building; out-of-sample data is held back to test honestly.
- 02 A large in-sample / out-of-sample gap is a sign of overfitting.
- 03 Reputable validation always reports the out-of-sample result.
- 04 TRION reserves out-of-sample windows so a tuned strategy can't masquerade as a real edge.
In-depth analysis
The two buckets
In-sample data is what you use to design and tune a strategy. Out-of-sample data is held back, untouched, and used only to check whether the strategy works on information it never saw. The gap between the two tells you how much of the performance was real edge versus curve-fitting.
A common split is to reserve the most recent 20-30% of your history as out-of-sample, build and optimize on the older 70-80%, then run the finished strategy once on the held-back portion. The exact ratio matters less than the discipline: the out-of-sample data must stay untouched while you tune.
A concrete example
Say you have ten years of OMXS30 data. You build and refine your rules on years 1-7 until the backtest looks strong. Then -- without changing anything -- you test on years 8-10. If the strategy still performs, that is meaningful evidence. If it falls apart, you just learned the edge was an illusion, for free, before risking a krona.
Why marketing skips it
A strategy tuned and tested on the same data almost always looks great. Showing the out-of-sample result is where most "profitable bots" quietly stop talking -- because that is where the impressive curve usually breaks.
The one rule you cannot break
Once you have looked at the out-of-sample result, you cannot tweak the strategy and retest on the same data. The moment you do, that data has effectively become in-sample -- it has informed your choices -- and its value as an honest test is gone. If you must iterate, you need a fresh untouched window.
How to use it
Reserve a meaningful out-of-sample window, never peek at it while tuning, judge the strategy on that window -- then forward-test on paper for a final, genuinely live check.
What TRION adds
TRION was built around an honest validation sequence rather than a promise. It is a paper-only research and validation workstation: you describe a strategy idea in plain English, read the compiled logic line by line, and backtest it against real stored market data. When a metric cannot be computed honestly, TRION shows "N/A" instead of inventing a number.
TRION does not place real orders, does not connect to a broker, and does not promise profit. The current beta is simulation-only and paper-only. AI assists with drafting and explanation; it does not approve, activate, or execute anything. Humans make every decision.
Frequently asked questions
What is a good in-sample / out-of-sample ratio?
Common splits reserve a meaningful chunk (for example, the most recent portion) as out-of-sample. The exact ratio matters less than never tuning on the held-out data.
What if out-of-sample results are much worse?
That's the strategy telling you it was overfit. It's a useful, honest signal — far better to learn it on paper than with real money.
Does TRION do out-of-sample testing?
Yes, it's central to how TRION validates. Beta is simulation-only — no live orders, no guarantees.
Sources & References
- [1] Out-of-sample data, explained — Investopedia
- [2] Backtesting: definition and how it works — Investopedia
- [3] Research and insights on investment analysis — CFA Institute
TRION is a simulation-only, paper-only research and validation workstation. It is not a broker, exchange, investment adviser, or live trading system, and it does not provide investment, financial, legal, or tax advice. Trading and investing involve substantial risk of loss. Backtests and simulations are based on historical data and assumptions and are not guarantees of future results. Reviewed by TRION Research.