FreqAI Overfitting: How ML Trading Models Fool You
How overfitting appears in FreqAI and ML trading systems, from feature selection and model tuning to repeated strategy changes after seeing forward results.
Overfitting is larger than model fitting
The model can overfit, but so can feature selection, target design, retraining cadence, thresholds, exits and the decision to keep only successful experiments.
Validation must respect time
Randomly mixing future and past observations can make validation unrealistically easy. Trading evaluation needs chronological boundaries that resemble the intended deployment.
Repeated experimentation leaks information
If the same historical or forward period repeatedly influences design decisions, it gradually becomes part of the training process even when it is called a test set.
Keep an untouched forward record
Versioning and preserved PAPER outcomes make it harder to erase inconvenient evidence. The objective is not a perfect historical curve; it is a system whose behavior remains useful on unseen data.
A practical validation checklist
Keep chronological boundaries, record the exact feature/target/model version, separate model metrics from trading metrics, and evaluate the frozen system on unseen forward data. A change made after observing the result belongs to a new experiment.