Curve Fitting: Optimizing Your Way to Failure

Listen, I’ve been trading for twenty-three years, and I’ve seen enough blown accounts to fill Canary Wharf. But nothing—and I mean nothing—compares to the special kind of self-sabotage that is curve fitting. It’s the financial equivalent of that mate who thinks he’s figured out how to cheat the casino, only to realize he’s just invented a very expensive way to lose money more efficiently.

Let me tell you a story.

The Optimisation Trap

Five years ago, I watched a kid—couldn’t have been more than twenty-four, fresh out of uni with a first-class degree in maths—set up shop in a WeWork in Shoreditch. Smart lad, genuinely. He’d built a trading bot using some fancy machine learning algorithm. Spent three months optimising the parameters on historical data. Three. Months.

When he showed me the results, I nearly spat out my coffee. A 47% annual return. Zero drawdown. Only three losing trades in a two-year backtest. It was the financial equivalent of a Megan Fox Instagram filter—absolutely perfect and completely divorced from reality.

“How much is in the account?” I asked.

“Thirty grand,” he said, grinning like he’d already bought the Porsche.

Two weeks live. Two weeks. He’d blown through £28,000 before I even got the chance to say, “I told you so.”

That’s curve fitting. That’s optimisation. That’s the beautiful tragedy of backtesting.

What’s Actually Happening?

Here’s the thing about historical data: it’s dead. It’s a corpse. It doesn’t move, it doesn’t surprise you, and it certainly doesn’t care about your ego. When you sit down at your computer and start tweaking your strategy’s parameters—moving your stop loss by five pips here, adjusting your take profit there, adding a filter for when the moon is in Pisces or whatever—you’re not building a trading system. You’re building a lie.

You’re using a process called curve fitting, and it’s insidious because it feels legitimate. You’re using real data. Real price movements. Real outcomes. But you’re fitting your strategy so precisely to that historical data that you’ve essentially memorised the exam answers instead of learning the material. The moment the market serves you a question it hasn’t asked before—which it does every single day—you’re screwed.

The technical term is overfitting. In layman’s terms, it’s when your system is so perfectly tailored to the past that it becomes completely useless in the future. You’ve optimised yourself into a corner so tight that the only way out is through your trading account equity.

The Statistics Don’t Lie (But Your Backtest Does)

Let me break down how this actually works. Say you’ve got a moving average crossover strategy. Simple, right? Two lines on a chart.

Now, you’ve got about 50 parameters you could possibly tweak:

• MA period 1: anywhere from 5 to 200
• MA period 2: anywhere from 10 to 500
• Entry filter (do you need another confirmation?)
• Exit condition (fixed TP? Trailing stop? Random tea leaves?)
• Time of day (only trade during London session? Asian session? When you’ve had your morning coffee?)
• Risk per trade
• Maximum drawdown tolerance
• Whether Mercury is in retrograde

That’s roughly 50 million possible combinations.

Run those through your backtester long enough, and statistically speaking, something will look phenomenal. It’s the lottery paradox—if you buy enough tickets, one of them will eventually win. Except in this case, you’re buying millions of tickets to a game where the odds never changed. You’re just finding the lucky ticket from last year’s draw.

This is called the multiple comparisons problem, and it’s why every retail trader with MetaTrader and too much time on their hands thinks they’ve found the Holy Grail.

The Walk-Forward Validation Gambit

Now, some of you clever dicks are going to say, “Right, but I’m using out-of-sample testing. I backtest on 2023 data and validate on 2024 data. Checkmate.”

Adorable. But no.

Walk-forward analysis helps—sure, I’ll give you that. It reduces the problem. But it doesn’t eliminate it. Why? Because you’re still testing on historical data. You’re still optimising to conditions that have already occurred. And the moment your strategy hits live market conditions—volatility you’ve never seen, correlations that break, news events that make your indicators meaningless—your beautifully optimised system is about as useful as a chocolate teapot.

I once watched a trader backtest a GBP/USD strategy on five years of data. Gorgeous results. Then the Bank of England had an unexpected policy announcement, volatility spiked 400%, and his “low-risk” strategy exploded his account in sixteen minutes. The data had never seen that volatility profile before. The system had never been tested against it. So when it happened, the strategy didn’t adapt—it just died.

The Human Cost

What bothers me most about curve fitting isn’t the money lost (though Christ, it’s a lot). It’s the opportunity cost and the psychological damage.

You spend months optimising, backtesting, refining. You build an emotional attachment to your strategy. You convince yourself it’s brilliant. You tell your mates about it. You fantasise about the yacht. Then live trading begins, and reality doesn’t cooperate. Your “perfect” system starts losing. Your confidence evaporates. You either blow the account or—worse—you abandon trading altogether, telling yourself you’re not cut out for it.

You were never not cut out for it. You were just cut out for backtesting.

What Actually Works

So what’s the alternative? How do you build a system that doesn’t immediately die the moment it meets the real market?

Simplicity. Fewer parameters mean fewer opportunities to overfit. An MA crossover might not sound sexy, but if it works consistently across multiple timeframes and instruments, you’ve got something real.

Robust testing. Test across different market regimes. Test on data your strategy has never seen. Test on different instruments entirely. If it breaks immediately, it was curve-fitted.

risk management that doesn’t rely on optimization. Your position sizing, stop losses, and profit targets shouldn’t be fine-tuned to the decimal point. They should be robust.

Accept mediocrity in backtests. If your backtest shows 15% annual returns with reasonable drawdowns, that’s probably real. If it shows 47%, it’s probably fiction.

And here’s the hardest part: respect the market’s randomness. You can’t predict it perfectly. Anyone who thinks they can is just a curve-fitted strategy away from a margin call.

The Bottom Line

Curve fitting is seductive because it promises certainty in an uncertain world. It whispers that if you just tweak one more thing, you’ll crack the code. But the code doesn’t exist. Markets are dynamic, adaptive organisms that don’t give a toss about your optimised parameters.

The best traders I know don’t hunt for the perfect strategy. They hunt for principles that work most of the time and risk management that protects them when those principles don’t.

Your backtester isn’t a crystal ball. It’s a historical simulator. Treat it as such, and you might actually make it to your second year of trading.

Treat it as a promise, and you’ll be another cautionary tale we cynical veterans tell at the bar.

Your choice, mate.