Is Backtesting Effective?

It worked last time, so it should work again or so the theory goes…

But that’s exactly what’s behind backtesting. Trading aside, repeating what’s worked in the past makes total sense as a starting point in most ventures. If you knew a marketing campaign had worked well in the past, you’d probably be thinking about using those same parameters when you started a new one.

In trading terms, where huge sums of money are involved, it makes perfect sense to mitigate your risks by testing. Or backtesting.

Backtesting looks at the historical performance of stocks to understand how they might perform if you were to invest in them today.

A trader would hypothesize a strategy and then backtest it using the historical performance data to determine if that strategy could be profitable as an investment.

As with anything of a speculative nature, there are no guarantees, but backtesting strategies allows traders to make informed guesses about the viability of a stock’s performance.

Simplified, if a stock’s performance did well in backtesting it could likely perform well as an investment today. Equally, if a strategy tests badly, it gives investors the opportunity to think again without risking their capital.

Backtesting can also help an investor see what could happen during unexpected events, such as a financial crisis or other unexpected market downturn and thus create a plan around how much to invest.

What’s important in backtesting?

In coming to a backtesting strategy there are certain factors that are important.  You need to consider the market you want to trade in, the programme language, the data sets and a plan.

Calling it a plan might be oversimplified, but the logic behind what and why you’re backtesting is the most important thing. The best quantitative strategy books suggest that you need to know what you are looking for. And while many people might publish a successful strategy, success lies in the detail. It’s the exact parameters that matter here. And it’s crucial.

Choosing the right market is perhaps better described as choosing the right one for your investment expectations. Certain markets might bring better returns but be high risk, others might bring returns over a long period of time. But picking the wrong market for your investment needs means you won’t see the returns you want.

A programming language is a matter of personal choice, although some do lend themselves to speed better than others, if speed is an issue for your portfolio. Typically investors might use Python, R, C++ or MATLAB, each of which has its own strengths or weaknesses.

Choose high-quality data for your backtesting. It’s important that the data is good so you can get accurate test results.

How far to backtest?

There are various arguments about this. Some people will look back 10-15 years, depending on the type of trading they are doing (for example those looking at intraday trading might backtest for something like 10 years for anything longer it might be 15 years).

However, others think this is too long a time frame and the difference in the market of 15 years ago is too much to be able to get accurate results in the present.

For example, if the holding period is between one and thirty days then you might want to go back around three years.

The longer the holding period the longer the testing. But even with very short holding periods – nanoseconds for example – you should look to test for at least six months.

Final thoughts

There are no guarantees, but backtesting allows you to test before committing real money and helps traders mitigate the risks of investing by giving them information. It’s a vital part of any quant strategy.