Recently, a reader questioned the trading systems that are presented in my book and course.

I do not actually know whether they bought the book or the course or not but they emailed me to say that none of the trading systems presented work.

Well the first thing I will say is this:

As I mention many times throughout, these are trading system IDEAS. I did not create them for anyone to trade out of the box. Doing so would be lazy and foolish, since there are many more factors that come into play before you should trade a system live.

And do you really expect to make millions of dollars using a system that you have bought for what is in comparison, a minuscule amount of money?

The second thing I will say is this:

What have you done to prove they don’t work?

I am the first to admit, a few of the trading systems have struggled in recent years, but many are doing very well!

And the good thing with the trading systems in my book is that they were tested over ten years between 2000 and 2010. That means we now have four years of out-of-sample data with which we can run the systems and see how they did.

So I shall now take one of the trading systems and see how it’s held up over the last few years.

Trading System 20: Finding cheap stocks with linear regression and ATR

As I write in the book, Trading System 20 is a trend following system that utilises the linear regression indicator to find stocks that are trending upwards. It is a simple trend following system that favours lower priced stocks using a ranking mechanism based on ATR (average true range).

This is a portfolio type system that holds a basket of 12 stocks, weighted equally, and goes both long and short.

2000 – 2010 performance

In the book, performance was recorded as having a CAR of 20.42% with a maximum system drawdown of -21% and a recovery factor of 4.54. Remember that this test was run on S&P 500 stocks and did not include delisted securities.

2010 – 2014 performance

We can now test the system on the out-of-sample data between 2010 and 2014 to see what would have been achieved over this period. The results are shown below:

trading systems that work results tableequity curve first test

Compounded annual return was lower than the in-sample at 19.18% and this is to be expected. The maximum system drawdown was -10% and the system made 13.6% in 2011, 13.1% in 2012 and 41.4% in 2013.

Trading systems that work: stress-testing the systems

However, as mentioned already, the systems in the book are ideas so it pays to analyse them further so I shall now stress-test the system by taking different factors into account. First of all, I will test the system on data that includes delisted securities:

2010-2014 performance with delisted tickers

You can see now that incorporating delisted tickers (historical constituents) took the overall CAR down to 15.39% while the maximum drawdown went up to 17%. The system made -2.8% in 2011, 11.7% in 2012 and 49.5% in 2013.

delisted results tabledelisted results equity curve

Adding slippage

Since this trading system trades relatively cheap stocks it is necessary to add slippage since we will not always be able to trade on the open price as recorded by the data. So in this instance I will add a hefty measure of 50% slippage.

slippage results tableslippage equity curve

So now we can see that using 50% slippage, CAR went down to 12.28% and the maximum drawdown went up to -19.30%. The system made -5% in 2011, 8.8% in 2012 and 45.1% in 2013.

Moving start dates

Now we can see what happens if we move the start dates slightly to make sure the results are not a fluke.

move start dates results

By dropping the start date back by two months we get a slightly lower return of 11.52% and the maximum drawdown increases to -24%. We could keep going with this process, however, the system seems to perform well enough for now to continue.

Try a different market

Another thing we can do to stress-test the system is to try a different market or watch-list. For this test I will use the Russell 2000 Index. This includes historical constituents (delisted tickers).

russell 1000 resultsrussell 1000 equity curve

The system produced a CAR of 14.35% with a maximum drawdown of -13% and made -0.6% in 2011, 23.4% in 2012 and 35.1% in 2013.

Running Monte Carlo

We have performed a number of stress-tests so now we can also run some Monte Carlo analysis. In this test I will take the trade results and I use Monte Carlo to randomly alter the sequence of trades. This will be done using the software from Equity Monaco.

monte carlo straw bloommonte carlo percentile results

The Monte Carlo results indicated that 90% of all runs finished with a terminal equity of at least $12,000.

Conclusion

I have now performed a number of stress-tests in order to decide whether this trading system works. Personally, I am pretty happy with the results. The out-of-sample returns have been good, even after including delisted tickers and including slippage of 50%.

Can the system be improved?

From further tests that I have since run, I believe that the system can be improved by:

1. Reducing or turning off short positions completely.
2. Introducing a market timing filter and fundamental elements.
3. Increasing the total number of stocks held in the portfolio.

Does that mean you should trade this system?

Not necessarily. Trading is a personal affair and some traders would find this system difficult to follow. Furthermore, markets are dynamic and the future is never known. This system is now in the public domain and it could easily stop working.

The system has performed well but the last few years have been very good for trading stocks. It may well perform poorly over the next couple of years.

As always, traders should prefer to build their own trading systems in line with their own trading styles and risk preferences. Doing otherwise would be foolish.

I know from experience how difficult it is to produce a profitable, technical trading system. So you should only trade a system if you are 100% confident that you know exactly what you are doing.

Trading systems were back-tested using Amibroker and historical data was obtained from Premium Data. Monte Carlo was conducted with Equity Monaco.


 

Summary
Trading Systems That Work: Stress Testing Trading System 20
Article Name
Trading Systems That Work: Stress Testing Trading System 20
Description
Looks at a trading system from the How to Beat Wall Street course and asks the question 'does it work'? Puts the system through a number of stress tests in order to answer that question.
Author

Tags: , ,


4 opinions

    • Mike

    • November 7, 2014

    • 12:51 pm

    • Reply

    Hi Joe,

    It sure would be nice if you shared the code for the strategy in this instance. After all, you have written the piece in part to respond to accusations that the system may not work. If there is an error in the code then that could explain why you received the doubtful readers email.

    I’m particularly interested in your code for this system because when I test it I get a CAR of -5.46%!!

    I may be going wrong somewhere. Or maybe you are?

    • Hi Mike

      Not sure if this link will work but try:
      http://jbmarwood.com/wp-content/uploads/2014/11/20-Linear-Reg-ATR1.txt

        • Mike

        • November 8, 2014

        • 12:29 am

        • Reply

        Hi Joe,

        I tested the rules using my own code but as the results were so different to yours I decided to see how you do it.

        As it turned out, the recent code that you shared is different to the code-pack from the book anyway so I am glad that I asked.

        Thanks for sharing. I can see now why your results are so much better than mine.

        • No problem Mike, I’m glad you managed to get some decent results with it.

          The code is the same, I just added some buy/sell constraints to simulate slippage. Looks like the margin is different too which wasn’t intentional but shouldn’t make much difference.

          Cheers.

Leave a Reply

%d bloggers like this: