Traps in Testing

Testing is a very critical thing for both discretionary and algo trading. As usual, the problems come from what you don't know that you don't know. As with most endeavors, if you don't even know what questions to ask then you will find it difficult to succeed.

One of the many traps in back testing is which combination of platform and data feed you are using, as well as both the latency between you and the data feed and you and your broker. Then add in the bandwidth of your internet connection and finally the load on your PC and the actual combination of these that any one trader has, is pretty unique. That means getting identical results from testing using the same trading rules is therefore pretty impossible.

The two tables below represent the results of a new algo we are developing. Both are EXACTLY the same easylanguage text testing the ES on the same PC located in the cloud, with the first running on MultiCharts and eSignal data and the second running on TradeStation and TradeStation data.

My first conclusion is that after I have finalised a configuration for Flo for a particular market, if anything changes then I have to re-configure. If I am in London it's different to the config I use in France. If I am running out of the cloud I must do a new config for different PCs.