Performance Metrics — Data Quality (Part 3 of 4)
Welcome
Hello and welcome to our twelfth article. This issue discusses the importance of Data Quality with reference to backtesting reports in MetaTrader 4 (MT4). All articles are saved at our Medium page and on the IQT website, which has a clickable version of the map of all articles: https://theiqt.com/blog.
These articles are based on my experience from consulting and product development at The IQT. Do let us know if there are topics you would like us to cover, questions you would like to resolve, or if there are insights you would like to share from your own experience. Hit that reply button!
Last Time
In Article 11 — “Performance Metrics — The Equity Curve and Drawdown (Part 2 of 4)”, we explored key metrics used to define and assess a strategy’s Equity Curve with reference to backtesting reports in MT4.
Overview
Data Quality is crucial as it underpins all inference drawn from performance analysis.
1) Data Quality is shown by the type of data model used, the modelling quality and the number of mismatched chart errors.
2) Assumptions over the values of spread and strategy parameters also impact inference on a strategy’s performance.
3) Finally, the larger the Data Quantity we have, the more confident we can be that the trading strategy is stable. Data Quantity is shown by the number of bars / candles, ticks, trading days and trades entered.
Main Points
This is the third article dealing with strategy performance metrics but arguably the most important; without good quality data we cannot be sure that our analysis or usage of performance metrics has a solid foundation. Below is the sample MT4 backtesting report from previous articles. We will use it to discuss the metrics related to Data Quality.
1) Data Quality
· “Modelling quality” in the MT4 report shows the overall quality of the data, which is 99% here. MT4 usually uses broker data which has a maximum quality level of 90% but can be much lower. Third party tools such as Tick Data Suite enable the use of data which has been cleaned (errors removed); data errors sometime arise in real time, and these are not always corrected by brokers later. Tick Data Suite was recommended to me in 2013 by one of the co-founders of Atom8, a broker which was later taken over by Vantage FX (https://www.forexbrokerz.com/news/Vantage-FX-acquires-Atom8-forex-broker).
· “Model” refers to the level of detail / granularity of the data. It is best to select “Every tick” so that the most detailed level of data is used. Note that even if “Every tick” is selected, the broker data may not contain tick data, so may instead use one-minute data to estimate (interpolate) the movement of price within each minute.
· As Birt from Tick Data Suite (TDS) says of MT4’s interpolation routine during backtesting, “if a position has both its SL and its TP within a bar’s price range, it’s a coin toss whether it will hit stop/loss or take profit.” This is why we recommend using TDS, as by ensuring you have tick data, which is the lowest level of data detail possible, there is no interpolation and you can have full confidence in your strategy’s results. This is especially important for higher frequency trading strategies which make use of short timeframes (less than hourly), such as the one-minute and thirty-minute timeframes.
· “Mismatched charts errors” refers to the number of inconsistencies between data from different timeframes, e.g. broker data may have hourly data which does not align with one-minute data. The report shows 0 such errors for this backtest.
2) Assumptions
· “Spread” is the level of spread (in points) assumed for the backtest. This can be set to a fixed level e.g. 10, or by choosing 0, the current level of spread in the market is used, as is the case here; the report says “Current (12)”. An unrealistically low choice of spread may incorrectly give the impression that a strategy is profitable, especially if it involves frequent transactions and small profit targets, as in scalping strategies.
· “Parameters” shows the input parameters to the trading strategy. The key ones in The IQT’s implementation of the MA crossover strategy are:
- general_percentage = 0.1: this sets the amount of funds to be risked per trade to be 0.1% of the account equity.
- general_max_trades = 1; this sets the maximum number of concurrent trades as 1
- general_slippage = 3; this allows slippage in price fills up to 3 pips
- short_period = 5; this sets the short / fast MA to use 5 periods’ data.
- long_period = 50; this sets the long / slow MA to use 50 periods’ data.
3) Data Quantity
The more data we use in a backtest, the more confident we can be that the strategy predictably generates returns. We can assess Data Quantity in MT4 reports using the following measures (the larger the better):
· “Period” (Δt in our Fundamental Equation of Trading) tells us the timeframe used, hourly here, and the total time period over which the strategy was tested, which is August — November 2021 (4 months). The number of trading days within this period = 86 (we calculated this number by omitting weekends).
· “Total trades” (N) is the total number of trading positions entered and exited, which is 55 here. The number of transactions used to enter, modify and exit the positions may be larger than this, so analysis of the list of trades is needed if we wish to investigate transaction costs.
· “Bars in test” is the number of bars / candles in the chart during the period, which is 2164 here.
· “Ticks modelled” is the number of ticks / transactions in the market during the period, which equals 4637600 here.
· The only caveat to the general rule that the larger these measures are, the better, is that regime changes may occur after certain market events. E.g. a trading strategy which assumes gold will always appreciate would not likely be viable if a cheap way to convert tin into gold were discovered, and increasing deposits of tin were found.
Further Reading
In the next article, we will discuss return (return) Distributions and the Sharpe Ratio.
Thanks and happy trading!
