Many merchants like renko and different “timeless” charts for his or her readability and worth noise discount. The charts are used not just for technical evaluation however for constructing buying and selling methods on high of them. And so they require a correct backtesting. Nonetheless, not many merchants notice that Renko backtesting could also be (and normally is) extremely misleading resulting from specificities of the chart and platforms that present it.
On this publish we’ll deal with the pitfalls of renko realization in MetaTrader 5 and methods to overcome a few of them. However earlier than we start and for the sake of brevity allow us to remind you of earlier publications on this matter.
The story begins with the introduction of the Renko chart generator – RenkoCharts – it is technical preview (out of date) first appeared on this weblog as properly. It builds renko from quotes/charges and don’t think about precise worth actions inside each bar. If the field dimension is smaller than a spread of particular bar, this will result in a simplified, unrealistic formation of renko containers on historical past. Because of this, backtests can run smoother than in actuality. Please notice that, when on-line, the generator works by incoming ticks, so the standard of renko is unaffected, however not everybody can afford to gather ticks for months on-line to construct a devoted renko (although it might be unfeasible resulting from possible interruptions resulting from bugs, connection losses).
The plain answer of the issue is to construct renko on the historical past from ticks as an alternative of quotes. In fact, that is doable provided that the tick historical past is offered.
Let’s pay attention to the issue and its answer:
A. Keep away from constructing renko from common timeframe bars, even M1, and use ticks if doable
This method is applied in one other product – RenkoFromRealTicks.
It supplies higher high quality than the previous generator, however does so on the expense of upper assets utilization (disk area, bandwidth, CPU load) and execution instances.
And guess what? It doesn’t clear up all the issues with renko accuracy.
One in every of them arises from the truth that implementation of renko charts in MetaTrader 5 is predicated on {custom} symbols. So far as they’re generated regionally in your PC and have distinctive names totally different to the names of unique symbols supplied by dealer, it is unattainable to commerce renko symbols on-line. Satirically, we are able to commerce {custom} symbols within the tester simply. However this might be theoretical backtests, which cannot be mapped to actual buying and selling with out extra coding and don’t assure precisely the identical (good) outcomes.
To satisfy renko charts and unique symbols your robotic ought to problem orders for unique symbols by alerts detected on corresponding renkos. There’s a devoted blogpost describing some strategies of Adapting EA robots to buying and selling on {custom} renko charts in MetaTrader 5Â and high quality metrics of their backtests.
In truth, such robots are multi-symbol ones and will deal with synchronization between not less than 2 tickers: {custom} and unique. Mainly, it is best to detect formation of renko containers, however this isn’t so trivial as chances are you’ll assume. Nuances exist relying from buying and selling library you utilize (for instance, commonplace library) and concerned instruments (akin to indicators or charges evaluation immediately, any of which depends on bars completion). We’ll talk about this matter a bit later, in part D.
Now simply pay attention to the duty:
B. Guarantee cross-symbol synchronization
Some consideration on constructing extra dependable renko-driven checks have been made in one other blogpost – Buying and selling on renko charts constructed from actual ticks.
At these stage there have been nonetheless 3 important issues – their frequent impact may be described as peeking into the long run, which produces unrealistically good buying and selling outcomes:
- a niche in entrance of reversal field;
- each field is M1 bar aligned by 1 minute boundary, whereas its precise formation (both of open time and finish time) occur at some moments inside close by minutes;
- quick and outstanding worth actions might produce containers forward of actual time, as a result of they’re M1 bars (bars cannot be positioned extra tightly to one another than as soon as each 1 minute);
Allow us to think about the clauses extra intently, ranging from the 1-st.
Do not forget that certainly one of doable approaches to adapt EA to work on a {custom} image is to put EA on the chart (renko in our case, which is used for synchronization and technical evaluation) however redirect buying and selling orders to unique image. Usually, all renko alerts are taken from closed containers (as a result of the right-most forming field can find yourself in any course, and even its “open” worth). And each reversal field is began with the hole, as a result of the tester is able to simulating common bars solely. In actuality, there isn’t any such hole between the shut worth of the earlier field and the open worth of the subsequent field. It is only a conference of drawing reversal renko field with the hole.
Execution of trades on 100pt renko chart of EURUSD with synthetic ticks or no ticks (left) vs actual ticks (proper). Notice how pretend gaps on the left chart decrease losses compared to the best chart the place there aren’t any gaps.
As you may even see, if a buying and selling sign is occured in the mean time when reversal field is began, the deal worth is shifted from the true worth for that second by the field dimension (in common). Normally this unfair correction goes in favour of the dealer – a dropping commerce turns into much less unprofitable, or worthwhile commerce will increase.
In case your EA is positioned on a chart with unique image and trades it immediately, acquiring alerts from containers and/or indicators on renko image, then you definately remove the 1-st downside. But when it is extra appropriate so that you can place EA on renko (for instance, EA is constructed so by a developer and cannot be tailored), we’ll present another answer beneath.
In fact, the gaps are there in standard renko solely. As chances are you’ll know, there are extra sorts of renko, which can not require the hole. For instance, certainly one of particular renkos is the gapless renko (rail-to-rail), which can be supported by RenkoFromRealTicks. As standard renkos are extra frequent, most of buying and selling alerts are designed with standard shapes in thoughts. For this reason a nonconventional renko require to re-think buying and selling alerts and re-implement EA for nonconventional renko. We’ll not think about these variations right here.
Now allow us to think about the 2-nd case.
The second when a worth crosses a boundary for subsequent field completion is random. It lies someplace between two consecutive 1 minute marks. When it occurs we create a brand new field with a timestamp rounded as much as nearest entire minute. The actual worth at this precise minute (with out seconds and its fractions) was not the identical as the value in the mean time when the earlier field was closed and the brand new field was began. The imply of this distinction is a half of M1 bar vary in common. Once more, it signifies that trades are opened and closed by extra useful worth than in actuality, prematurely, so to talk. Chances are you’ll give it some thought as if renko chart is at all times nearly shifted half-minute forward of time.
Execution of trades on common EURUSD chart pushed by sign from 100pt renko with synthetic ticks or no ticks (left) vs actual ticks (proper). The sign is shaped by upward field accomplished at 1.08100 on 16:05 M1 bar. The left purchase is executed too early (at bar opening, therefore with too low worth and surprising bonus), whereas the best purchase is executed at actual time 16:05:46 and actual worth.
The three-rd downside may be eradicated by selecting sufficiently huge field dimension that point overflows don’t occur (not at all times doable for devices with massive adjustments in volatility), however overwise it’s sadly unsolvable by canonical charts, and requires to invent some methods (mendacity exterior the scope). It does not essentially imply that all of “run-through” containers results in a displaced sign, however a few of them may be mishandled.
Execution of the identical alerts on overflown (wanting forward) containers when EA is positioned on common M1 chart (left) vs renko (proper). Large draw back transfer round 13:30 M1 was mapped into a number of renko containers with “future” time, the place 2 MAs crossing sign produced at synthetic 13:35. When that sign is handed to EA operating on common M1 chart, 13:35 bar stands at earlier second, in consequence the promote is executed prematurely with greater (higher) worth.
One method to probe right here is to run EA solely on renko chart (when within the tester). So far as EA takes alerts from newest accomplished field, it is unimportant that the time marks of the containers might go forward of actual time.
Fortunately the 1-st and 2-nd issues talked about above may be mounted by the next method:
C. Generate actual ticks for renko containers
Do not forget that bars (containers) are simply containers for ticks, so an M1 bar does not have to start out with a tick at 0-th second or find yourself with a tick at 59-th second. As we create our {custom} image, we are able to generate ticks for renko field with precise timestamp when the field was open. The primary tick will include precise precise worth as properly. All different ticks will even include actual costs, however we have to modify their time to the identical timestamp so as to maintain them contained in the field. If we protect actual time to the renko ticks, the tester and the terminal would type extreme bars (in response to they “understanding”), which break correct renko formations.
With such “actual” ticks inside renko containers we are able to run checks primarily based on actual ticks for renko symbols and execute trades with precise costs. No synthetic gaps on reversals, no parasitic half-M1 deviations.
So okay, we’d like the true ticks. The truth that RenkoFromRealTicks builds containers from actual ticks doesn’t imply mechanically that there are ticks contained in the containers. Till model 1.4 RenkoFromRealTicks didn’t generate ticks for the containers on historical past.
Now it could actually.
New parameter TickHistoryMode lets you select one of many tick era modes (so as of accelerating accuracy on the expence of useful resource consumption):
- Single tick per field
- OHLC ticks per field
- Bulk ticks per field
The primary mode is sort of the identical because it was earlier than. It generates a single tick per field in historical past. The distinction is that new variations maintain timing with seconds and milliseconds (the outdated ones used M1 time).
The second mode generates a number of ticks per field, normally 2 or 3 (relying from whether or not a field has a wick), which mimics OHLC attribute costs of a bar (therefore the identify), with 1 extra heat up tick.
The third mode generates all ticks from unique image.
Chances are you’ll select an optimum mode in response to your buying and selling algorithm and assets.
When OHLC or bulk ticks modes are enabled, the ensuing renko image helps testing with the settings “each tick primarily based on actual ticks”. It is advisable to verify tester log to make it possible for the true ticks are in impact. For instance, if the primary image, chosen within the tester, has tick historical past, the start of the log incorporates some strains about ticks synchronization (downloading them from the terminal, if it is 1-st run for that image, or about synchronized already ticks, in any other case).
A number of strains beneath there needs to be particular line:
EURUSD_r100,M1 (Your Dealer): producing primarily based on actual ticks
In case you use a number of renko symbols, all of them must also be talked about as having actual ticks:
... EURUSD_T_r100: ticks synchronized already [57 bytes] EURUSD_T_r200: ticks synchronized already [57 bytes] EURUSD_T_r300: ticks synchronized already [57 bytes] ... EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00 EURUSD_T_r200 : actual ticks start from 2023.01.02 00:00:00 EURUSD_T_r300 : actual ticks start from 2023.01.02 00:00:00 ...
Ensure to generate renko historical past with not less than 6-12 months reserve forward of deliberate beginning date of testing. The dates within the log excerpt above is chosen by the tester in response to the settings to check from starting of 2023 12 months. The renko symbols have been generated because the starting of 2022.
The tester logs this data just for the primary tester image, for instance:
EURUSD_r100: historical past information begins from 2022.01.02 00:00 EURUSD_r100: ticks information begins from 2022.01.02 00:00
For all extra symbols, loaded out of your EA ad-hoc (by way of CopyRates and so forth) and added to the tester’s market watch, this data shouldn’t be supplied within the log (I do not know why).
In case you use different {custom} image mills for comparability or cross-validation, please, control additional messages in logs. Doubtlessly, even when actual ticks are enabled, there may be issues with tick historical past on some bars. RenkoFromRealTicks doesn’t enable this, however different instruments might do.
Right here is an instance of a {custom} image with 17 problematic bars:
EURUSD_T_r100 : 2023.01.02 23:59 - actual ticks absent for 1 minutes out of 16 whole minute bars inside a day EURUSD_T_r100 : 2023.01.12 23:59 - actual ticks absent for 4 minutes out of 51 whole minute bars inside a day EURUSD_T_r100 : 2023.02.03 23:59 - actual ticks absent for 3 minutes out of 37 whole minute bars inside a day EURUSD_T_r100 : 2023.03.12 23:59 - actual ticks absent for 2 minutes out of 8 whole minute bars inside a day EURUSD_T_r100 : 2023.03.22 23:59 - actual ticks absent for 1 minutes out of 22 whole minute bars inside a day EURUSD_T_r100 : 2023.05.05 23:59 - actual ticks absent for 1 minutes out of 14 whole minute bars inside a day EURUSD_T_r100 : 2023.07.07 23:59 - actual ticks absent for 2 minutes out of 18 whole minute bars inside a day EURUSD_T_r100 : 2023.09.20 23:59 - actual ticks absent for 1 minutes out of 15 whole minute bars inside a day EURUSD_T_r100 : 2023.10.08 23:59 - actual ticks absent for 1 minutes out of 3 whole minute bars inside a day EURUSD_T_r100 : 2023.11.14 23:59 - actual ticks absent for 1 minutes out of 18 whole minute bars inside a day EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00 EURUSD_T_r100 : 2023.01.01 00:00 - 2024.02.01 00:00Â Â actual ticks absent for 17 minutes of 3230 whole minute bars, each tick era used
For these bars the tester will generate synthetic ticks that are incompatible with renko and should have an effect on accuracy.
One necessary factor to recollect: the tester retains its personal native cache of all symbols utilized in checks. In case you recreate {custom} image with altered properties (together with full removing and constructing from scratch) the terminal ought to usually present new historical past to the tester, or in different phrases, the tester ought to detect the adjustments and obtain the historical past anew. Sadly, it does not work so on the time of writing, so you might want to clear up the tester cache manually should you plan to alter {custom} image. The cache is positioned within the following 2 folders for charges and ticks, correspondingly:
/Tester/bases/<dealer>/historical past/<custom-symbol>/ /Tester/bases/<dealer>/ticks/<custom-symbol>/
One might imagine that while you delete a {custom} image from the terminal, the image is mechanically deleted from the tester cache as properly, nevertheless it’s not.
D. Implementation
Because it’s talked about earlier, a multi-symbol robotic wants synchronization between symbols. The alerts of renko is sensible just for accomplished bars, therefore we have to verify bar completion (or initiation of a brand new bar, if you want). As you understand, MetaTrader 5 is an asynchronous factor. The incoming ticks are processed by the platform and collected as bars, however it might occur with a little bit delay. In different phrases, your EA can obtain new tick marked by subsequent minute, however corresponding M1 bar shouldn’t be but accessible (continues to be being constructed). Because of this, indices utilized in expressions like iClose(…,0), iClose(…,1), CopyBuffer(…,1) nonetheless returns values from the previous (virtually outdated, about to be shifted) bars.
Normally this unsychronized state lasts not more than 1 tick, if it occurs in any respect. But it surely’s ample to get incorrect alerts from outdated bars and run into losses.
Sadly, this facet shouldn’t be taken into consideration in the usual library (should you use MQL5/Embrace/Knowledgeable) and likewise might not correctly dealt with in different supply libraries.
For instance, allow us to look into the tactic CExpert::Refresh which known as upon new ticks.
bool CExpert::Refresh(void) { Â Â MqlDateTime time; Â Â if(!m_symbol.RefreshRates()) Â Â Â Â Â Â return(false); Â Â Â Â TimeToStruct(m_symbol.Time(), time); Â Â if(m_period_flags != WRONG_VALUE && m_period_flags != 0) Â Â Â Â Â Â if((m_period_flags & TimeframesFlags(time)) == 0) Â Â Â Â Â Â Â Â return(false); Â Â m_last_tick_time = time; Â Â m_indicators.Refresh(); Â Â return(true); }
RefreshRates of m_symbol calls SymbolInfoTick internally, and the time from the tick is used as a latch for not processing the bars twice. For that function the tick is theoretically mapped in all timeframes (calculated in TimeframesFlags). If the tick advances on one of many timeframes, the code shops m_last_tick_time and refresh indicators. In different instances, the refresh is skipped. However as we already know, the brand new tick may be not but propagated to bars, so the refresh (which is definitely equal of required CopyBuffer’s) shops the outdated information, which is about to alter in a fraction of a second. But when the charges are up to date, and indexing is shifted, the latch doesn’t enable to refresh anymore (as a result of subsequent ticks have instances throughout the theoretical bar, detected on the 1-st tick).
For this reason we have to apply some patches to the usual library or do related stuff in our supply codes.
Right here is an instance, the way it might be performed in an professional adviser, primarily based on the usual library.
class SyncSymbol { Â Â protected: Â Â Â Â const string image; Â Â Â Â const ENUM_TIMEFRAMES timeframe; Â Â Â Â const string interval; Â Â Â Â datetime lastUnsync; Â Â Â Â lengthy countUnsync; Â Â Â Â Â Â public: Â Â Â Â SyncSymbol(const string s = NULL, const ENUM_TIMEFRAMES tf = PERIOD_CURRENT): Â Â Â Â Â Â image(!StringLen(s) ? _Symbol : s), timeframe(tf == PERIOD_CURRENT ? _Period : tf), Â Â Â Â Â Â interval(StringSubstr(EnumToString(timeframe), StringLen("PERIOD_"))) Â Â Â Â Â Â { Â Â Â Â Â Â } Â Â Â Â Â Â Â Â bool isReady() Â Â Â Â { Â Â Â Â Â Â MqlTick t; Â Â Â Â Â Â SymbolInfoTick(image, t); Â Â Â Â Â Â const string now = StringFormat("%s'%03d", TimeToString(t.time, TIME_SECONDS), t.time_msc % 1000); Â Â Â Â Â Â if(!SeriesInfoInteger(image, timeframe, SERIES_SYNCHRONIZED)) Â Â Â Â Â Â { Â Â Â Â Â Â Â Â PrintFormat("%s wait unsync %s %s", now, image, interval); Â Â Â Â Â Â Â Â return false; Â Â Â Â Â Â } Â Â Â Â Â Â Â Â Â Â Â Â if(t.time / 60 * 60 != iTime(image, timeframe, 0)) Â Â Â Â Â Â { Â Â Â Â Â Â Â Â PrintFormat("%s wait unsync with %s on %s %s", now, (string)iTime(image, timeframe, 0), image, interval); Â Â Â Â Â Â Â Â lastUnsync = t.time / 60 * 60; Â Â Â Â Â Â Â Â countUnsync++; Â Â Â Â Â Â Â Â return false; Â Â Â Â Â Â } Â Â Â Â Â Â Â Â Â Â Â Â if(lastUnsync == t.time / 60 * 60 && countUnsync) Â Â Â Â Â Â { Â Â Â Â Â Â Â Â PrintFormat("Synced after %ld ticks on %s %s", countUnsync, image, interval); Â Â Â Â Â Â Â Â countUnsync = 0; Â Â Â Â Â Â } Â Â Â Â Â Â return true; Â Â Â Â } }; Â Â void OnTick() { Â Â static SyncSymbol image; Â Â Â Â if(!image.isReady()) return; Â Â Â Â ExtExpert.OnTick(); }
This demonstrates check-up of present _Symbol solely. For multi-symbol EAs it is a good suggestion to verify all symbols in the identical method.
In case of unsynchronization the item will output one thing like this:
2023.01.03 10:25:53Â Â 10:25:53'645 wait unsync with 2023.01.03 09:42:00 on EURUSD_r100 M1 2023.01.03 10:25:53Â Â Synced after 1 ticks on EURUSD_r100 M1
Chances are you’ll discover a new revision of the check EA MA2Cross2.mq5 and its sign module connected to the publish. Additionally a few necessary patches for ExpertSignal.mqh and ExpertTrade.mqh are included.
And listed here are some outcomes achieved with this EA backtested in numerous modes.
Tester report for buying and selling EURUSD by artificially generated ticks in EURUSD 100pt renko, equal of finest accuracy supplied by former variations of RenkoFromRealTicks (pre-1.4)
Tester report for buying and selling EURUSD by actual ticks in EURUSD 100pt renko, with improved accuracy
Tester report for buying and selling immediately on EURUSD 100pt renko by actual ticks, which demonstrates related outcomes to earlier one and is appropriate for validation at smaller useful resource footprint
You’ll be able to check EA in numerous modes and count on on-line buying and selling efficiency to be close to the decrease certain amongst all outcomes.
F. Remaining notes
Visualization of the renko within the tester can look unusual – containers could also be smaller or bigger than predefined dimension. It could possibly’t be mounted – that is how the tester works.
Allow us to make clear. Renko containers/bars are correctly formed by the generator in static historical past and correctly re-shaped on-the-fly on-line. When the ticks are “performed” within the visible tester, the generator is uncontrolled, and the bars are shaped by ticks solely. For this reason they give the impression of being unusual.
On this regard, it is necessary to notice that the attribute worth of the renko field is its shut worth. So, should you use circumstances for checking alerts, as an alternative of shut(i) vs open(i) think about using shut(i) vs shut(i+1). Open(i) may be someplace in a spread of 0-2 field sizes from shut(i) and even equal to shut(i), if i-th bar is a single tick bar drawn throughout huge bounce of worth.
If you are going to use indicators (like MAs) utilized to renko, additionally setup them for shut costs solely. Although highs and lows must also be dependable.
The same issues of exact backtesting pertain to different particular charts which remove a time in a way that bars aren’t alined by time scale however shaped upon information pushed circumstances, for instance, by quantity restrict. Additionally a lot of such charts have gaps in entrance of reversal bars, for instance Level And Figures.
