TickCompressor - MetaTrader 5 Script | Trading Script Download - MT4/MT5 Resources

In MQL, quotes are stored in a special structure - MqlTick. It's versatile, but a little fat. If your algorithm handles large tick arrays, this may be a source of RAM shortage. The TickCompressor class provides a simple solution to minimize memory utilization.

Instead of the standard structure MqlTick, it provides the use of 2 new data types: MqlTickBidAsk for FX/CFDs and MqlTickBidAskLastVolume for exchanges, as well as a series of methods to convert back and forth between them.

Let's compare the standard MqlTick and mini-tick structures.

 Structure MQTick { 
datetime time;
Double the bid;
double ask;
double final;
Oolong volume;
long time_msc;
unit flag;
Double the actual volume;
};

There are too many 'time' fields because it is a duplicate of 'time_msc', but with less precision.

In fact, the price field does not have to be double. For most symbols, the float type is sufficient. When it comes to price calculations, double is indeed a must, but as long as it's a question of passive storage, float will do the job.

In addition, for instruments traded on centralized markets (not exchanges), the "last" price is always 0. Additionally, "volume" and "volume_real" are not used.

Finally, the "flags" field can easily be put into ushort.

Therefore, MqlTickBidAsk for Forex and CFDs looks quite streamlined.

 Structure MqlTick bid price {
float bid;
float ask;
long time_msc;
ultra short flag;
};

MqlTickBidAskLastVolume is more verbose, but still compact.

 StructureMqlTickBidAskLastVolume
{
float bid;
float ask;
float final;
Oolong volume;
long time_msc;
ultra short flag;
};

Actually in both cases we can use "ushort spread" instead of "float Ask" and build "ask" from "bid" but it remains as an option.

The test script outputs original and reduced ticks to the log.

For GBPUSD it uses MqlTickBidAsk and for S&P 500 Mini Futures it uses MqlTickBidAskLastVolume.

As you can see, MqlTickBidAsk is 3 times smaller than MqlTick and MqlTickBidAskLastVolume is 2 times smaller.

One of the areas where the tick compressor can help is with virtual testing and optimization using tick arrays. For example, Library Virtual provides methods for virtual transactions:

 static integer tester( const MQTick &Tick(&T)[], ...)

where the Ticks array should be provided in memory by the calling code.

If a strategy is to be tested over a long history (several years), the tick array can easily grow to many Gb and exceed physical RAM. In this case, one can store a reduced array (compressed ticks) and then patch the virtual library to consume the mini-ticks via decompression, keeping all internal logic intact (in the same way as standard ticks).

#ifdef TICK_COMPRESSOR
Template < type name TC>
static integer tester TC( const TC&Ticks[], const STRATEGY, const DBL_MIN = DBL_MIN , const boolean STOP= True , const boolean bNETTING= False )
  {
constant integer handle = VIRTUAL::GetHandle();
const boolean resource =:: array_size (ticks) && VIRTUAL::SelectByHandle(VIRTUAL::Create(balance, (scale[ 0 ].time_msc/ 1000 /( 24 * 3600 )) *( 24 * 3600 ), bnetting));

if (research)
    {
      VIRTUAL::NewTickTC(quote, strategy);

if (stop)
        virtual::stop();
    }

Return (resource? handle: - 1 );;
  }
 
Template < type name TC>
static blank NewTickTC ( const TC &Ticks[], const strategy strategy = invalid )
  {
if (virtual::select order)
    {
const int size =:: array_size (ticks);
 
MQTick result;

if (strategy)
      {
for ( integer i = 0 ; (i < size) && (!:: stopped ());i++)
        {
          TickCompressor::Decompress(Ticks[i], result);
          VIRTUAL::SelectOrders.NewTick(result);

          strategy();
        }
      }
other {
for ( integer i = 0 ; i < size; i++)
        {
          TickCompressor::Decompress(Ticks[i], result);
          VIRTUAL::SelectOrders.NewTick(result);
        }
      }
    }
  }
#endif

Additionally, a tick compressor can be used to store tick arrays in a file in a more compact way.

Attachment download

📎 tickcompressor.mq5 (1.64 KB)

📎 tickcompressor.mqh (4.61 KB)

Source: MQL5 #30791