TickCompressor - compresses 1 tick to 2-3 bytes on average - MetaTrader 5 library

Compressed quote data is stored in a compact form that is 3.5 times more compact than .tcs MQ files. To use them quickly, because reading 3 bytes takes less time than reading a 60-byte MqlTick structure.

The file size for 2023 can be seen in the screenshot, including ask, bid, time, and additional ZIP compression of data blocks:

File size in .tcs format in 2023:

3.56x compression.

To store price changes, the difference between the ask and bid prices and the previous price is used. Typically (up to 50...70% of all scales) it does not exceed (-8...7) points and can be recorded with 4 bits. Ask and Bid are combined into 1 byte.
Add 1 byte to store the time difference from 0 to 255 milliseconds (in codes up to 229, values ​​above 229 are used to encode scales beyond -8...7 points).

If the price or time differs by a large value, they will be packed into a larger number of bytes.

For additional compression, you can apply a ZIP archive. Data size is reduced by up to 2x.

Alternatively, it can also be compressed into 3 bytes, and Ask and Bid from -129 to 128 can be compressed into 8 bits or 1 byte respectively. Plus 1 byte for the time - a total of 3 bytes for most ticks.
Sometimes (https://www.mql5.com/ru/forum/499639/page6#comment_58544810) , if the number of ticks compressed to 2 bytes is more than the number of ticks compressed to 4 bytes, then compressing to 3 bytes is more efficient. You have to look at the instrument statistics.
You can switch the maximum compression to 3 bytes using the following command:

 #define compress to 3 bytes // Compress ticks to 3 bytes instead of 2 bytes.

Check the boxes to store elements in compressed form

3 variants of scale elements for storage have been programmed:

They can also be additionally compressed to ZIP. There will be 6 versions in total

 Method = 1 ; //1...6 BidAsk_=1, BidAskVolume_=2, All_=3, BidAsk_Zipped=4, BidAskVolume_Zipped=5, All_Zipped=6

Before starting compression, you need to pass to this class a variant of the price tick storage and some standard parameters for price calculation and normalization.

    TickCompressor compressor2;
double volume_step_= symbolinfodouble ( symbol (), SYMBOL_VOLUME_STEP );
    compressor2.start(method, _view ,volume_step_, _number );

If the Expert Advisor uses flags, they can be restored from price changes via commands

 #Define recovery flags // Recover quote markers from changes to ask, bid, volume - will add 7% to quote generation time 931 instead of 869 ms

EA comes with a compression test, which gives statistics on speed and compression ratio. You can see an example of tick compression and decompression inside.

An example of an Expert Advisor for trading can be viewed here https://www.mql5.com/en/code/65821 .

Statistics for 2 and 3 byte compression:

tick compression

Block by block:

 int postcode = 0 ; //Compressed byte counter if (amount > ticks_per_block) { // > 1 block - glue blocks of tmp to Ticks2 for ( integer start = 0 ; start < amount; start += ticks_per_block) {
          Compressor2.Compress(Ticks, tmp, start, (quantity > start+ticks_per_block ?ticks_per_block : quantity - start));
          Postal code += array copy (Ticks2,tmp,ZIPpos); //Copy to the end of Ticks2 }
    } else { //1 block - decompress directly into Ticks2 compressor2.compression(ticks, ticks2, 0 , number);
    }

If the number of ticks in 1 block is set to be greater than the total number of ticks in the array, it will be compressed into 1 block.

If you always need to compress into 1 block, you can use

 Compressor2.Compression(Ticks,Ticks2);

But decompression of such large or very large blocks may be 2x slower. Also, large chunks consume a lot of memory.

Unpacking ticks

When unpacking, you need to know the number of ticks you are packing. The receiver array must be of this size.

 ArrayResize (Ticks3,Amount);

For example, the size can be saved in a file. Then use it when unpacking.

If the size is unknown you can change the size inside the loop by the number of ticks in the block

 //slow down
Array resize (Ticks3,total_ticks+ticks_per_block, 10000000 ); //Resize large array - slower than covering small blocks

This code gets the ticks block by block. It also counts correctly if there is only 1 chunk. The ticks are not collected into a large array, but can be processed immediately by your Strategy(Ticks3[j]) strategy;

although (zip< arraysize (scale2)){
      nextSize=Compressor3.ArrToInt(Ticks2,ZIPpos); //Adjust the size of the next block, increase ZIPpos by 4 units s = array copy (tmp, ticks 2, 0 , ZIPpos, next size); //Copy the new block to tmp with size nextSize //3 times slower ArrayResize(Ticks3,total_ticks+ticks_per_block,10000000); //Resizing large arrays - slower than covering small blocks //total_ticks=Compressor3.DeCompress(tmp,Ticks3,nextSize,total_ticks); //Decompress the block and add it to Ticks3.  
           Total_ticks+=Compressor3.DeCompress(tmp,Ticks3,nextSize, 0 ); //Decompress the block and overwrite it in Ticks3 ZIPpos+=NextSize;
for ( integer j = 0 ; j < tick; j++){ strategy(Ticks3[j]);} //strategy };

Collect price changes across all blocks into a large array:

 although (zip< arraysize (scale2)){
      nextSize=Compressor3.ArrToInt(Ticks2,ZIPpos); //Adjust the size of the next block, increase ZIPpos by 4 units s = array copy (tmp, ticks 2, 0 , ZIPpos, next size); //Copy the new block to tmp with size nextSize //Slower ArrayResize(Ticks3,total_ticks+ticks_per_block,10000000); //Resizing large arrays - slower than covering small blocks Total_ticks=Compressor3.DeCompress(tmp,Ticks3,nextSize,total_ticks); //Decompress the block and add it to Ticks3  
//total_ticks+=Compressor3.DeCompress(tmp,Ticks3,nextSize,0); //Decompress the block and overwrite it in Ticks3 ZIPpos+=nextSize;
//for (int j = 0; j     };

 Or a single line. Only 1 block should be logged. If more - use the 2 code variations above.
 Total_ticks=Compressor3.DeCompress(Ticks2,Ticks3);

 JSON library supporting MQL4/MQL5
 This function performs the main logic of opening a trade. It calculates the opening price, take profit level and stop loss based on the trading symbol information and parameters provided by the user. Prepare a trading request (MqlTradeRequest) that contains necessary information, such as trading symbol, trading volume, order type, slippage, comments, magic number, etc. Call the OrderSend function to send a transaction request and obtain the result. SetTypeFillingBySymbol function: Determines the order fulfillment type (Fill or Cancel, Immediate or Cancel, or Return) based on the tag's fulfillment policy. GetMinTradeLevel function: Calculates the minimum trading level based on the freeze level and stop level of the trading symbol. Adjust the minimum level to ensure it is within certain limits and return results.
 This filter was created to simplify the process of finding assets trading at discounted prices. Initial use may take slightly longer due to the data loading process for all selected instruments. The tool can scan all available broker assets or be limited to specific asset classes.
 Overview Grid Master EA is an automated trading system that implements a two-way grid strategy. It places multiple pending orders above and below the current market price, profiting from market oscillations in both directions.
 Attachment download
 📎 TickCompressor.mqh (39.54 KB)
 📎Test_TickCompressor.mq5 (6.91 KB)
 Source: MQL5 #66201