Using CatBoost AI Models for Forex Trading in MetaTrader 5 - Part 2

This is Part 2 of the guide on using CatBoost AI models for Forex trading. For a complete understanding, I recommend reading Part 1 as well.

ONNX and MetaTrader 5

ONNX¹ stands for Open Neural Network Exchange. It is an open-standard format for representing machine learning (ML) models that was originally co-developed by Microsoft and Facebook (now Meta). The primary goal of ONNX is to allow models to be easily transferred between different frameworks, tools, runtimes, and hardware platforms without requiring heavy custom integration or code rewrites.

MetaTrader 5² (MT5) is a multi-asset trading platform developed by MetaQuotes Software Corp. It is widely used by traders and brokers worldwide for financial markets, including Forex, stocks, futures, and CFDs (Contracts for Difference). It features a built-in editor and debugger for creating and testing automated trading systems called Expert Advisors (EAs) using the MQL5 programming language.

To bridge the gap between ONNX and MetaTrader, I opted to use Ubuntu within WSL2. A clean Windows setup will be covered in a future article.

Saving the CatBoost Model to ONNX

To integrate the CatBoost model with MetaTrader 5, we need to save it in the ONNX format. However, saving a CatBoost model can be a bit more challenging compared to Sklearn and Keras models, which provide built-in methods for easier conversion.

Saving the model is based on their instructions from the documentation.

from onnx.helper import get_attribute_value
import onnxruntime as rt
from skl2onnx import convert_sklearn, update_registered_converter
from skl2onnx.common.shape_calculator import (
    calculate_linear_classifier_output_shapes,
)  # noqa
from skl2onnx.common.data_types import (
    FloatTensorType,
    Int64TensorType,
    guess_tensor_type,
)
from skl2onnx._parse import _apply_zipmap, _get_sklearn_operator_name
from catboost import CatBoostClassifier
from catboost.utils import convert_to_onnx_object

def skl2onnx_parser_castboost_classifier(scope, model, inputs, custom_parsers=None):
    
    options = scope.get_options(model, dict(zipmap=True))
    no_zipmap = isinstance(options["zipmap"], bool) and not options["zipmap"]

    alias = _get_sklearn_operator_name(type(model))
    this_operator = scope.declare_local_operator(alias, model)
    this_operator.inputs = inputs

    label_variable = scope.declare_local_variable("label", Int64TensorType())
    prob_dtype = guess_tensor_type(inputs[0].type)
    probability_tensor_variable = scope.declare_local_variable(
        "probabilities", prob_dtype
    )
    this_operator.outputs.append(label_variable)
    this_operator.outputs.append(probability_tensor_variable)
    probability_tensor = this_operator.outputs

    if no_zipmap:
        return probability_tensor

    return _apply_zipmap(
        options["zipmap"], scope, model, inputs[0].type, probability_tensor
    )


def skl2onnx_convert_catboost(scope, operator, container):
    """
    CatBoost returns an ONNX graph with a single node.
    This function adds it to the main graph.
    """
    onx = convert_to_onnx_object(operator.raw_operator)
    opsets = {d.domain: d.version for d in onx.opset_import}
    if "" in opsets and opsets[""] >= container.target_opset:
        raise RuntimeError("CatBoost uses an opset more recent than the target one.")
    if len(onx.graph.initializer) > 0 or len(onx.graph.sparse_initializer) > 0:
        raise NotImplementedError(
            "CatBoost returns a model initializers. This option is not implemented yet."
        )
    if (
        len(onx.graph.node) not in (1, 2)
        or not onx.graph.node[0].op_type.startswith("TreeEnsemble")
        or (len(onx.graph.node) == 2 and onx.graph.node[1].op_type != "ZipMap")
    ):
        types = ", ".join(map(lambda n: n.op_type, onx.graph.node))
        raise NotImplementedError(
            f"CatBoost returns {len(onx.graph.node)} != 1 (types={types}). "
            f"This option is not implemented yet."
        )
    node = onx.graph.node[0]
    atts = {}
    for att in node.attribute:
        atts[att.name] = get_attribute_value(att)
    container.add_node(
        node.op_type,
        [operator.inputs[0].full_name],
        [operator.outputs[0].full_name, operator.outputs[1].full_name],
        op_domain=node.domain,
        op_version=opsets.get(node.domain, None),
        **atts,
    )


update_registered_converter(
    CatBoostClassifier,
    "CatBoostCatBoostClassifier",
    calculate_linear_classifier_output_shapes,
    skl2onnx_convert_catboost,
    parser=skl2onnx_parser_castboost_classifier,
    options={"nocl": [True, False], "zipmap": [True, False, "columns"]},
)

Final model conversion and saving of the onnx file.

model_onnx = convert_sklearn(
    pipe,
    "pipeline_catboost",
    [("input", FloatTensorType([None, X_train.shape[1]]))],
    target_opset={"": 12, "ai.onnx.ml": 2},
)

# And save.
with open(onnx_file, "wb") as f:
    f.write(model_onnx.SerializeToString())

Creating a CatBoost Trading Robot

We begin by embedding the ONNX model as a resource within the main Expert Advisor.

#resource "\\Files\\CatBoost.EURUSD.OHLC.D1.onnx" as uchar catboost_onnx[]

We import the library required to load the CatBoost model.

#include <MALE5\Gradient Boosted Decision Trees(GBDTs)\CatBoost\CatBoost.mqh>
CCatBoost cat_boost;

We need to collect data in the same way as the training data was gathered, within the OnTick function.

void OnTick()
 {
...
...
...     

   if (CopyRates(Symbol(), timeframe, 1, 1, rates) < 0) //Copy information from the previous bar
    {
      printf("Failed to obtain OHLC price values error = ",GetLastError());
      return;
    }
   
   MqlDateTime time_struct;
   string time = (string)datetime(rates[0].time); //converting the date from seconds to datetime then to string
   TimeToStruct((datetime)StringToTime(time), time_struct); //converting the time in string format to date then assigning it to a structure
      
   vector x = {rates[0].open,
               rates[0].high, 
               rates[0].low, 
               rates[0].close, 
               rates[0].tick_volume, 
               time_struct.day, 
               time_struct.day_of_week, 
               time_struct.day_of_year, 
               time_struct.mon,
               time_struct.hour,
               time_struct.min}; //input features from the previously closed bar
...
...
...     
  }

Finally, we can retrieve the predicted signal along with the probability vector for the bearish signal (class 0) and the bullish signal (class 1).

   vector proba = cat_boost.predict_proba(x); //predict the probability between the classes
   long signal = cat_boost.predict_bin(x); //predict the trading signal class
   
   Comment("Predicted Probability = ", proba,"\nSignal = ",signal);

We can finalize this Expert Advisor by implementing a trading strategy based on the model’s predictions.

The strategy is straightforward: when the model predicts a bullish signal, we open a buy trade and close any existing sell trades. Conversely, when the model predicts a bearish signal, we open a sell trade and close any existing buy trades.

void OnTick()
  {
//---
   
   if (!NewBar())
    return;
   
//--- Trade at the opening of each bar

   if (CopyRates(Symbol(), timeframe, 1, 1, rates) < 0) //Copy information from the previous bar
    {
      printf("Failed to obtain OHLC price values error = ",GetLastError());
      return;
    }
   
   MqlDateTime time_struct;
   string time = (string)datetime(rates[0].time); //converting the date from seconds to datetime then to string
   TimeToStruct((datetime)StringToTime(time), time_struct); //converting the time in string format to date then assigning it to a structure
      
   vector x = {rates[0].open,
               rates[0].high, 
               rates[0].low, 
               rates[0].close, 
               rates[0].tick_volume, 
               time_struct.day, 
               time_struct.day_of_week, 
               time_struct.day_of_year, 
               time_struct.mon,
               time_struct.hour,
               time_struct.min}; //input features from the previously closed bar
               
   vector proba = cat_boost.predict_proba(x); //predict the probability between the classes
   long signal = cat_boost.predict_bin(x); //predict the trading signal class
   
   Comment("Predicted Probability = ", proba,"\nSignal = ",signal);
   
//---
      
      MqlTick ticks;
      SymbolInfoTick(Symbol(), ticks);
      
      if (signal==1) //if the signal is bullish
       {
          if (!PosExists(POSITION_TYPE_BUY)) //There are no buy positions
            m_trade.Buy(min_lot, Symbol(), ticks.ask, 0, 0); //Open a buy trade
          
          ClosePosition(POSITION_TYPE_SELL); //close the opposite trade  
       }
      else //Bearish signal
        {
          if (!PosExists(POSITION_TYPE_SELL)) //There are no Sell positions
            m_trade.Sell(min_lot, Symbol(), ticks.bid, 0, 0); //open a sell trade
            
            
          ClosePosition(POSITION_TYPE_BUY); //close the opposite trade
        }
  }

The EA performed reasonably well in backtests, but it requires additional technical indicators and further adjustments. I will cover these enhancements in a future article.

Final Thoughts

CatBoost and other Gradient Boosted Decision Trees are ideal solutions for limited-resource environments when you need a model that “just works” without the need for extensive feature engineering or complex model configuration—tasks that can often be tedious and unnecessary when working with many machine learning models.

Despite their simplicity and low entry barrier, these models rank among the best and most effective AI tools, widely used in real-world applications.