Group Method of Data Handling (GMDH) in MATLAB

Group Method of Data Handling (GMDH) is a family of mathematical modeling and nonlinear regression algorithms. It was proposed by Alexey Grigorevich Ivakhnenko, a Ukrainian scientist and mathematician, in 1968. An often cited English account of the method is his paper “Polynomial Theory of Complex Systems” (IEEE Transactions on Systems, Man, and Cybernetics, 1971 (opens in a new tab)). The approach is also known as a Polynomial Neural Network and can be seen as a specific type of supervised Artificial Neural Network (ANN). GMDH also uses the idea of natural selection to control the size, complexity and accuracy of the network.

The main applications of GMDH are modeling of complex systems, function approximation, nonlinear regression and pattern recognition.

The problem

GMDH solves a supervised regression problem. We have $N$ samples of an input vector $\mathbf{x} = (x_1, \dots, x_n)$ and a target $y$, and we want a model $\hat{y} = f(\mathbf{x})$ with a small prediction error on new data. Unlike a standard neural network, GMDH does not fix the structure of the model in advance. It builds the network layer by layer and decides from the data which neurons to keep and when to stop.

How GMDH works

Each neuron takes two inputs, $x_i$ and $x_j$, and fits a quadratic polynomial to the target:

The six coefficients are found by least squares. Stacking such neurons in layers gives a model that approximates a high order polynomial of all inputs, known as the Kolmogorov-Gabor polynomial.

The key idea is to split the training data into two parts. The coefficients are fitted on the first part, and each neuron is judged by its error on the second part, which it has not seen. This external criterion helps choose model complexity using held-out data, but repeated selection on that data does not guarantee generalization. The network grows as follows:

1. For a layer with $n$ inputs, create one neuron for every pair of inputs, that is $n(n-1)/2$ neurons.
2. Fit each neuron by least squares on the first part and compute its root mean square error (RMSE) on the second part.
3. Sort the neurons by this validation error and keep only the best ones (selection).
4. The outputs of the kept neurons become the inputs of the next layer.
5. Stop when the best validation error no longer improves, when only one neuron is left, or when the maximum number of layers is reached. The best neuron of the last layer is the output of the model.

Implementation in MATLAB

This post shares a structured MATLAB implementation of GMDH. You can use it for modeling, function approximation and regression. The framework can also be modified for other supervised learning tasks, such as time series prediction and classification.

The code follows the steps above:

• FitPolynomial.m fits one neuron with the quadratic polynomial, using pinv for the least squares solution, and computes its training and validation errors.
• GetPolynomialLayer.m creates all pairs of inputs, fits a neuron for each pair, and sorts the neurons by validation RMSE.
• GMDH.m shuffles the training data, splits it into the fitting and validation parts, and builds the layers.
• ApplyGMDH.m evaluates a trained network on new inputs.

In each layer, GMDH.m keeps the neurons whose validation error is below a threshold set by the selection pressure $\alpha$:

A larger $\alpha$ keeps fewer neurons. The layer is then cut to at most MaxLayerNeurons neurons, and the last allowed layer keeps only its best neuron. Training also stops early when the best validation error of a new layer is larger than that of the previous layer.

Parameters in the code

The parameters are set in main.m:

How to run the code

The download includes the sample data chemical_data_file.mat, with 8 input variables and 1 target for 498 samples. To run the example, open the GMDH folder in MATLAB and run main.m. The script:

1. loads the data and splits it at random into training data (85%) and test data (15%),
2. trains the network with GMDH and prints the number of neurons and the minimum error of each layer,
3. applies the network to all data with ApplyGMDH,
4. plots the targets and outputs for the training, test and full data with PlotResults, and also draws regression plots when the plotregression function is available.

Type gmdh.Layers in the Command Window to see the neurons of each layer, with the selected input pairs (vars) and the coefficients (c).

To use your own data, replace the two arrays Inputs (one row per variable, one column per sample) and Targets (one row, one column per sample).

Check the result

Keep the outer test set separate from the internal validation set used to select neurons and layers. Inspect the number of neurons and prediction shape after training, especially if early stopping occurs. A source review flagged that the early-stop path may leave an empty last layer and cause ApplyGMDH to return an empty result. Review that path before relying on the package. No optimizer code was executed or changed in this content pass. Compare held-out RMSE with a simpler regression baseline.

Applications

GMDH has been used for modeling and prediction in many fields, for example in system identification, economic and energy forecasting, environmental modeling, and engineering problems where a compact, explicit model of the data is useful.

Further reading

• Group method of data handling on Wikipedia (opens in a new tab)
• gmdh.net (opens in a new tab), with detailed descriptions of GMDH algorithms, news, tutorials and other resources

Related posts

• Time-Series Prediction using GMDH in MATLAB
• Nonlinear Regression using ANFIS

Mostapha Kalami Heris, Group Method of Data Handling (GMDH) in MATLAB (YPML113), Yarpiz, 2015. https://yarpiz.com/gmdh-ypml113

@misc{yarpiz-ypml113,
  author = {{Mostapha Kalami Heris}},
  title = {{Group Method of Data Handling (GMDH) in MATLAB}},
  howpublished = {Yarpiz},
  year = {2015},
  note = {Code YPML113},
  url = {https://yarpiz.com/gmdh-ypml113}
}