Neural Gas and GNG Networks in MATLAB

The Neural Gas network is a competitive Artificial Neural Network (ANN) proposed by Martinetz and Schulten in 1991. It is very similar to the Self-Organizing Map (SOM). Neural Gas can solve unsupervised learning tasks such as clustering, dimensionality reduction and topology learning. It is used in pattern recognition, data compression, speech recognition and image segmentation. For more information, see the Wikipedia article on Neural Gas (opens in a new tab).

In 1995, Bernd Fritzke proposed an extended version called the Growing Neural Gas (GNG) network. It starts with only 2 neurons and grows while the algorithm runs. The learning and adaptation mechanisms of GNG also differ slightly from those of the basic Neural Gas.

This post shares open-source MATLAB implementations of both the Neural Gas network and the Growing Neural Gas (GNG) network. The code is structured, so if you know MATLAB, you should find it easy to use in your research projects.

The problem

Both networks place a set of neurons, each with a weight vector $w_i$ in the data space, so that the neurons represent the distribution of the data. Each neuron then acts like a cluster center. Both networks also learn links between neurons that are close in the data, and these links show the shape (topology) of the data, for example a ring or a spiral.

Unlike the SOM, Neural Gas has no fixed grid. The neurons move freely, like the particles of a gas, and the links are learned from the data.

How Neural Gas works

For each input $x$, the neurons are ranked by their distance to $x$. The closest neuron has rank $k_i = 0$, the next one rank 1, and so on. Every neuron moves toward the input, and the step becomes smaller with the rank:

The step size $\varepsilon$ and the neighborhood range $\lambda$ decrease during training. In the code, they change exponentially from an initial to a final value over tmax steps:

The same rule is used for $\lambda$ and for the age limit $T$ of the links. After each update, the code connects the two neurons closest to the input with a link and sets its age to zero. The other links of the closest neuron get older, and links older than $T$ are removed. This way of learning links is called competitive Hebbian learning.

How Growing Neural Gas works

GNG starts with two neurons at random positions and adds neurons during training. For each input $x$:

1. Find the nearest neuron $s_1$ and the second nearest $s_2$.
2. Increase the age of all links of $s_1$, and add the squared distance $\lVert x - w_{s_1} \rVert^2$ to its local error.
3. Move $s_1$ toward $x$ with the rate $\varepsilon_b$, and move its linked neighbors with the smaller rate $\varepsilon_n$.
4. Connect $s_1$ and $s_2$ (or reset the age of their link to zero).
5. Remove links older than $T$, and remove neurons that have no links left.
6. Every $L$ inputs, while the network has fewer than $N$ neurons, insert a new neuron halfway between the neuron with the largest error and its neighbor with the largest error. The errors of these two neurons are multiplied by $\alpha$, and the new neuron gets the new error of the first one.
7. Multiply all errors by $\delta$, so that old errors fade.

Because new neurons go where the error is large, the network grows into the regions that it does not yet represent well.

What is in the download

The download has two folders, 01 Neural Gas Network and 02 Growing Neural Gas Network. Each folder has:

• NeuralGasNetwork.m or GrowingNeuralGasNetwork.m: the training function, net = NeuralGasNetwork(X, params, PlotFlag). X holds one sample per row, params is a structure with the parameters, and PlotFlag turns the live plot on or off. The result net holds the weights w and the link matrix C (and the errors E for GNG).
• app1.m, app2.m and app3.m: three examples with the data sets in mydata.mat (circles), jain.mat (Jain) and spiral.mat (spiral).
• main.m: a dialog to choose one of the three examples.
• PlotResults.m: plots the data, the neurons and their links after each pass over the data.

How to run it

Open one of the two folders in MATLAB and run main.m, then choose an example. The plot is updated after every pass over the data, so you can watch the network learn the shape of the data. To use your own data, load it into X in one of the app scripts and adjust the parameters. The code calls pdist2 and unifrnd, so it probably needs the Statistics and Machine Learning Toolbox.

Parameters in the code

Neural Gas, set in app1.m, app2.m and app3.m:

Growing Neural Gas:

Check the result

Inspect both prototype positions and the learned links rather than treating every neuron as a confirmed natural cluster. Scale features before distance-based training and compare several runs because initialization and sample order can affect the result. For Growing Neural Gas, the maximum neuron count is a stopping cap, not an estimate of the true cluster count. Evaluate the representation on separate data if it will feed a later prediction task.

Related posts

• Evolutionary Data Clustering in MATLAB
• DBSCAN Clustering in MATLAB
• Intelligent Image Color Reduction and Quantization, which uses a Self-Organizing Map for clustering.

Mostapha Kalami Heris, Neural Gas and GNG Networks in MATLAB (YPML111), Yarpiz, 2015. https://yarpiz.com/neural-gas-network-ypml111

@misc{yarpiz-ypml111,
  author = {{Mostapha Kalami Heris}},
  title = {{Neural Gas and GNG Networks in MATLAB}},
  howpublished = {Yarpiz},
  year = {2015},
  note = {Code YPML111},
  url = {https://yarpiz.com/neural-gas-network-ypml111}
}