Evolutionary ANFIS Training in MATLAB

Training an ANFIS (Adaptive Neuro-Fuzzy Inference System) is a special kind of optimization problem. So metaheuristics and evolutionary algorithms can be used to train it, that is, to tune its parameters.

This post shares a MATLAB implementation of evolutionary ANFIS training. The code first creates an initial raw ANFIS structure. It then trains the ANFIS with either a Genetic Algorithm (GA) or Particle Swarm Optimization (PSO).

The evolutionary-trained ANFIS is used to solve a nonlinear regression and function approximation problem.

Training as an optimization problem

The standard ANFIS training (Jang, 1993) uses a hybrid of least squares and gradient descent. It is fast, but the gradient part can get stuck in a local minimum and needs a differentiable error. A GA or PSO only needs the value of the error for a given set of parameters, so it can search the parameter space more globally.

An ANFIS with first order Takagi-Sugeno rules has two groups of parameters:

• the parameters of the input membership functions (for a Gaussian, its width and center),
• the coefficients of the linear output functions of the rules.

Let $\mathbf{p}_0$ be the vector of all these parameters in the initial FIS. In this code, the decision variables are not the parameters themselves but a vector $\mathbf{x}$ of multipliers, one for each parameter:

where $\odot$ is the element by element product. The vector $\mathbf{x} = (1, 1, \dots, 1)$ gives back the initial FIS, and the first member of the population starts there. Multipliers very close to zero are moved to $\pm 10^{-5}$. This does not prevent a parameter from remaining zero if its initial value in $\mathbf{p}_0$ is zero.

The cost of a solution is the root mean square error of the FIS on the training data:

where $t_k$ are the targets and $y_k$ the outputs of the FIS with parameters $\mathbf{p}$.

How the code works

1. LoadData loads engine_data.mat (1199 samples, two inputs), takes the first of the two outputs as the target, shuffles the samples, and keeps 70% for training and 30% for testing.
2. CreateInitialFIS builds a Sugeno FIS with Fuzzy c-Means clustering (genfis3) and 10 clusters, so the FIS has 10 rules.
3. main.m asks in a list dialog for the training method: GA or PSO.
4. TrainAnfisUsingGA or TrainAnfisUsingPSO runs the algorithm with TrainFISCost as the cost function. GetFISParams and SetFISParams read and write the parameter vector of the FIS.
5. The script evaluates the trained FIS on the training and test data and plots the results with PlotResults.

With two inputs and 10 clusters, the FIS has 10 Gaussian membership functions per input (2 parameters each) and 10 linear output functions (3 coefficients each), so the search has 70 decision variables.

Parameters in the code

Both algorithms use 1000 iterations and a population of 25. The other settings are:

The GA is real coded. It uses roulette wheel selection with Boltzmann style probabilities, blend crossover with random weights in $[-\gamma, 1+\gamma]$, and Gaussian mutation of 15% of the variables. The PSO limits the velocity to 10% of the variable range and reflects the velocity of a particle that leaves the bounds.

How to run the code

Open the folder in MATLAB and run main.m, then choose the training method in the dialog. The algorithm prints the best cost of each iteration in the Command Window. When it ends, the script shows the results for the training and test data. Both algorithms evaluate the FIS many times (about 25,000 cost evaluations for PSO and 28,000 for GA), so a run can take a while. Lower Params.MaxIt in TrainAnfisUsingGA.m or TrainAnfisUsingPSO.m for a quick test.

To use your own data, change LoadData.m. To change the size of the FIS, change the number of clusters in the call CreateInitialFIS(data,10) in main.m.

Check the result

Inspect parameter validity as well as training RMSE, especially membership-function widths after applying signed multipliers. A zero initial parameter cannot change under purely multiplicative encoding. Use validation data to choose the algorithm and settings, then keep the final test set untouched. This package models the first engine output, unlike the standard ANFIS example. It uses legacy FIS structure fields and probably needs Fuzzy Logic Toolbox, with Statistics and Machine Learning Toolbox functions in the GA path. Current-release compatibility was not tested.

Applications

Evolutionary training of fuzzy and neuro-fuzzy models is used when the error is not smooth, when a global search is wanted, or when the cost includes terms that a gradient method cannot handle. Typical uses are system modeling, prediction and the tuning of fuzzy controllers.

Related posts

• Nonlinear Regression using ANFIS, which trains the same kind of model with the standard anfis function
• Binary and Real-Coded Genetic Algorithms in MATLAB
• Particle Swarm Optimization in MATLAB

Mostapha Kalami Heris, Evolutionary ANFIS Training in MATLAB (YPFZ104), Yarpiz, 2015. https://yarpiz.com/evolutionary-anfis-training-ypfz104

@misc{yarpiz-ypfz104,
  author = {{Mostapha Kalami Heris}},
  title = {{Evolutionary ANFIS Training in MATLAB}},
  howpublished = {Yarpiz},
  year = {2015},
  note = {Code YPFZ104},
  url = {https://yarpiz.com/evolutionary-anfis-training-ypfz104}
}