Nonlinear Regression using ANFIS

Adaptive Neuro-Fuzzy Inference System (ANFIS) combines an artificial neural network (ANN) with a Takagi-Sugeno type fuzzy system. It was proposed by Jyh-Shing Roger Jang in 1993 in this paper (opens in a new tab) (IEEE Transactions on Systems, Man, and Cybernetics, 23(3), 665 to 685). ANFIS inherits the benefits of both neural networks and fuzzy systems: it learns parameters from data and expresses the model as if-then rules, whose interpretability depends on the number and complexity of the rules. It is used for supervised learning tasks such as regression and classification.

The problem

In nonlinear regression, we have samples of an input vector $\mathbf{x}$ and a target $y$, and we want a model $\hat{y} = f(\mathbf{x})$ that predicts $y$ well, also for samples that were not used in training. ANFIS builds $f$ as a fuzzy rule base and tunes the rules on the data.

How ANFIS works

ANFIS uses first order Takagi-Sugeno rules (Takagi and Sugeno, 1985). For two inputs $x_1$ and $x_2$, rule $i$ has the form:

Here $A_i$ and $B_i$ are fuzzy sets, such as Gaussian membership functions, and $p_i$, $q_i$, $r_i$ are the linear consequent parameters. ANFIS writes the rule base as a network with five layers:

1. Fuzzification. Each node computes a membership grade, such as $\mu_{A_i}(x_1)$.
2. Rule strength. Each node multiplies the incoming grades: $w_i = \mu_{A_i}(x_1)\,\mu_{B_i}(x_2)$.
3. Normalization. $\bar{w}_i = w_i / \sum_j w_j$.
4. Consequents. Each node computes $\bar{w}_i f_i$.
5. Output. One node adds everything up: $\hat{y} = \sum_i \bar{w}_i f_i$.

Training uses a hybrid learning rule. In the forward pass, the membership functions are fixed, the output is linear in $p_i$, $q_i$ and $r_i$, and these are found by least squares. In the backward pass, the consequents are fixed, and the parameters of the membership functions are updated by gradient descent on the error.

ANFIS in MATLAB

Fuzzy Logic Toolbox provides the basic functions to create and train fuzzy inference models. The package uses three legacy functions to create the initial FIS structure:

• Grid Partitioning, using the genfis1 function
• Subtractive Clustering, using the genfis2 function
• Fuzzy c-Means (FCM) Clustering, using the genfis3 function

The original workflow uses two interfaces to design and train ANFIS models:

• A graphical user interface (GUI) tool, using the anfisedit function
• Programmatic training of the ANFIS structure, using the anfis function

Grid partitioning gives every input the same number of membership functions and creates one rule for every combination, so the number of rules grows quickly with the number of inputs. The two clustering methods create one rule for each cluster, which suits problems with more inputs.

About this implementation

This post shares an implementation of nonlinear regression using ANFIS in MATLAB. The whole process of designing and training the ANFIS is done programmatically. You can choose the generation method (genfis1, genfis2 or genfis3) and the parameters of the training algorithm. If you know MATLAB, you should find the code easy to use in your research and projects.

The data

The file engine_data.mat holds 1199 samples with two inputs and two outputs. The code models the second of the two outputs. The samples are shuffled and split into 85% training data and 15% test data.

How to run the code

Open the folder in MATLAB and run main.m. The script asks for its settings in dialog boxes:

1. Select the FIS generation method: grid partitioning, subtractive clustering or FCM (the default).
2. Enter the parameters of that method.
3. Enter the training parameters of anfis.

The default values in the dialogs are:

With grid partitioning and two inputs, 5 membership functions per input give $5 \times 5 = 25$ rules. The script trains the FIS with the hybrid method (OptimizationMethod = 1), evaluates it with evalfis, and computes the mean square error, the RMSE, and the mean and standard deviation of the errors for the training and test data. It then plots the targets and outputs for the training, test and full data, draws regression plots when plotregression is available, and shows the surface of the trained FIS with gensurf.

To use your own data, replace Inputs and Targets with your arrays (one row per sample after the transpose in main.m) and select the output column you want to model.

Check the result

Compare training and held-out errors rather than only the fitted surface or a regression plot. Choose membership functions and training settings using training and validation data, then evaluate once on an untouched test set. The script models the second engine output, while the evolutionary ANFIS package models the first, so their raw errors are not directly comparable. Fit preprocessing on training data only and check that predictions remain finite over the intended input range.

Applications

ANFIS is used for modeling and prediction in many fields, for example system identification, process modeling and control, time series forecasting, and engineering problems where both accuracy and readable rules matter.

Related posts

• Time-Series Prediction using ANFIS in MATLAB
• Evolutionary ANFIS Training in MATLAB
• Group Method of Data Handling (GMDH) in MATLAB

Mostapha Kalami Heris, Nonlinear Regression using ANFIS (YPFZ101), Yarpiz, 2015. https://yarpiz.com/nonlinear-regression-using-anfis-ypfz101

@misc{yarpiz-ypfz101,
  author = {{Mostapha Kalami Heris}},
  title = {{Nonlinear Regression using ANFIS}},
  howpublished = {Yarpiz},
  year = {2015},
  note = {Code YPFZ101},
  url = {https://yarpiz.com/nonlinear-regression-using-anfis-ypfz101}
}