# Linear Discriminant Analysis (LDA) in MATLAB

URL: https://yarpiz.com/linear-discriminant-analysis-ypml114
Author: Mostapha Kalami Heris
Published: 2015-09-22
Updated: 2026-10-30
Topics: Machine Learning

> Extract supervised linear features in MATLAB with Fisher LDA, within- and between-class scatter, a generalized eigenproblem, and the iris dataset.

This post shares an open-source MATLAB implementation of Linear Discriminant Analysis (LDA), also called Fisher Discriminant Analysis (FDA). It is used for dimensionality reduction and linear feature extraction from labeled data.

The method goes back to Ronald A. Fisher, who introduced it in 1936 in the same paper that made the iris data set well known. The example in this package uses the iris data too.

## The idea

Like [Principal Component Analysis (PCA)](https://yarpiz.com/principal-component-analysis-in-python-and-matlab-video-tutorial-yppca191211), LDA finds a linear transformation $y = W^T x$ of the data. The difference is the goal. PCA is unsupervised: it looks for the directions with the largest variance and ignores the labels. LDA is supervised: it uses the class labels and looks for the directions that separate the classes best. A direction with a large variance is not always a direction that separates the classes, so the two methods can give very different features.

## The Fisher criterion

Assume there are $c$ classes. Class $j$ has $n_j$ samples with the mean $\mu_j$, and $\mu$ is the mean of all the data. LDA uses two scatter matrices.

The **within-class scatter** measures how spread out each class is around its own mean:

$$
S_W = \sum_{j=1}^{c} \sum_{x \in \text{class } j} (x - \mu_j)(x - \mu_j)^T
$$

The **between-class scatter** measures how far the class means are from the overall mean:

$$
S_B = \sum_{j=1}^{c} n_j \, (\mu_j - \mu)(\mu_j - \mu)^T
$$

A good direction $w$ puts the class means far apart and keeps each class compact. Fisher's criterion is the ratio of the two:

$$
J(w) = \frac{w^T S_B \, w}{w^T S_W \, w}
$$

The directions that maximize $J$ are the solutions of the generalized eigenvalue problem

$$
S_B \, w = \lambda \, S_W \, w
$$

The eigenvalue $\lambda$ of each direction is its value of $J$, so the directions are sorted by their eigenvalues in descending order. Since $S_B$ is built from $c$ class means, at most $c - 1$ eigenvalues are nonzero. For the iris data, with three classes, there are at most two nonzero between-class discriminant directions.

## What is in the package

- `LDA.m`: the function `[Y, W, lambda] = LDA(X, L)`. It takes the data matrix `X` (one sample per row) and the class labels `L`. It returns the transformed data `Y`, the transformation matrix `W` and the values `lambda`.
- `main.m`: an example script that loads the iris data, calls `LDA`, and plots each original variable next to the matching transformed variable.
- `iris.mat`: the iris data used in the example.
- `license.txt`: the license terms.

Inside `LDA.m`, the function computes $S_W$ and $S_B$ as above, solves the generalized eigenvalue problem with `eig(Sb, Sw)`, sorts the eigenvectors by their eigenvalues in descending order, and computes `Y = X*W`. The values in `lambda` are these sorted eigenvalues, so they show how much each new feature separates the classes.

## How to run

Run `main.m` in MATLAB to see the example. To use the code on your own data, call `LDA` with your data matrix and a vector of class labels.

```matlab {title="Project labeled data with LDA"}
[Y, W, lambda] = LDA(X, L);
```

To reduce the data to $m$ dimensions, keep the first $m$ columns of `Y`. `LDA.m` itself uses only core MATLAB functions. The example script `main.m` converts the iris targets to labels with `vec2ind`, which probably needs the Deep Learning Toolbox.

## Check the result

Fit the projection using training samples and labels only, then apply the same matrix to validation or test samples. Fitting it on all labels before evaluation leaks information from the test set. A singular within-class scatter matrix can occur with redundant features or too few samples, so inspect the eigenproblem rather than assuming every returned direction is stable. The package returns projected features, not a separately trained classifier or a measured classification accuracy.

## Applications

LDA is used to reduce the number of features before classification, and also directly as a linear classifier. Well known uses include face recognition (the Fisherfaces method), medical diagnosis from measured variables, and other pattern recognition tasks with labeled data.

## Related posts

- [Principal Component Analysis (PCA) in Python and MATLAB](https://yarpiz.com/principal-component-analysis-in-python-and-matlab-video-tutorial-yppca191211), the unsupervised counterpart of LDA.

## How to cite

Mostapha Kalami Heris, Linear Discriminant Analysis (LDA) in MATLAB (YPML114), Yarpiz, 2015. https://yarpiz.com/linear-discriminant-analysis-ypml114