Classifier

Classifier

Consider a set $\Omega$ of elements. With Bayes' decision rule, it is possible to divide the elements of $\Omega$ into p classes $C_1, C_2, \ldots, C_p$ , from n discriminant attributes $A_1, A_2, \ldots, A_n$ . We must already have examples of each class to choose typical values for the attributes of each class.

The probability of meeting element $\omega \in \Omega$ , having attribute A_i, given that we consider class C_l, will be denoted by $p_{A_i}(\omega / C_l)$ .

If we put all these probabilities together for each attribute, we obtain the global probability of meeting element $\omega$ , given that the class is C_l:

$\begin{displaymath}p(\omega / C_l) = \prod_{i=1}^n p_{A_i}(\omega / C_l)\end{displaymath}$

(5)

Classification must allow the class of unknown element $\omega$ to be decided with the lowest risk of error. Decision in Bayes' theory chooses the class C_l for which the a posteriori membership probability $p(C_l / \omega)$ is the highest:

$\begin{displaymath}p(C_l / \omega) = \max_{j=1}^p (p(C_j / \omega))\end{displaymath}$

(6)

According to Bayes' rule, the a posteriori probability of membership $p(C_l / \omega)$ is calculated from the a priori probabilities of membership of element $\omega$ to class C_l :

$\begin{displaymath}p(C_l / \omega) = \frac{p(\omega / C_l) \cdot P(C_l)}{p(\omega)}\end{displaymath}$

(7)

Denominator $p(\omega)$ is a normalization factor. It ensures that the sum of probabilities $p(C_l / \omega)$ is equal to 1 when l varies.

Some classes appear more frequently than others, and P(C_l) denotes the a priori probability of meeting class C_l.

$p(\omega / C_l)$ denotes the conditional probability of meeting the element $\omega$ , given that we focus on class C_l( given that class C_l is true).

IRIT-UPS