Accuracy measure

The accuracy of a classifier can be measured in any set: training, evaluation, and test. Let $Z$ be any one of these sets and $N$ be the number of samples in $Z$. The accuracy $Acc$ is measured by taking into account that the classes may have different sizes in $Z$. If there are two classes, for example, with very different sizes and a classifier always assigns the label of the largest class, its accuracy will fall drastically due to the high error rate on the smallest class.

Let $N(i)$, $i=1,2,\ldots,c$, be the number of samples in $Z$ from each class $i$. We define

$$e_{i,1}=\frac{FP(i)}{\vert Z\vert-\left\vert N(i)\right\vert} e_{i,2}=\frac{FN(i)}{\left\vert N(i)\right\vert}, i=1,\ldots,c$$ (1)

where $FP(i)$ and $FN(i)$ are the false positives and false negatives, respectively. That is, $FP(i)$ is the number of samples from other classes that were classified as being from the class $i$ in $Z$, and $FN(i)$ is the number of samples from the class $i$ that were incorrectly classified as being from other classes in $Z$. The errors $e_{i,1}$ and $e_{i,2}$ are used to define

$$E(i)=e_{i,1}+e_{i,2},$$ (2)

where $E(i)$ is the partial sum error of class $i$. Finally, the accuracy $Acc$ of the classification is written as

$$Acc=\frac{2c-\sum_{i=1}^{c}E(i)}{2c}=1-\frac{\sum_{i=1}^{c}E(i)}{2c}.$$ (3)