Evaluation: from Precision, Recall and F-measure to ROC, Informedness and Markedness

This article discusses the limitations of common performance metrics like precision and recall, and introduces alternative measures like informedness and markedness to provide a clearer picture of a model's true performance.

Details

The article explains that while metrics like precision and recall are commonly used to evaluate machine learning models, they can sometimes be misleading. These scores can reward lucky guesses and hide how much a system is actually learning. The author introduces the concepts of informedness and markedness as alternative measures that provide a clearer picture of a model's performance. Informedness tells you how useful a decision is, not just how popular it is, while markedness looks at how strongly the label predicts the outcome. Together, these measures tie into familiar ideas like ROC curves and correlation, but without the need for complex math. The key message is to be cautious when looking at performance reports and to seek out evaluation metrics that account for bias and chance.