metric formula effective Accuracy 𝑇𝑃 + 𝑇𝑁 𝑇𝑃 + 𝐹𝑃 + 𝑇𝑁 + 𝐹𝑁 the proportion of the total number of predictions that were correct Error rate 𝐹𝑃 + 𝐹𝑁 𝑇𝑃 + 𝐹𝑃 + 𝑇𝑁 + 𝐹𝑁 The number of all incorrect predictions divided by the total number of the dataset. The best error rate is 0.0, whereas the worst is 1.0. Recall = TP rate = sensitivity 𝑇𝑃 𝑇𝑃+𝐹𝑁 TN rate = specificity 𝑇𝑃 𝑃 𝑇𝑃 𝑇𝑃 + 𝐹𝑃 Precision FP rate = 𝐹𝑃 𝐹𝑃+𝑇𝑁 = 1 – TN rate 𝑇𝑁 𝑇𝑁 = 𝑇𝑁 + 𝐹𝑃 𝑁 the proportion of actual positive cases which are correctly identified the proportion of positive cases that were correctly identified the proportion of negative data points that are mistakenly considered as positive, with respect to all negative data points Number of correct negative predictions divided by the total number of negatives. It is also called true negative rate (TNR). The best specificity is 1.0, whereas the worst is 0.0 F1 𝐹𝛽 2 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 . 𝑟𝑒𝑐𝑎𝑙𝑙 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙 (1 + 𝛽 2 ) . (𝛽 2 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 . 𝑟𝑒𝑐𝑎𝑙𝑙 . 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛) + 𝑟𝑒𝑐𝑎𝑙𝑙 𝑁 Like F1 but can add more weight to either precision or recall Mean absolute error (L1 loss) 1 ∑|𝑦𝑖 − ŷ𝑖 | 𝑁 Not preferred in cases where outliers are prominent. MAE does not penalize large errors Mean square error (L2 loss) 𝑁 1 . ∑ (𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖 − 𝑎𝑐𝑡𝑢𝑎𝑙𝑖 )2 𝑁 𝑖 MSE penalizes large errors. 𝑖=1 1. It avoids the use of absolute error values which is highly undesirable in mathematical calculations. Root mean squared error ∑𝑁(𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖 − 𝑎𝑐𝑡𝑢𝑎𝑙𝑖 )2 √ 1 𝑁 N: total number of observations 2. When we have more samples, reconstructing the error distribution using RMSE is considered to be more reliable. 3. RMSE is highly affected by outlier values. Hence, make sure you’ve removed outliers from your data set prior to using this metric. 4. As compared to mean absolute error, RMSE gives higher weightage and punishes large errors ROC AUC Precisionrecall curve mAP 1 mAP = 𝑁 ∑𝑁 𝑘=1 𝐴𝑃𝑘 với 𝐴𝑃𝑘 : 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑐ủ𝑎 𝑙ớ𝑝 𝑘 N: số class Matthews correlation coefficient Total sum of squares (SST) 𝑇𝑃. 𝑇𝑁 − 𝐹𝑃. 𝐹𝑁 √(𝑇𝑃 + 𝐹𝑃). (𝑇𝑃 + 𝐹𝑁). (𝑇𝑁 + 𝐹𝑃). (𝑇𝑁 + 𝐹𝑁) Matthews correlation coefficient (MCC) is a correlation coefficient calculated using all four values in the confusion matrix 𝑁 ∑(𝑦𝑖 − ȳ)2 𝑖=1 𝑁 Sum of squared error (SSE) ∑(𝑦𝑖 − ŷ𝑖 )2 𝑅 2 score 𝑆𝑆𝐸 𝑅2 = 1 − 𝑆𝑆𝑇 Top1 accuracy in classification Model dự đoán → xác suất của n class Nếu class có xác suất cao nhất = expected answer thì mới được tính là dự đoán đúng Top5 accuracy in classification Model dự đoán → xác suất của n class expected answer nằm trong 5 class có xác suất cao nhất thì được tính là dự đoán đúng 𝑖=1 R² score ranges from 0 to 1. The closest to 1 the R², the better the regression model is. If R² is equal to 0, the model is not performing better than a random model. If R² is negative, the regression model is erroneous.