Reply to post:

Decision time for AI: Sometimes accuracy is not your friend

ibmalone

I'd say that, while the ROC doesn't take account of the cost of errors, it does provide you with the information about the algorithm that you need to work on that. If you've got the full curve then you know the potential trade-offs between sensitivity and specificity you can make, so these can now be weighted by costs and/or population true positive/negative rates as appropriate to answer whatever question you want, in a way that single or sets of performance indicators (specificity, sensitivity, accuracy) can't, because they represent single points on the curve. (E.g. in the simple case of a threshold on your detection, those numbers represent a single choice of threshold, while the ROC curve shows what happens if you tune the threshold, and you can convert that into what happens to cost or FP/TP rates.)

Of course, it can be hard to obtain, you need a big number of observations of differing difficulty to do it with precision.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon