Latent

Choosing a threshold

Drag a spam filter's threshold and watch the confusion matrix, precision, recall, and ROC curve respond.

BeginnerExplained in Measuring a classifier

Choosing a threshold

A simulated spam filter gives each of 1000 emails a score from 0 to 1. Emails scoring at or above the threshold are flagged as spam.

  • Spam (above the line)
  • Good email (below)
  • Threshold (drag it)
Confusion matrix
FlaggedDelivered
Really spam275spam caught (true positives)25spam missed (false negatives)
Really good70good mail flagged (false positives)630good mail delivered (true negatives)

ROC curve. The dot is the current threshold; the dashed diagonal is random guessing.

0.50
2.5
30%
Accuracy90.5%
Precision79.7%
Recall91.7%
False positive rate10.0%
Area under ROC0.968

For comparison, a filter that flags nothing at all would be 70.0% accurate on this mix of email, and catch no spam.

Try this

  • Sweep the threshold from left to right and watch precision and recall trade places.
  • Lower the share of spam to 2% and compare accuracy with the filter that flags nothing.
  • Set model quality to 0, then to 5, and watch the ROC curve.

Try "embedding", "softmax", "overfitting", or "backpropagation".