Metrics and Calibration

Learning objectives

• • Compare precision, recall, F1, ROC-AUC, PR-AUC, NDCG, and calibration
• • Explain why threshold tuning is not the same thing as model improvement
• • Relate evaluation metrics to product and operational constraints

Interview prompts

• • When would you optimize for recall over precision?
• • How do you know if a classifier's predicted probabilities are trustworthy?

Prerequisites