A list of papers that studies out-of-distribution (OOD) detection and misclassification detection (MisD)

PyTorch implementation of our ECCV 2022 paper "Rethinking Confidence Calibration for Failure Prediction"

A decision-safety lab for loan approval: trains a baseline classifier, calibrates probabilities (ECE/Brier), sweeps confidence thresholds to build a coverage, quality frontier and outputs a defensible abstention policy (auto-decide vs review). Includes a Streamlit dashboard for report cards, triage UI, and data quality checks.

Longform article reframing abstention (reject option / selective prediction) as product design, not model weakness. Covers coverage as a KPI, calibration as a prerequisite, threshold selection under review capacity and risk, queue/UX design for human-in-the-loop workflows, and anti-patterns that break safety in production.

Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.

Promises and Pitfalls of Threshold-based Auto-labeling (NeurIPS 2023, Spotlight)

Pearls from Pebbles: Improved Confidence Functions for Auto-labeling (NeurIPS 2024)

Code for "Budgeted Classification with Rejection: An Evolutionary Method with Multiple Objectives", at IEEE CEC 2022

Robust Selective Classification of Skin Lesions with Asymmetric Costs

More human oversight can make an AI agent less safe. Headroom is a human-in-the-loop firewall for coding agents that measures when to trust the human: oversight as resource allocation, not just classification.

Reliable and Efficient Selective Machine Learning

Confidence-aware calibration framework for blood agglutination image classification (MobileNetV2/EfficientNetV2 + AutoGluon, isotonic calibration, class-conditional review policy). Externally validated on NIH Malaria dataset. Paper + reproducible pipeline.

Event-level risk-controlled triage for conjunction data message streams

Safety-centered uncertainty evaluation for RETFound retinal diabetic retinopathy screening under dataset shift.

Official implementation of the ICLR paper "Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It"

Code for our paper analyzing the looseness of the upper bound on selective classification performance.

Official code for the paper "Adaptive Mahalanobis Prototype Ratios for Machine-Learning Confidence Estimation" — APGC, a post-hoc confidence score combining a classifier's probability margin with adaptive Mahalanobis prototype geometry. Includes the full 16-task benchmark, results and figures.

New Zealand-localized phishing and smishing detection with selective abstention. Combines a localized feature set (IRD, NZ Post, banks, te reo cues, .govt.nz lookalikes) with classical baselines, fine-tuned transformers, and LLM-as-classifier (Gemini), under a class-conditional deferral layer for human review. Independent research.

Missingness-aware abstention for selective classification under MCAR/MAR/MNAR label missingness.

TTETT — Today's Technology Empowering Tomorrow's Technology. A Continuant Learning Runtime for governed adaptation.