APGC: Adaptive Prototype-Geometric Confidence

Official research code for Adaptive Mahalanobis Prototype Ratios for Machine-Learning Confidence Estimation.

Repository: https://github.com/pointers1/APGC

APGC is a post-hoc confidence score for probabilistic classifiers. It combines the classifier's probability margin with a bounded geometric ratio computed from adaptive class prototypes in a regularized Mahalanobis space.

Authors

• Shallom Danjuma Bitrus - Department of Mathematics, Taraba State University, Jalingo, Nigeria
  Email: shallombitrus7@gmail.com
• Peter Anthony - Department of Applied Informatics, Comenius University Bratislava, Slovakia
  Email: peter.anthony@fmph.uniba.sk
• Michael Sunday Olawuni - Ardova Plc, Lagos, Nigeria
  Email: olawuni.michael@gmail.com

Repository structure

.
├── scripts/
│   ├── run_experiments.py      # datasets, models, APGC and all baselines
│   ├── analyse_results.py      # aggregation, bootstrap intervals and tests
│   ├── benchmark_runtime.py    # query-time and storage benchmark
│   └── generate_assets.py      # figures and LaTeX result tables
├── tests/                      # lightweight unit tests
├── docs/METHOD.md              # mathematical definition and protocol
├── results/                    # generated CSV/JSON outputs
├── figures/                    # generated PNG/PDF figures
├── tables/                     # generated LaTeX tables
├── requirements.txt
├── environment.yml
├── CITATION.cff
└── LICENSE

The generated outputs under results/, figures/ and tables/ are committed so the reported numbers and plots can be inspected without rerunning the benchmark. They are also fully recreated by the scripts below.

Installation

Python 3.10 or newer is recommended.

Using venv

python -m venv .venv
# Linux/macOS
source .venv/bin/activate
# Windows PowerShell
# .venv\Scripts\Activate.ps1

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Using Conda

conda env create -f environment.yml
conda activate apgc

Quick verification

Run one dataset and one split:

python scripts/run_experiments.py --output results --datasets Iris --seeds 11

or:

make quick

This creates the experiment files under results/.

Reproduce the complete benchmark

From the repository root:

python scripts/run_experiments.py --output results
python scripts/analyse_results.py
python scripts/benchmark_runtime.py
python scripts/generate_assets.py

Equivalent Make commands:

make full
make analyse
make runtime
make assets

The full protocol uses 16 classification tasks, four classifiers and five stratified train/validation/test splits. Runtime depends on the machine and available CPU cores.

Classifiers

• Logistic regression: max_iter=1500
• Histogram-based gradient-boosted decision trees (HistGBDT): max_iter=150, learning_rate=0.08, max_leaf_nodes=31
• Random forest: n_estimators=200, min_samples_leaf=2
• Multilayer perceptron: hidden layers (48, 24), max_iter=250, early stopping, validation fraction 0.15

HistGBDT is used as a nonlinear boosted-tree baseline. Its histogram-based split search provides efficient and scalable training while preserving the flexibility of gradient-boosted decision trees. Unlisted parameters retain their scikit-learn defaults.

Main output files

• results/all_results.csv: per-dataset, split, classifier and method results
• results/dataset_model_aggregate.csv: seed-averaged statistical units
• results/method_summary_overall.csv: overall method summary
• results/pairwise_tests.csv: Wilcoxon tests with Holm correction
• results/bootstrap_cis.csv: paired bootstrap confidence intervals
• results/statistics.json: benchmark-level statistics
• figures/: publication-ready result plots
• tables/: generated LaTeX tables

Confidence methods evaluated

• APGC
• Adaptive prototype geometry
• Single Mahalanobis centroid
• Euclidean centroid
• Class-wise 1-nearest-neighbour distance ratio
• Probability margin
• Maximum probability
• Negative entropy

Tests

python -m pip install -r requirements-dev.txt
pytest -q

Reproducibility notes

• Scaling, covariance estimation, prototype fitting and hyperparameter selection use only the training and validation data.
• Test labels are used only for final evaluation.
• The selected prototype count is from {1, 2, 3, 5}.
• The APGC mixture weight is selected from 21 equally spaced values in [0,1].
• Random seeds are fixed in the scripts.
• Some tasks are explicitly labelled derived-real; they should not be described as independently collected datasets.

Citation

The repository contains CITATION.cff. Add the paper DOI or proceedings information to CITATION.cff when it becomes available.

License

MIT License. See LICENSE.