Skip to content

New dispersion options, neighbour-based class distance, and DistanceAnomaly fixes - #43

Merged
sidchaini merged 8 commits into
mainfrom
aiqr-cdf
Nov 16, 2026
Merged

sidchaini merged 8 commits into
mainfrom
aiqr-cdf

Conversation

@sidchaini

@sidchaini sidchaini commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Closes #24. Closes #25.

New options for DistanceMetricClassifier (and DistanceAnomaly, which uses it):

  • dispersion_stat="aiqr" / "astd" / "cdf" (Implement asymmetric scaling for dispersion_stats #24): asymmetric scaling. aiqr measures deviations above and below the median in units of the upper and lower quartile half-widths; astd does the same with the upper and lower semi-deviations about the centroid; cdf maps each feature through the class's empirical CDF.
  • reference="knn", n_neighbors (Implement distances between density and point #25): the distance to a class can now be the mean distance to its n_neighbors nearest training objects (in the class's scaled space) instead of the distance to its centroid, i.e. a local density estimate. reference="centroid" stays the default.
  • Parameter validation in fit(): unknown dispersion_stat, central_stat, reference, or invalid cdf_grid_size / n_neighbors now raise a clear ValueError in fit (previously some only failed at predict, or inside pandas).

DistanceAnomaly fixes:

  • Unknown constructor arguments now raise TypeError (they were silently swallowed by **kwargs).
  • Score normalisation is learned in fit from the training scores. Previously decision_function min-max scaled each metric over whichever batch it was given, so a sample's score depended on the rest of the batch, and any sample scored alone got 0 (e.g. predict on a single far outlier returned "inlier").
  • With reference="knn", the training scores that set the predict threshold are computed leave-one-out.

Notes:

  • The per-class scaling now lives in one helper (_class_transform), and all distance computations (predict, predict_and_analyse, DistanceAnomaly) go through _distance_matrices. With the default options, predictions, distances and anomaly scores are bit-identical to 0.3.0 (checked on 43 metrics × 4 scaling configurations × 4 datasets, and 32 anomaly configurations).
  • reference="knn" processes query rows in blocks to bound memory.

Results on DistClassiPy paper and DiMMAD paper

  • Classification (light-curve paper, hidden set, F1 of the metric with the best cross-validated F1, features re-selected with SFS for each option): reference="knn", n_neighbors=15 improves all three tasks, multiclass 92.7% → 96.5%, binary 68.4% → 70.7%, one-vs-rest 93.1% → 94.7%. The new dispersion options do not help there.
  • Anomaly detection (DiMMAD paper splits): the paper's configuration gives the same results within ~1 point. reference="knn" helps rare in-distribution anomalies (ELAsTiCC purity at 50: 16.5% → 34.9%; best method on ZTF) but lowers out-of-distribution purity on ELAsTiCC, so it is an option rather than a default.

Bumps the version to 0.4.0.

Also note breaking changes: DistanceAnomaly no longer accepts unknown keyword arguments, and its decision_function scores for new data are now scaled by the training scores (scores on the training data, offset_ and predict on the training data are unchanged).

Reverts 0c327b0 ("Defer aiqr/cdf dispersion options to 0.4.0") on top of
the 0.3.0 release work, bringing back dispersion_stat="aiqr"/"cdf",
cdf_grid_size, their tests, and DistanceAnomaly's all-NaN metric-column
drop.

Before releasing as 0.4.0, consider: fit() should reject unknown
dispersion_stat values (currently only predict() raises), and the aiqr
docstring caveat ("a score rather than a metric distance in a fixed
space") reads inconsistently with the cdf one - both are fixed monotone
per-class maps.
DistanceMetricClassifier.fit() now rejects an unknown dispersion_stat or
central_stat and a cdf_grid_size < 2 with a clear ValueError. Previously
an unknown dispersion_stat was accepted and only failed at predict(),
and an unknown central_stat crashed inside pandas ("Empty data passed
with indices specified."). DistanceAnomaly and EnsembleDistanceClassifier
inherit the check through the classifier they fit.

The aiqr caveat called the result "a score rather than a metric distance
in a fixed space"; like cdf, aiqr is a fixed continuous monotone
per-class reparametrisation, so the metric axioms hold in the
transformed space.
Closes #24 together with "aiqr".

"astd" is the second-moment counterpart of "aiqr": per class and feature,
deviations above the centroid are measured in units of the upper
semi-deviation (RMS deviation of the class's training objects above the
centroid) and deviations below it in units of the lower one. On symmetric
data both sides equal the standard deviation. Each side is floored at 5%
of the full std so a one-sided feature does not blow up the scale (same
rationale as the aiqr floor against the IQR).

aiqr and astd now share one _asymmetric_rescale helper (unit-reference
convention unchanged). Also update DistanceAnomaly's dispersion_stat
docstring, which still listed only std/iqr.
predict(), predict_and_analyse() and DistanceAnomaly.decision_function()
each had their own copy of the per-class scale-then-cdist loop (the
classifier ones also special-cased scale=False). Route all three through
DistanceMetricClassifier._distance_matrices(), which loops classes on the
outside and reuses each scaled copy across metrics.

No behaviour change: bit-identical predictions and distances to 0.3.0
across 43 metrics x 4 scaling configs x 4 datasets (688 cases), and
identical DistanceAnomaly scores (32 cases).
Closes #25.

New `reference` parameter on DistanceMetricClassifier and DistanceAnomaly.
"centroid" (default) is unchanged: bit-identical to 0.3.0 on 688
classifier and 32 anomaly cases. With "knn", the distance from a test
object to a class is the mean distance to the class's `n_neighbors`
(default 5) nearest training objects, in that class's scaled space and
with any metric. It follows the shape of the class's distribution rather
than a single prototype and acts as a local density estimate: on a toy
class with two modes straddling another class, accuracy goes from 0.49
(centroid) to 1.00. n_neighbors=1 is the nearest-training-object
distance; n_neighbors >= class size is the mean distance to the whole
class.

- Query rows are processed in blocks (at most 2**22 distances in memory).
- DistanceAnomaly.fit computes its threshold scores leave-one-out: with
  the training objects counting themselves as neighbours the threshold
  is biased low (new inliers flagged: ~1.0 at k=1 and ~0.23 at k=5 for
  contamination=0.1, vs ~0.10 with the fix).
- `reference` and `n_neighbors` are validated in fit().

Cost: O(n_test x n_train) distances, e.g. 5000 train x 2000 test x 20
features: classifier predict 2.5 ms -> 0.26 s; DistanceAnomaly with the
16 default metrics 0.02 s -> 17 s.
__init__ accepted **kwargs and stored them in self.kwargs, which nothing
ever read, so a typo such as DistanceAnomaly(n_neighbours=5) was silently
ignored (and dropped by sklearn's clone/get_params). Remove it; unknown
arguments now raise TypeError like any other scikit-learn estimator.
decision_function() recomputed its per-metric normalisation (min-max
scaling, NaN fill values, and which metrics are usable) from whatever
batch it was given, so a sample's score depended on the other samples
scored with it. With the default normalize_scores=True, any sample
scored on its own always got 0: e.g. in 0.3.0 predict() on a single
point far from all training data returned "inlier". It also made the
predict() threshold (learned on training scores) inconsistent with
scores of new batches.

These are now learned once in fit() from the training scores
(usable_metrics_, score_fill_, score_scaler_) and applied unchanged to
every batch. Training-set scores, offset_ and predict() on the training
data are bit-identical to 0.3.0 (72 configurations); new data can now
score above 1 when it lies beyond the training range. Calibration of
predict() on new inliers with normalize_scores=True and
reference="knn", n_neighbors=1: 15.3% flagged at contamination=0.1
before, 11.0% now.
New dispersion options (aiqr, astd, cdf), the neighbour-based class
distance (reference="knn"), parameter validation in fit(), and the
DistanceAnomaly changes (unknown arguments rejected, score normalisation
learned in fit).
@sidchaini
sidchaini merged commit b5a6098 into main Nov 16, 2026
20 checks passed
@sidchaini
sidchaini deleted the aiqr-cdf branch November 16, 2026 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement distances between density and point Implement asymmetric scaling for dispersion_stats

1 participant