Repository navigation
Conversation
Reverts 0c327b0 ("Defer aiqr/cdf dispersion options to 0.4.0") on top of the 0.3.0 release work, bringing back dispersion_stat="aiqr"/"cdf", cdf_grid_size, their tests, and DistanceAnomaly's all-NaN metric-column drop. Before releasing as 0.4.0, consider: fit() should reject unknown dispersion_stat values (currently only predict() raises), and the aiqr docstring caveat ("a score rather than a metric distance in a fixed space") reads inconsistently with the cdf one - both are fixed monotone per-class maps.
DistanceMetricClassifier.fit() now rejects an unknown dispersion_stat or
central_stat and a cdf_grid_size < 2 with a clear ValueError. Previously
an unknown dispersion_stat was accepted and only failed at predict(),
and an unknown central_stat crashed inside pandas ("Empty data passed
with indices specified."). DistanceAnomaly and EnsembleDistanceClassifier
inherit the check through the classifier they fit.
The aiqr caveat called the result "a score rather than a metric distance
in a fixed space"; like cdf, aiqr is a fixed continuous monotone
per-class reparametrisation, so the metric axioms hold in the
transformed space.
Closes #24 together with "aiqr". "astd" is the second-moment counterpart of "aiqr": per class and feature, deviations above the centroid are measured in units of the upper semi-deviation (RMS deviation of the class's training objects above the centroid) and deviations below it in units of the lower one. On symmetric data both sides equal the standard deviation. Each side is floored at 5% of the full std so a one-sided feature does not blow up the scale (same rationale as the aiqr floor against the IQR). aiqr and astd now share one _asymmetric_rescale helper (unit-reference convention unchanged). Also update DistanceAnomaly's dispersion_stat docstring, which still listed only std/iqr.
predict(), predict_and_analyse() and DistanceAnomaly.decision_function() each had their own copy of the per-class scale-then-cdist loop (the classifier ones also special-cased scale=False). Route all three through DistanceMetricClassifier._distance_matrices(), which loops classes on the outside and reuses each scaled copy across metrics. No behaviour change: bit-identical predictions and distances to 0.3.0 across 43 metrics x 4 scaling configs x 4 datasets (688 cases), and identical DistanceAnomaly scores (32 cases).
Closes #25. New `reference` parameter on DistanceMetricClassifier and DistanceAnomaly. "centroid" (default) is unchanged: bit-identical to 0.3.0 on 688 classifier and 32 anomaly cases. With "knn", the distance from a test object to a class is the mean distance to the class's `n_neighbors` (default 5) nearest training objects, in that class's scaled space and with any metric. It follows the shape of the class's distribution rather than a single prototype and acts as a local density estimate: on a toy class with two modes straddling another class, accuracy goes from 0.49 (centroid) to 1.00. n_neighbors=1 is the nearest-training-object distance; n_neighbors >= class size is the mean distance to the whole class. - Query rows are processed in blocks (at most 2**22 distances in memory). - DistanceAnomaly.fit computes its threshold scores leave-one-out: with the training objects counting themselves as neighbours the threshold is biased low (new inliers flagged: ~1.0 at k=1 and ~0.23 at k=5 for contamination=0.1, vs ~0.10 with the fix). - `reference` and `n_neighbors` are validated in fit(). Cost: O(n_test x n_train) distances, e.g. 5000 train x 2000 test x 20 features: classifier predict 2.5 ms -> 0.26 s; DistanceAnomaly with the 16 default metrics 0.02 s -> 17 s.
__init__ accepted **kwargs and stored them in self.kwargs, which nothing ever read, so a typo such as DistanceAnomaly(n_neighbours=5) was silently ignored (and dropped by sklearn's clone/get_params). Remove it; unknown arguments now raise TypeError like any other scikit-learn estimator.
decision_function() recomputed its per-metric normalisation (min-max scaling, NaN fill values, and which metrics are usable) from whatever batch it was given, so a sample's score depended on the other samples scored with it. With the default normalize_scores=True, any sample scored on its own always got 0: e.g. in 0.3.0 predict() on a single point far from all training data returned "inlier". It also made the predict() threshold (learned on training scores) inconsistent with scores of new batches. These are now learned once in fit() from the training scores (usable_metrics_, score_fill_, score_scaler_) and applied unchanged to every batch. Training-set scores, offset_ and predict() on the training data are bit-identical to 0.3.0 (72 configurations); new data can now score above 1 when it lies beyond the training range. Calibration of predict() on new inliers with normalize_scores=True and reference="knn", n_neighbors=1: 15.3% flagged at contamination=0.1 before, 11.0% now.
New dispersion options (aiqr, astd, cdf), the neighbour-based class distance (reference="knn"), parameter validation in fit(), and the DistanceAnomaly changes (unknown arguments rejected, score normalisation learned in fit).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24. Closes #25.
New options for
DistanceMetricClassifier(andDistanceAnomaly, which uses it):dispersion_stat="aiqr"/"astd"/"cdf"(Implement asymmetric scaling for dispersion_stats #24): asymmetric scaling.aiqrmeasures deviations above and below the median in units of the upper and lower quartile half-widths;astddoes the same with the upper and lower semi-deviations about the centroid;cdfmaps each feature through the class's empirical CDF.reference="knn",n_neighbors(Implement distances between density and point #25): the distance to a class can now be the mean distance to itsn_neighborsnearest training objects (in the class's scaled space) instead of the distance to its centroid, i.e. a local density estimate.reference="centroid"stays the default.fit(): unknowndispersion_stat,central_stat,reference, or invalidcdf_grid_size/n_neighborsnow raise a clearValueErrorinfit(previously some only failed atpredict, or inside pandas).DistanceAnomalyfixes:TypeError(they were silently swallowed by**kwargs).fitfrom the training scores. Previouslydecision_functionmin-max scaled each metric over whichever batch it was given, so a sample's score depended on the rest of the batch, and any sample scored alone got 0 (e.g.predicton a single far outlier returned "inlier").reference="knn", the training scores that set thepredictthreshold are computed leave-one-out.Notes:
_class_transform), and all distance computations (predict,predict_and_analyse,DistanceAnomaly) go through_distance_matrices. With the default options, predictions, distances and anomaly scores are bit-identical to 0.3.0 (checked on 43 metrics × 4 scaling configurations × 4 datasets, and 32 anomaly configurations).reference="knn"processes query rows in blocks to bound memory.Results on DistClassiPy paper and DiMMAD paper
reference="knn", n_neighbors=15improves all three tasks, multiclass 92.7% → 96.5%, binary 68.4% → 70.7%, one-vs-rest 93.1% → 94.7%. The new dispersion options do not help there.reference="knn"helps rare in-distribution anomalies (ELAsTiCC purity at 50: 16.5% → 34.9%; best method on ZTF) but lowers out-of-distribution purity on ELAsTiCC, so it is an option rather than a default.Bumps the version to 0.4.0.
Also note breaking changes:
DistanceAnomalyno longer accepts unknown keyword arguments, and itsdecision_functionscores for new data are now scaled by the training scores (scores on the training data,offset_andpredicton the training data are unchanged).