Repository navigation
Conversation
update() counted the item in n before processing a full base buffer, and processing allocated the next level. If that allocation failed, the base buffer stayed at 2k items and was never compacted again: memory then grew without bound, and serialize() to bytes threw. update() now compares and copies first, allocates everything next, and commits n, min and max only after the full base buffer is processed, as datasketches-java does since apache/datasketches-java#805. Merges: - allocate the target's levels, base buffer and scratch buffers before changing the target, so that an allocation failure leaves it unchanged; - set min and max first, then advance n with each level, so that a copy that throws from a const source leaves a valid sketch; - undo a partly copied level; - move instead of copy when the source is an rvalue, including the tails in merge_two_size_k_buffers(); - no longer move this sketch's items into a copy of the other sketch before the allocations that could fail; - remove levels that a failed update or merge added (the iterator expects none above the highest level in use); - restore the capacity of empty levels after levels_ grows: with an allocator whose moves can throw, growing levels_ copies the levels. A comparator that throws while items are sorted or merged leaves the sketch undefined, as decided for Java; the docs say so. optional::emplace() now forwards its arguments. It copied instead of moving while declaring noexcept for a move, so a copy that threw called std::terminate. Results are byte-identical to before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 44594066230Coverage increased (+0.06%) to 96.948%Details
Uncovered Changes
Coverage Regressions5 previously-covered lines in 2 files lost coverage.
Coverage Stats
💛 - Coveralls |
proost
approved these changes
Nov 20, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
update()counted the item in n before processing a full base buffer, and that processing allocates the next level. If the allocation failed, the base buffer stayed at 2k items and was never compacted again. From then on the base buffer grew without bound, andserialize()to bytes threw. This affected every item type,doubleincluded. The merges had similar gaps when an allocation failed or an item's copy threw.This follows apache/datasketches-java#805, which fixed the same flaw in Java's
QuantilesItemsSketch.Changes to
update():Changes to the merges:
merge_two_size_k_buffers(), which copied even before.levels_grows, empty levels get back their capacity of k. With an allocator whose moves can throw, growinglevels_copies the levels, and a copy of an empty level has no capacity. Before this change, that only went unnoticed because of the order of the steps.A comparator that throws while items are sorted or merged, or that is inconsistent, leaves the sketch undefined. That matches the decision for Java in #805, and the docs of
update()andmerge()now say so. Java sorts a copy of the base buffer so that a throwing comparator leaves it as it was. In C++ that would copy every item of the base buffer, so this PR doesn't.optional::emplace()(in common) now forwards its arguments. Before, it copied when it should have moved, while declaring itselfnoexceptfor a move. A copy that threw therefore calledstd::terminate, and that affected every sketch that uses it.Tests, in a new
quantiles_sketch_allocation_test.cpp:const&and rvalue merges. The target must be unchanged, and repeating the merge must give the same bytes as a merge that never failed.const&merges. The sketch must stay consistent and usable, with no leaks.All four fail on master and pass here. Serialized results are byte-identical to master; I checked 1,296 seeded merges with k = 4, 16 and 128, covering all merge paths,
const&and rvalue merges, and double and string sketches. All tests pass under ASan and UBSan.🤖 Generated with Claude Code