Skip to content

fix(density): use family ID 23 for serialization - #590

Open
tisonkun wants to merge 1 commit into
masterfrom
density-family-id-23
Open

tisonkun wants to merge 1 commit into
masterfrom
density-family-id-23

Conversation

@tisonkun

Copy link
Copy Markdown
Member

Closes #589.

density_sketch and ebpps_sketch previously shared family ID 19, so consumers could not distinguish the two sketches from the family byte alone. Following the cross-language discussion (apache/datasketches-tck#23), EBPPS keeps 19 and Density adopts 23 (0x17).

Summary of changes

  • Change density_sketch::FAMILY_ID to 23 so both stream and byte-buffer serialize entry points write 23.
  • Retain backward-compatible deserialization for legacy version-1 Density images carrying LEGACY_FAMILY_ID (19), while tightening check_header_validity to require only the IS_EMPTY flag bit alongside the Density preamble size (3 when empty, 6 when non-empty). EBPPS images (preamble longs 1 or 5) continue to be rejected.
  • Check stream state before header validation in stream deserialization, and validate version, family, and header consistency before checking k.
  • Add unit tests verifying family byte 23 on stream and byte-buffer serialization, round-tripping and upgrading legacy ID-19 images (empty, single-item, and estimation mode) to ID 23, and rejecting unrelated family IDs, EBPPS headers, and unexpected flag bits.

Verification

  • Built and ran density_test with -DGENERATE=ON, verifying all 11 test cases pass and all 12 generated density_{float,double}_n*_cpp.sk snapshots emit family byte 23.
  • Built and ran density_test with -DSANITIZE=address,undefined.

@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 44696511243

Coverage increased (+0.02%) to 96.906%

Details

  • Coverage increased (+0.02%) from the base build.
  • Patch coverage: 19 of 19 lines across 1 file are fully covered (100%).
  • 2 coverage regressions across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

2 previously-covered lines in 1 file lost coverage.

File Lines Losing Coverage Coverage
common/include/serde.hpp 2 98.02%

Coverage Stats

Coverage Status
Relevant Lines: 17422
Covered Lines: 16883
Line Coverage: 96.91%
Coverage Strength: 1878410.21 hits per line

💛 - Coveralls

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use family ID 23 for serialized Density sketches

3 participants