Repository navigation
Conversation
Capture scan schemas and FileIO, validate projections and snapshot selection, and open generic snapshot tasks without depending on concrete metadata tables. Share partition projection and lazy manifest iteration; validate fallible row stream inputs and terminal state. Add focused support regressions and a reusable scan fixture. Co-authored-by: Codex <codex@openai.com>
This was referenced Nov 14, 2026
Include the Arrow schema definition in the standalone scan fixture and compare partition child counts with std::cmp_equal without narrowing. Co-authored-by: Codex <codex@openai.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Snapshot-backed metadata views need to retain their source metadata and FileIO, derive schemas from captured metadata, and reuse partition and manifest-reading logic. Add the common support needed by the focused FilesTable (#876), ManifestsTable (#1121), and PartitionsTable (#1122) follow-ups.
The scan builder obtains its schema from the captured metadata, and
schema()returns shared ownership by value. Generic snapshot tasks retain metadata, projection, FileIO, and a row-reader callback; the task reader opens them without depending on concrete table classes. Shared helpers resolve snapshot IDs/references/timestamps, unify partition specs, project partition values, and read live files one manifest at a time.The fallible row-stream overload validates its schema, source, callback, and batch size before exporting a C stream. Errors and exhaustion are terminal. Shared utilities are exported for both static and shared library consumers. The common scan fixture and focused regressions cover input errors, lazy batching, repeated failures, task ownership/reopening, projection validation, and partition evolution.
Validation: all 61 metadata tests and the C++20 compatibility test passed in Debug for both static and shared library builds with Arrow/Parquet 23.0.1. Pre-commit and diff checks passed. The complete split composition also passed 132 tests with one expected format-version skip, plus C++20 compatibility.