Skip to content

fix: make the pg13–17 lanes round-trip every statement valid for their major - #381

Merged
pyramation merged 3 commits into
mainfrom
fix/pg13-17-round-trip
Nov 14, 2026
Merged

pyramation merged 3 commits into
mainfrom
fix/pg13-17-round-trip

Conversation

@pyramation

@pyramation pyramation commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #379 and #380. The generated pg13–17 packages (old AST β†’ @pgsql/transform-ast β†’ PG18 AST β†’ PG18 deparser) still failed parse β†’ deparse β†’ re-parse on some kitchen-sink statements. With this PR every statement that the major's own parser accepts round-trips, on all five lanes.

packages/deparser/src and packages/parser/src are not touched: PG18 output does not change.

1. transform-ast: fields the upgrade chain was dropping

These are plain bugs in the upgrade step; each fix copies or maps the missing field.

Step Node Was lost / wrong
13β†’14 ReindexStmt concurrent (now a concurrently param)
13β†’14 AlterFunctionStmt func.objname turned into { "0": … } when there are no args (ALTER PROCEDURE f() VOLATILE)
13β†’14 AlterSubscriptionStmt ALTER_SUBSCRIPTION_PUBLICATION not renamed to ALTER_SUBSCRIPTION_SET_PUBLICATION
14β†’15 CreatePublicationStmt / AlterPublicationStmt tables / tableAction not converted to pubobjects / action
14β†’15 AlterSubscriptionStmt enabled stayed an Integer, so DISABLE printed as ENABLE
15β†’16 ColumnDef, Constraint, GrantRoleStmt compression, fk_del_set_cols, grantor
16β†’17 ColumnDef storage_name, compression

This code only runs when upgrading a 13–17 AST; a PG18 AST never passes through it.

2. generate-version-deparsers.ts: old-server spellings, in the generated packages only

The PG18 deparser prints a few forms an older server cannot parse back to the same AST. Rather than change the PG18 deparser, the generated index.ts for the affected majors overrides them:

Majors Override
13–15 GRANT role … WITH ADMIN OPTION instead of WITH ADMIN TRUE
13–16 ALTER OPERATOR … SET (restrict = NONE) instead of SET (restrict)
13–16 keep a bare json type bare (the 16β†’17 upgrade qualifies it as pg_catalog.json)
13–15 keep json_object() bare (the 15β†’16 upgrade qualifies it)
13–14 TABLE a, b in publications instead of TABLE a, TABLE b
13 REINDEX TABLE CONCURRENTLY x instead of REINDEX (CONCURRENTLY) TABLE x
13 substring() always printed in its SQL form, the one spelling that re-parses to pg_catalog.substring

The json / substring overrides wrap methods of the per-step transformers held by the lane's PGnnToPG18Transformer (reached through its private v16to17 / v15to16 / v13to14 fields), so they depend on those field names.

3. config/versions.json

pgsql-deparser for 16 β†’ 16.2.1; 16.2.0 is already on npm without these fixes. The 17 lane's output is unchanged, so 17.19.0 stands.

Results

Clean regeneration and build of all lanes from this branch; ASTs compared ignoring location, stmt_location, stmt_len. Kitchen-sink is 24,021 statements, each run with pretty off and on.

Major 7 statements Γ— pretty off/on Pass Fail Not valid for that major
13 14/14 23,823 0 (was 94) 198
14 14/14 23,875 0 (was 38) 146
15 14/14 23,928 0 (was 38) 93
16 14/14 23,984 0 (was 15) 37
17 14/14 24,000 0 21

packages/deparser (316 suites, 424 snapshots) and packages/transform-ast (1,443 tests) pass unchanged.

Review notes

  • No tests were added for the new transformer behaviour or the generated overrides; CI still does not run prepare-versions, so the numbers above come from a local harness.
  • Parser lanes were tested with a pnpm override pointing pgsql-deparser at the local lane builds, since 13.20.0 / 14.2.0 / 15.2.0 / 16.2.1 are not on npm yet.

Link to Devin session: https://app.devin.ai/sessions/20f34d49f2df465b8a7c94defbd6ac9a
Open in Devin Desktop: https://app.devin.ai/desktop/session/20f34d49f2df465b8a7c94defbd6ac9a?variant=devin
Requested by: @pyramation


🟒 Devin Review found nothing to fix · View findings · Follow Devin's work

Devin Review

…r major

transform-ast: keep fields the upgrade chain was dropping (column COMPRESSION/STORAGE,
FK SET NULL/DEFAULT column lists, GRANTED BY, REINDEX CONCURRENTLY, publication tables,
ALTER SUBSCRIPTION kind/enabled, ALTER PROCEDURE name).

generate-version-deparsers: the generated pg13-16 packages override the few spellings
the PG18 deparser prints that an older server cannot parse back. packages/deparser/src
is untouched, so PG18 output does not change.

config: pgsql-deparser 16.2.1 (16.2.0 is already published without these fixes).
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@luxor-code-review

luxor-code-review Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Review complete. 🟑 2 medium

πŸ’¬ Inline comments (2)

🧹 Nitpicks (1) β€” 🟒 1 low
  • 🟒 16 lane deparser bumped without matching parser bump (versions.json:26) β€” config/versions.json:26-27 moves pgsql-deparser to 16.2.1 but leaves pgsql-parser at 16.2.0.

The change regenerates per-PG-version deparsers with targeted post-processing overrides and fixes several cross-version AST shape mismatches so PG13-14 SQL upgrades and re-deparses correctly for newer servers.

Files Change
packages/deparser/scripts/generate-version-deparsers.ts Adds string-templated overrides (KEEP_BARE_JSON_TYPE/OBJECT, OPERATOR_OPTION_NONE, GRANT_ROLE_ADMIN_OPTION, KEEP_CATALOG_SUBSTRING, REINDEX_CONCURRENTLY, PUBLICATION_TABLE_LIST) spliced into generated per-version deparser classes.
packages/transform-ast/src/transformers/v13-to-v14.ts Handles AlterFunctionStmt.objname arrays, renames the ALTER_SUBSCRIPTION_PUBLICATION kind, and converts ReindexStmt.concurrent into a concurrently DefElem param.
packages/transform-ast/src/transformers/v14-to-v15.ts Converts publication tables to pubobjects, maps tableAction to action, and migrates subscription enabled Integer options to Boolean.
packages/transform-ast/src/transformers/v15-to-v16.ts, v16-to-v17.ts Passes through newly added fields (compression, fk_del_set_cols, grantor, storage_name).
config/versions.json Bumps the PG16 lane's pgsql-deparser to 16.2.1.

Two medium-severity defects remain: the enabled Integer-to-Boolean conversion skips ival === 0 (DISABLE), and the publication table-list flag is never reset when object kinds interleave, producing SQL PG13/14 servers reject.

Reviewed commit: a9c9fc0

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

βœ… Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@luxor-code-review luxor-code-review Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds version-specific deparser monkey-patches and PG13-17 AST transformer fixes (subscription enable/disable, publication object lists, REINDEX CONCURRENTLY, GRANT ADMIN OPTION, bare json/json_object) plus a pgsql-deparser 16.2.1 version bump.

Key findings

if (result.kind === 'ALTER_SUBSCRIPTION_ENABLED' && Array.isArray(result.options)) {
result.options = result.options.map((option: any) => {
const ival = option?.DefElem?.arg?.Integer;
if (option?.DefElem?.defname !== 'enabled' || !ival) return option;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 bug · medium

ALTER SUBSCRIPTION ... DISABLE skips Boolean conversion

In packages/transform-ast/src/transformers/v14-to-v15.ts:1050, the guard !ival treats an Integer node with ival === 0 as absent, so ALTER SUBSCRIPTION ... DISABLE (enabled=0) keeps a raw Integer arg in the PG15 output where a Boolean node is required. Deparsers and consumers expecting the PG15 shape receive an unexpected node for the most common disable case.

πŸ“‹ Prompt for AI Agents

In packages/transform-ast/src/transformers/v14-to-v15.ts at line 1050, inside AlterSubscriptionStmt, replace the guard if (option?.DefElem?.defname !== 'enabled' || !ival) return option; with a check that does not treat 0 as missing, e.g. if (option?.DefElem?.defname !== 'enabled' || typeof ival?.ival !== 'number') return option; so Integer { ival: 0 } maps to Boolean { boolval: false } as intended for DISABLE.

Comment on lines +128 to +134
PublicationObjSpec(node: any, context: any): string {
const sql = super.PublicationObjSpec(node, context);
if (node.pubobjtype !== 'PUBLICATIONOBJ_TABLE') return sql;
const first = !this.publicationTableListed;
this.publicationTableListed = true;
return first ? sql : sql.replace(/^TABLE /, '');
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 bug · medium

Publication table-list flag breaks interleaved object kinds

The generated PublicationObjSpec override strips the leading TABLE from every object after the first PUBLICATIONOBJ_TABLE, keyed on a state flag that is only reset at CreatePublicationStmt/AlterPublicationStmt entry (packages/deparser/scripts/generate-version-deparsers.ts:128-132). For a publication whose objects interleave kinds, FOR TABLE a, TABLES IN SCHEMA s, TABLE b, the flag set by the first object is never reset, so the third is deparsed as bare b. PG13/14 grammar only accepts TABLE <list> or TABLES IN SCHEMA ... per object, so the round-trip silently emits SQL those servers reject.

πŸ“‹ Prompt for AI Agents

In packages/deparser/scripts/generate-version-deparsers.ts, in the PUBLICATION_TABLE_LIST template (lines 115-135), inside PublicationObjSpec, when node.pubobjtype !== 'PUBLICATIONOBJ_TABLE', set this.publicationTableListed = false before returning sql. This ensures a bare table name is emitted only when the previous object in the list was also a TABLE object; otherwise PG13/14 output like FOR TABLE a, TABLES IN SCHEMA s, b is unparseable.

@pyramation
pyramation merged commit 2a09d88 into main Nov 14, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant