Skip to content

fix(deparser): preserve multi-line string literals in pretty mode and parenthesize indirection targets - #378

Merged
pyramation merged 6 commits into
mainfrom
feat/deparser-multiline-literals-indirection-parens
Nov 13, 2026
Merged

pyramation merged 6 commits into
mainfrom
feat/deparser-multiline-literals-indirection-parens

Conversation

@pyramation

@pyramation pyramation commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Two deparser bugs found by round-tripping a real migration (parse → deparse → parse, compare ASTs).

1. Pretty mode indented the inside of multi-line string literals (value silently changed)

CREATE TABLE t (memo text DEFAULT 'line one\nline two')
before:  memo text DEFAULT 'line one\n  line two'
after:   memo text DEFAULT 'line one\nline two'

Root cause: SqlFormatter.indent split on every newline. It now uses a new splitSqlLines, which only breaks at newlines the formatter emitted — never inside '...', E'...', $tag$...$tag$, quoted identifiers or comments. One fix for every statement type. The old per-call-site containsMultilineStringLiteral regex guard is removed: it missed most paths and also misfired on any two single-line literals separated by a newline, which skipped indentation entirely — hence the re-indented CASE / CTE / JOIN ... ON blocks in the pretty snapshots (now consistently indented).

plpgsql-deparser had the same defect in its own indent (function bodies); it now reuses splitSqlLines, exported from pgsql-deparser.

2. Parentheses around an indirection target were dropped (output did not re-parse)

SELECT (ARRAY[1, 2, 3])[2:3]
before:  SELECT ARRAY[1, 2, 3][2:3]      -- syntax error at or near "["
after:   SELECT (ARRAY[1, 2, 3])[2:3]

A_Indirection parenthesized only a fixed list of node types. Per the grammar, only a ParamRef takes indirection bare, so every other arg is now parenthesized (ARRAY, ROW, CASE, COALESCE, boolean/null tests, JSON/XML constructors, …).

Also fixed (found while broadening coverage)

  • Pretty ON CONFLICT ... DO UPDATE SET no longer has a trailing space.
  • Pretty CREATE TRIGGER emits EXECUTE FUNCTION like non-pretty, and no longer drops FROM <rel> on constraint triggers.
  • CREATE EXTENSION ... VERSION was emitted as NEW_VERSION ..., and ALTER EXTENSION ... UPDATE lost UPDATE; neither re-parsed.
  • EXTRACT('Year' FROM d) / EXTRACT('time zone' FROM d) were emitted as a bare upper-cased word (AST change or syntax error); a quoted field now stays quoted unless it round-trips bare.
  • PL/pgSQL CASE: a WHEN value containing a newline leaked the internal "__Case__Variable_N__" IN (...), and an ELSE with an empty body (ELSE NULL;) was dropped, changing behaviour to CASE_NOT_FOUND.

CI

  • Five packages had no CI job: pgsql-parse, plpgsql-parse, @pgsql/utils, @pgsql/cli, pgsql-types. All are in the matrix now. The last two had a test script but no test files (jest exited 1, so root pnpm test failed); they get real tests — a CLI parse → deparse → parse round trip, and field-metadata.json checked against the runtime schema.
  • scripts/check-ci-matrix.ts fails CI when a package with tests is missing from the matrix (or the matrix names a package without tests), so this can't recur.
  • New generated-files job runs pnpm build plus every generator and fails if the tree is dirty, so a fixture .sql can't be added without its tests and generated sources can't drift.
  • To make that check hold across releases, pg-proto-parser gets an includeHeaderVersion option (default true, published behaviour unchanged). The in-repo generators turn it off, so committed generated files no longer say pg-proto-parser@1.32.5 and go stale on every release.
  • pgsql-types/src/types.ts was stale against its metadata (npm run generate is not part of build); regenerated, and now covered by the check.

Docs: DEPARSER_USAGE.md claimed multi-line literals are preserved; the claim is now true and spells out exactly what is protected, with an example.

Fixtures (SQL → npm run kitchen-sink; both new files fail on main's deparser)

  • misc/multiline-string-literals.sql — 130 statements: defaults, column/table/domain CHECKs, policies, trigger WHEN, rules, INSERT/ON CONFLICT/UPDATE/DELETE/MERGE, CASE, CTEs, views, window/aggregate args, array/row literals, COMMENT ON, function bodies, E'...' and dollar quoting.
  • misc/indirection-parens.sql — 95 statements: every target kind × [i], [a:b], .field, .*, nested, across SELECT/DML/DDL.
  • misc/extract.sql — quoted EXTRACT fields.
  • plpgsql_deparser_fixes.sql tests 68–70 for literals and CASE inside nested PL/pgSQL blocks.

Beyond the fixtures, a throwaway hunt (not committed) appended a multi-line suffix to every string constant, and wrapped every select-list expression in [1], [1:2], .f1 and .*, across all ~24k existing fixture statements: no pretty-only failures and no indirection failures remain.

Snapshot/golden updates elsewhere (parse, plpgsql-parse, transform) are only EXECUTE PROCEDURE → EXECUTE FUNCTION and one SET a = x,b = y → SET a = x, b = y; pgsql-types/field-metadata.json is regenerated from the new fixtures.

Link to Devin session: https://app.devin.ai/sessions/48cf9b2b452f426abcdcd3cb3d591773
Open in Devin Desktop: https://app.devin.ai/desktop/session/48cf9b2b452f426abcdcd3cb3d591773?variant=devin
Requested by: @pyramation


🟢 Devin Review found nothing to fix · View findings · Follow Devin's work

Devin Review

@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@luxor-code-review

luxor-code-review Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Review complete. No blocking issues — 1 nitpick below.

🧹 Nitpicks (1) — 🟢 1 low
  • 🟢 Dead new_version handler in extension DefElem branch (deparser.ts:6298) — The PR added a new_version DefElem handler returning VERSION ... inside the branch guarded by CreateExtensionStmt/CreateFdwStmt/AlterFdwStmt parents (packages/deparser/src/deparser.ts:6298).

This PR extends the deparser with two new fixture suites — multiline string literals and parenthesized indirection (subscripting/field selection) — and adjusts output formatting accordingly. The main code changes replace the previous indent logic in the SQL formatter with a token-aware splitSqlLines scanner (exported from packages/deparser/src/index.ts) and update ALTER EXTENSION option emission in the deparser, with snapshots and fixture JSON regenerated to match. Two producer passes (the premium diff sweep, twice) failed to publish candidates; the diff was nonetheless fully covered by four correctness shards, an api-data pass, a performance pass, and a holistic cross-file review, whose candidates were all adjudicated against the source.

Files Change
__fixtures__/kitchen-sink/misc/*.sql, __fixtures__/generated/generated.json, packages/deparser/__tests__/kitchen-sink/* New fixture SQL and regenerated fixture JSON plus kitchen-sink tests for multiline string literals and parenthesized indirection
packages/deparser/src/deparser.ts, packages/deparser/src/utils/sql-formatter.ts, packages/deparser/src/index.ts Deparser emission updates (indirection parenthesization, extension options) and token-aware splitSqlLines line-splitting in the formatter, newly re-exported
packages/plpgsql-deparser/src/plpgsql-deparser.ts, packages/pgsql-types/src/field-metadata.json Minor PL/pgSQL deparser output tweak and regenerated field metadata
packages/{deparser,parse,plpgsql-deparser,plpgsql-parse,transform} snapshots/fixtures Regenerated pretty-print and roundtrip snapshots plus transform kitchen-sink output reflecting the new emission

One low-severity finding remains: an unreachable new_version handler added in the CreateExtensionStmt/CreateFdwStmt/AlterFdwStmt branch of the DefElem emission, which is only ever produced by ALTER EXTENSION ... UPDATE TO and already handled earlier in that path.

Reviewed commit: 2b646ac

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 findings. Devin automatically fixed all 2 findings.

Fixed by Devin (2)
  • Broken multiline searched CASE conditions — Unwrap only the synthetic Case__Variable_N IN (...) of a simple CASE; searched CASE conditions are emitted as written (fixture test 70).
  • Empty newline configuration hangs formatting — splitSqlLines returns the text unsplit when the newline option is empty.

View findings in Devin Review · Follow Devin's work

Devin Review

… generated files

- scripts/check-ci-matrix.ts fails CI when a package with tests is missing from the matrix
- @pgsql/cli and pgsql-types get real tests and matrix entries (their test script used to exit 1)
- pg-proto-parser: includeHeaderVersion option; in-repo generated sources no longer carry
  the generator version, so a release does not make them stale
- regenerate pgsql-types narrowed types from the current field metadata
- generated-files job now checks the whole tree after build + regeneration
@pyramation
pyramation merged commit 229f43b into main Nov 13, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant