Skip to content

feat: add enum support across Arcscript interpreters - #24

Merged
panayotisk merged 10 commits into
mainfrom
feat/arcscript-enums
Nov 14, 2026
Merged

panayotisk merged 10 commits into
mainfrom
feat/arcscript-enums

Conversation

@panayotisk

@panayotisk panayotisk commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Adds string- and integer-backed enums to the TypeScript, C#, and C++ interpreters. questStatus = QuestStatus.Active assigns the member's primitive backing value; assignments from raw/computed values, cross-enum comparisons, and attempts to mutate members fail during validation, including in unreachable branches.

Parsing and execution share one declaration context per operation. The common resolver distinguishes variables from enum members and preserves their stable IDs and backing types. Runtime variables are accessed by ID; defaults, overrides, resets, output, and returned changes remain primitive.

Review order

  1. refactor: share declaration resolution across interpreters introduces the common context and ID-based access, and exposes validation-only entry points in C# and C++.

  2. feat: add enum support across Arcscript interpreters adds definitions, member constants, semantic validation, public options/native bridge support, documentation, and CI coverage.

  3. fix: preserve enum declaration lookup and parse diagnostics separates scope/name lookup keys, normalizes absent enum IDs, preserves syntax errors and source locations, validates declarations before legacy ambiguity probes, and keeps native name-based helpers consistent after renames. Native identifier validation also reuses its compiled regex.

  4. fix: correct enum scope boundaries and diagnostics keeps variables of unnamed C# containers scoped, makes enum/reference columns use UTF-16 offsets across runtimes, owns native exception messages without leaking, initializes rewrite contexts only when needed, and restores native dotted-name shorthand after exact scope/name lookup.

  5. fix: preserve enum references and mutable exception diagnostics protects enum namespaces during legacy rewriting, refreshes native exception messages after public-field edits, tightens TypeScript location assertions, and clarifies raw-source offsets and scoped-export requirements.

  6. fix: synchronize exception diagnostics and rewrite scope checks makes native exception reads safe across threads while preserving field edits and copying, unifies rewrite scope checks without losing duplicate containers, and adds shared reset/location regressions.

Validation

  • TypeScript: build, lint, and 434 tests pass.
  • C#: build and 331 tests pass.
  • C++: CMake build and all 8 CTest groups pass.
  • Shared fixtures cover no-argument resetAll() for ordinary and enum variables, including board/component values, and raw-source columns after HTML entities and CRLF. The branch includes the C# no-argument reset fix from fix: handle no-argument resetAll calls in C# #26.
  • All runtimes exercise 173 shared enum fixtures, including dotted-name collisions, missing/null/empty enum IDs, and error locations in later code blocks, Unicode tags, and multiline HTML. Additional regressions cover declaration snapshots, C# error normalization/recovery, legacy rewriting, native name-based helpers, and bridge validation.
  • Native concurrent reads, cache refresh after field edits, copy/move, and injected allocation-failure probes pass AddressSanitizer/UndefinedBehaviorSanitizer with leak detection enabled.
  • A temporary +20 column mutation is rejected by all five affected TypeScript location fixtures; the source was restored after the check.
  • Whitespace checks pass. The parser changes originate in the shared grammar; generated parser sources are built by CI.

Compatibility and scope

Existing interpreter constructor calls and native bridge layouts remain supported. Native C++ consumers must rebuild against the updated headers/library. Variable::enumId is optional: an omitted/null enum ID means an ordinary variable; an explicit empty string is invalid. Legacy global names containing dots cannot shadow scoped declarations.

The enum contract documents the host's responsibility for validating incoming membership and the export/native mapping required by later integration stages.

Native integrations must include their own standard headers and Win64 strdup mapping where needed; the public exception header no longer supplies them transitively.

@pkg-pr-new

pkg-pr-new Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@arcweave/arcscript@24

commit: 02768e2

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of the enum + shared declaration resolution changes (two passes: Claude and GPT-6 Astra, both at xhigh).

All existing suites pass: 385 TypeScript, 290 C#, 8 CTest targets, including the 142 shared enum cases. Parsers regenerate cleanly for all targets.

Suggested before merge:

  1. Scope/name key collision in the resolver — a regression vs main (inline on ArcscriptContext.ts / ArcscriptContext.cs).
  2. One rule for absent enum references across runtimes: TS rejects enumId: null, C++ accepts "" (inline).
  3. C# reports a bogus enum error in place of the real syntax error (inline on ArcscriptParserBase.cs).

The other inline comments are smaller correctness, parity, and perf issues.

Cleanup (not blocking):

  • Enum type checks run inside grammar actions while the tree is still being built, with three hand-synced copies (TS/C#/C++). That is the root cause of the C# error-recovery bug; a single validation pass over a successfully parsed tree would avoid it.
  • Name-based lookup is now dead code: TS ArcscriptState.getVar, C# ArcscriptState.GetVariable/GetVarValue/SetVarValue(IdentifierDef) and ArcscriptVisitor.IdentifierDef, C++ IdentifierDef/varNameToID.
  • The ~200-word reserved-identifier list is copied into all three runtimes (with internal duplicates like if, is, and), and the fixture only tests a few entries. It could live in tests/enums.json, or each runtime's list could be asserted against it.
  • assertAssignable re-runs assertVariable on reset/resetAll arguments that identifier_list already checked (all three runtimes).
  • Every parse()/runScript() rebuilds and re-validates all declarations (C# also scans all boards/components), and parsesAsArcscript repeats that up to 3× per legacy-rewrite candidate.

Not caused by this PR (possible follow-up): comparing an enum value to a plain number (QuestStatus.Active == number) throws in C# and C++ but returns false in TypeScript. Conditions on negative integer enum values also give different results. Both gaps already existed on main.

Comment thread TypeScript/src/ArcscriptContext.ts Resolved
Comment thread CSharp/Interpreter/ArcscriptContext.cs Resolved
Comment thread TypeScript/src/ArcscriptContext.ts Resolved
Comment thread Cpp/src/ArcscriptTranspiler.cpp Resolved
Comment thread CSharp/Interpreter/ArcscriptVisitor.cs Resolved
Comment thread Cpp/src/ArcscriptHelpers.h Resolved
bool unique = definition.type == "string"
? strings.insert(std::any_cast<std::string>(member.value)).second
: integers.insert(std::any_cast<int>(member.value)).second;
if (!unique) throw ParseErrorException("Enum " + definition.name + " has duplicate backing values");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-runtime error parity: duplicate backing values say "has duplicate backing values" here but "has duplicate or missing members" in TS/C#. The type/empty-members check (L44) also runs before the scope-collision check (L46), so an enum with no members that also collides with a scope reports "must have string or integer members" in C++ but "conflicts with a variable scope" in TS/C#.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at dc49115: duplicate backing values still say "has duplicate backing values" in C++ vs "has duplicate or missing members" in TS/C#. When an enum both collides with a scope and has no members, C++ still reports the empty members first while TS/C# report the scope collision. The unknown-variable fixture now matches only on the word "variable", so it no longer catches the C++ "Unknown variable" vs TS/C# "The variable X does not exist" difference.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at e0e3b6f. C++ still checks empty members before the scope collision (TS/C# check the collision first), still says "has duplicate backing values" vs "has duplicate or missing members", and still says Unknown variable "x" vs "The variable x does not exist". C++ reset() arity/type errors also carry no location (ArcscriptParserBase.cpp ~L176/180). The shared fixtures only pass because they match on the substring "variable".

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at 0ed9699, confirmed by running: duplicate-value messages, the unknown-variable wording (Unknown variable "x" vs "The variable x does not exist") and empty-members vs scope-clash ordering still differ. C++ reset(1) also throws "Invalid arguments type" with line 0, while TS reports "Invalid argument type for function reset. Expected variables. at line 2, col 0". The shared fixtures only match a substring.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at 8a0c46c, confirmed by running. C++ still says "has duplicate backing values" and Unknown variable "x", while TS/C# say "has duplicate or missing members" and "The variable x does not exist". For an empty enum that collides with a scope, C++ still reports the type/members error first. reset(1) / reset() still have no location in C++ (0:0, ArcscriptParserBase.cpp:176/180); TS reports 1:0.

Comment thread Cpp/src/ArcscriptContext.h Resolved
Separate scope/name keys, normalize absent enum IDs, and validate declarations before legacy rewriting. Preserve syntax failures and locate enum diagnostics, refresh native name-based lookups, and reuse the identifier regex. Add shared and runtime-specific regressions for the review findings.

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 0cdc5dc / dc49115 (Claude and GPT-6 Astra, both at xhigh). Thanks for the quick turnaround — most of the first round is fixed.

All suites pass: TypeScript 404/404, C# 312/312, CTest 8/8, including 158 shared enum fixtures.

First-round status (replies are on each thread):

  • Verified fixed: scope/name collision, TS enumId: null, C++ "" enum ID, C# misleading error after a syntax error, legacy rewriter probe, C++ name-based accessors after rename, C++ regex.
  • Partly fixed: error locations (see the new column comment below); C# Parse("") still throws, but it's now documented and tested as intentional.
  • Still open: C# ArcscriptVisitor default context, C++ validation messages and check order.

New, worth fixing before merge (both confirmed by running):

  1. C# promotes variables of a board/component with a null CustomId to globals (inline on ArcscriptContext.cs:81).
  2. Error columns mix units: Unicode in the opening tag gives negative (C#) or wrapped (C++) columns, and TS mixes a line-relative column with an absolute offset (inline on ArcscriptParserBase.cs:39 / .cpp:11).

Smaller, found by reading only (not reproduced):

  • C++ name-based accessors: exception type, dotted names, O(n) scan (inline on ArcscriptHelpers.h).
  • C++ ParseErrorException::what() leaks a strdup string whenever line > 0, and the message ends with std::endl (Cpp/src/ArcscriptErrorExceptions.h:54, outside the diff so noting it here). Pre-existing, but every enum/variable error now carries a line, so it's hit far more often. Storing the formatted string in a member at construction would fix both.
  • The TS legacy rewriter now builds and validates the context eagerly (inline on antlr4.interpreter.ts:167).

Still there from before this PR: comparing an enum value to a plain number throws in C#/C++ but returns false in TS, and conditions on negative integer enum values differ (false in C#/C++, true in TS). C++ also still resolves duplicate same-scope names last-match while TS/C# use first-match.

Comment thread CSharp/Interpreter/ArcscriptContext.cs Resolved
Comment thread Cpp/src/ArcscriptParserBase.cpp Resolved
Comment thread Cpp/src/ArcscriptHelpers.h Resolved
Comment thread TypeScript/src/antlr4.interpreter.ts Resolved
Keep unnamed C# container variables scoped and report enum/reference columns as UTF-16 block offsets. Reuse owned native exception messages, create legacy rewrite contexts only for ambiguity probes, and preserve dotted-name helper compatibility after exact lookup. Cover Unicode and multiline diagnostics, null scopes, lazy rewriting, and native error/helper behavior.

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3 re-review of e0e3b6f (Claude and GPT-6 Astra, both at xhigh).

All suites pass: TypeScript 417/417, C# 324/324, CTest 8/8 (310 fixture cases + 12 native checks). Astra also ran 1,080 location probes per runtime: surrogate pairs, multi-byte UTF-8, combining characters, CRLF, tabs, tag attributes, blockquotes, second/third blocks. Every enum/variable error column now matches across all three, with no negative or wrapped values. Nice work.

I've resolved the threads that are done and replied on the ones still open.

New, worth fixing before merge:

  1. The legacy rewriter can silently replace an enum constant with a renamed legacy variable. This has been in the PR since the first commit; we missed it in earlier rounds (inline on antlr4.interpreter.ts:168, confirmed by running).
  2. C++ what() returns empty or stale text after a host edits the public exception fields. This is a regression from e0e3b6f (inline on ArcscriptErrorExceptions.h:50, confirmed by compiled repro).
  3. The TS fixture location check is a substring match, so most column regressions pass (inline on enums.test.ts:52, confirmed by mutation).

Still open from earlier rounds (see thread replies): syntax-error coordinates still differ across runtimes; C++ name helpers' exception type and O(n) lookup; C# visitor default context (documented); C++ messages and check order.

Minor:

  • Docs claim the column matches editor offsets, but it counts raw HTML (inline on docs/enums.md:127).
  • The C++ dotted-name fallback can silently bind to a different variable (inline on ArcscriptHelpers.h:48).
  • Board/component variables with no custom ID are unreachable in C# but global in TS/C++ (inline on ArcscriptContext.cs:81).
  • Leftovers in Cpp/src/ArcscriptErrorExceptions.h (lines 3-11, outside the diff): now that what() no longer uses strdup/ostringstream, the <iostream>, <sstream>, <string.h> includes and the global #define strdup _strdup are unused. The macro leaks into every Win64 file that includes this public header.

Still there from before this PR: enum vs number equality throws in C#/C++ but returns false in TS; negative integer enum conditions differ; C++ uses last-match vs TS/C# first-match for duplicate same-scope names.

Comment thread TypeScript/src/antlr4.interpreter.ts Resolved
Comment thread Cpp/src/ArcscriptErrorExceptions.h Resolved
Comment thread TypeScript/src/__tests__/enums.test.ts Resolved
if (recognizer instanceof ArcscriptParser) {
column = column - recognizer.openTagEndPos;
if (offendingSymbol instanceof Token) {
column = offendingSymbol.start - recognizer.openTagEndPos;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enum/variable columns now agree everywhere, but syntax errors still don't:

  • A malformed second block reports 2:26 in TS and 2:36 in C#/C++ (confirmed by running). C# still leaves ParseException.Line/Column null for syntax errors, and C++ puts ANTLR's physical line and code-point column into the same fields its enum errors use for block line and UTF-16 offset.
  • offendingSymbol.start - openTagEndPos and currentLine only update when the codestart action runs. Errors found during prediction, before the opening tag is consumed, are reported against the previous block: <pre><code>show(1, x = 2)</code></pre> gives line 0, col 21, and a blockquote condition after another block gives line 1, col 43 instead of line 2, col 0 (found in TS by running).

Not a blocker if syntax-error coordinates are out of scope for this PR, but worth a follow-up issue so hosts get one coordinate system.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at 0ed9699: C# still leaves Line/Column null for syntax errors, and TS still reports <pre><code>show(1, x = 2)</code></pre> at 0:21 and <pre><code>show(1)</code></pre><pre><code>else</code></pre> at line 1, col 20 (against block 1, past its content), while C#/C++ give ANTLRs raw 1:31`. Fine as a follow-up issue if syntax-error coordinates are out of scope here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open at 8a0c46c, confirmed by running. <pre><code>show(1)</code></pre><pre><code>show(1, )</code></pre>: TS 2:8, C++ 1:50, C# Line/Column null (message says 1:50). <pre><code>show(1, x = 2)</code></pre>: TS line 0, C#/C++ line 1. Fine as a follow-up issue if it's out of scope here.

Comment thread docs/enums.md Resolved
Comment thread Cpp/src/ArcscriptHelpers.h Resolved
Comment thread CSharp/Interpreter/ArcscriptContext.cs Resolved
Protect enum namespaces during legacy rewrites without losing inferred or supplied scopes. Refresh owned exception messages after public field edits, keep what() noexcept under allocation failure, and remove unused public-header dependencies. Assert complete TypeScript error locations and clarify raw HTML offsets and scoped export requirements.

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 4 re-review of 0ed9699 (Claude and GPT-6 Astra, both at xhigh).

All suites pass, on the PR head and merged with main:

TypeScript C# CTest
PR head 0ed9699 421 324 8/8
merged with main (acd53fb) 421 326 8/8

Please merge main into the branch before merging. #26 (ab0a3e0, no-argument resetAll() in C#) merges cleanly and works with this PR. 22 focused reset/resetAll probes on the merged code all matched, covering ordinary, string/integer enum, and board/component variables. Until then, the PR head still hits the C# resetAll() null-reference.

I've resolved the threads that are fixed or now documented as intended, and replied on the two still open (syntax-error coordinates, C++ messages/check order).

New, worth fixing before merge:

  1. Concurrent what() calls can use freed memory. This is a regression from 0ed9699 (inline on ArcscriptErrorExceptions.h:22, confirmed with AddressSanitizer).

Minor:

  • The rewriter now blocks dotted legacy names whose first segment is an enum, even when the rest isn't a member (inline on LegacyVariableReferenceRewriter.ts:403, confirmed in TS).
  • Scope lists in the rewriter disagree (found by reading only, not reproduced). inherentlyBlockedCandidates (LegacyVariableReferenceRewriter.ts:227, outside the diff) still uses options.scopeNames ?? inferred and never sees the constructor's scopeNames. isAmbiguous now merges inferred, supplied and constructor names. So duplicate-scope detection can miss scopes that the ambiguity check honours, e.g. new Interpreter({ ..., scopeNames: ['Board', 'Board'] }) with no options. Computing the scope/namespace set once in rewrite() and passing it to both would fix it and avoid rebuilding it per candidate.
  • The ArcscriptErrorExceptions.h include cleanup is fine in-repo, but needs a release-notes line (inline on line 3).
  • Two coverage gaps: no shared fixture pins the column when an entity or CRLF precedes an error (inline on docs/enums.md), and none covers resetAll() with enum variables.

Still there from before this PR: enum vs number equality throws in C#/C++ but returns false in TS; negative integer enum conditions differ; C++ resolves duplicate names last-match vs TS/C# first-match.

Comment thread Cpp/src/ArcscriptErrorExceptions.h Resolved
]);
const dotIndex = tokens.tokenNames.indexOf('DOT');
const scope = dotIndex > 0 ? tokens.tokens[dotIndex - 1].text : '';
if (scope.length === 0 || scopeNames.has(scope)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the enum fix now blocks any dotted legacy global whose first segment is an enum name, even when the reference can never be enum syntax. With enum QuestStatus { Inactive, Active } and a legacy global named QuestStatus.Done, show(QuestStatus.Done) comes back with blockedVariableIds: ['legacy'] and unchanged code (confirmed in TS), so that script keeps failing with "variable does not exist". It errs on the safe side, so it's not a blocker. Checking the actual member names, or asking parsesAsArcscript with the declarations context for enum-prefixed candidates, would block only real member references.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed at 8a0c46c. The scope lists are now consistent: namespaces() (L232) feeds both checks. show(QuestStatus.Active) with a legacy global literally named QuestStatus.Active is still protected. But show(QuestStatus.Done) (Done not a member) is still blocked with blockedVariableIds: ["legacy"], because L411 checks only namespaces.has(scope). Confirmed by running. It errs on the safe side, so it's not a blocker. Leaving this open.

Comment thread Cpp/src/ArcscriptErrorExceptions.h Resolved
Comment thread docs/enums.md Resolved
Serialize native diagnostic cache access while preserving field edits, copy and move behavior. Share rewrite namespace checks and retain duplicate scopes from constructor and per-call declarations. Cover concurrent diagnostics, no-argument enum resets, and raw HTML entity/CRLF offsets; document native header compatibility.
Avoid assuming rethrow_exception preserves object identity, which is not guaranteed on MSVC. Exercise concurrent diagnostic reads and later field edits on the same exception on every platform.

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 5 re-review of 8a0c46c (Claude Opus and GPT-6 Astra). No new inline comments this round.

All suites pass (both reviewers, Linux):

TypeScript C# CTest
8a0c46c 434 331 8/8

Neither review found new bugs or cross-runtime differences from 9ccdd53, 8a0c46c or the merge with main.

Fixed, resolved:

  • Concurrent what(): the mutex-guarded cache holds up. The new test really shares one object across 8 threads, and the same test against the 0ed9699 header aborts with "double free" every run. A 16-thread stress test and ThreadSanitizer both came back clean. Pointer lifetime after edits is now documented.
  • Header release note: docs/enums.md:240.
  • Fixtures: exact locations after &gt;, CRLF and both, plus two resetAll() cases with enum variables. All three runtimes agree, as do extra probes (CR only, emoji, second block after CRLF).

Partly fixed: the rewriter's scope lists are consistent now. Unknown enum-prefixed names are still blocked (see thread).

Still open (see threads): syntax-error coordinates; C++ validation messages, check order and reset locations.

Still there from before this PR:

  • Comparing an enum to a plain number throws in C#/C++ but returns false in TS.
  • Conditions on negative integer enum values differ.
  • C++ resolves duplicate same-scope names to the last match; TS/C# use the first.
  • New, not enum-related: TS lists a variable whose starting value came from the host in changes even when the script never touches it. E.g. values {text: "changed"} with show(text) gives {"text":"changed"} in TS and {} in C#/C++.

@Zanzavar Zanzavar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 8a0c46c. All suites pass and CI is green on Linux, macOS and Windows. Everything blocking from earlier rounds is fixed and verified.

The three open threads aren't blockers and now have follow-up issues:

  • #27 Legacy rewriter blocks enum-prefixed names that aren't enum members
  • #28 Unify syntax-error coordinates across TS, C# and C++
  • #29 Align C++ validation messages, check order and reset() error locations with TS/C#

@panayotisk
panayotisk merged commit eca4588 into main Nov 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants