Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d59a8f3
Complete configured Volume operations in the native backend
aakash-saravanan-db Nov 13, 2026
018997c
Retain inline results for configured Volume transfers
aakash-saravanan-db Nov 13, 2026
0c45168
Correct Volume backend support documentation
aakash-saravanan-db Nov 13, 2026
2ed8d04
Pin the native source for Volume completion
aakash-saravanan-db Nov 14, 2026
28e4169
Select the Volume kernel and format its path option
aakash-saravanan-db Nov 14, 2026
15708ae
Regenerate Volume completion declarations from the pinned kernel
aakash-saravanan-db Nov 14, 2026
ab895d5
Preserve SQL execution errors during Volume completion
aakash-saravanan-db Nov 14, 2026
f910e8e
Pin native Volume storage retry and terminal response fixes
aakash-saravanan-db Nov 14, 2026
9038b5f
Cover portable native local file error decoding
aakash-saravanan-db Nov 14, 2026
9e6ef83
Preserve deferred Volume instruction HTTP errors
aakash-saravanan-db Nov 14, 2026
fee3d8f
Preserve portable file error fallback for unknown errno values
aakash-saravanan-db Nov 14, 2026
495ebe8
build(kernel): select corrected Volume completion
aakash-saravanan-db Nov 14, 2026
1b4aaaf
build: select native Volume cancellation and error handling
aakash-saravanan-db Nov 15, 2026
216fb20
build: use native root authorization fix
aakash-saravanan-db Nov 15, 2026
df53f88
build: update kernel source for replayable uploads
aakash-saravanan-db Nov 15, 2026
37c3d4a
build: update kernel for local stream close
aakash-saravanan-db Nov 15, 2026
dacfebb
fix(volume): select replayable file uploads and tighten validation
aakash-saravanan-db Nov 15, 2026
6b113b7
fix(kernel): use permission-preserving Volume downloads
aakash-saravanan-db Nov 16, 2026
f36e1f7
build(kernel): select sanitized storage redirects
aakash-saravanan-db Nov 16, 2026
1ea845f
Clarify Volume cancellation and consolidate routing coverage
aakash-saravanan-db Nov 16, 2026
3aef74a
Validate native file error numbers and select shared completion
aakash-saravanan-db Nov 16, 2026
30877d5
Pin kernel with statement lifecycle cleanup
aakash-saravanan-db Nov 16, 2026
3c9c3e6
build: select kernel with async result cleanup
aakash-saravanan-db Nov 16, 2026
e45bf99
build: select kernel with Volume instruction error telemetry
aakash-saravanan-db Nov 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

- Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608)
- Reduce result chunking and `fetchAll()` overhead for large query results.
- Support Volume file transfers on the kernel backend when `stagingAllowedLocalPath` is supplied. Symlink targets must remain within configured allowed roots; in-root symlinks are permitted.

- Thrift CloudFetch downloads are now bounded end to end by `retriesTimeout` (default 15 minutes), including every attempt and retry backoff. Previously a stalled download, where the response headers or body never arrived, blocked the fetch forever. A download still running when the deadline expires is aborted and fails with `RetryError` (`TIMEOUT_EXCEEDED`, message `CloudFetch result download failed: Retry timeout exceeded`). (#520)
- Thrift connections made through `proxy` now apply `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey`. Previously the proxied path ignored them and verified against Node's process-wide trust store, so a certificate outside the configured trust set could be accepted. **Compatibility:** proxied connections now verify unless `checkServerCertificate: false` is set; as on direct connections, `NODE_TLS_REJECT_UNAUTHORIZED=0` no longer turns verification off. For an `https` proxy these options also apply to the proxy's own certificate. (#485)
- Honor the client `useCloudFetch: false` default on the kernel backend so complete results can be fetched without cloud-storage downloads. (#574)
Expand Down
11 changes: 8 additions & 3 deletions CONNECTION_PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ telemetry implementation, so only the options explicitly forwarded by
| `maxRows` | `number \| bigint \| Int64 \| null` | ✅ | ❌ | `100000` / — | Thrift uses it for the initial Direct Results row count; `null` disables Direct Results. Kernel does not retain or forward the supplied value. |
| `useCloudFetch` | `boolean` | ✅ | ❌ | `true` / — | Kernel logs and ignores this per-statement hint; Kernel result fetching owns its CloudFetch behavior. |
| `useLZ4Compression` | `boolean` | ✅ | ❌ | `true` / — | Thrift uses it when supported and when the result is not CloudFetch. Kernel owns and auto-detects result compression. |
| `stagingAllowedLocalPath` | `string \| string[]` | ✅ | ❌ | — | Local allowlist for Thrift volume/staging operations. Kernel volume operations are unsupported. |
| `stagingAllowedLocalPath` | `string \| string[]` | ✅ | ✅ | — | Local allowlist for Volume operations. Kernel also requires resolved symlink targets within allowed roots; links within those roots are permitted. |
| `namedParameters` | `Record<string, DBSQLParameter \| DBSQLParameterValue>` | ✅ | ✅ | — | Named SQL parameters. Cannot be combined with non-empty `ordinalParameters`. Thrift requires a protocol that supports parameterized queries. |
| `ordinalParameters` | `Array<DBSQLParameter \| DBSQLParameterValue>` | ✅ | ✅ | — | Positional SQL parameters. Cannot be combined with non-empty `namedParameters`. |
| `queryTags` | `Record<string, string \| null \| undefined>` | ✅ | ✅ | — | Serialized into the per-statement `query_tags` conf overlay on both. |
Expand Down Expand Up @@ -228,11 +228,16 @@ session parameter surface.

Thrift-only capabilities include custom/token-provider authentication, custom
OAuth persistence, SOCKS proxies, authenticated/driver-buffer telemetry knobs,
query timeout, Direct Results `maxRows`, per-statement CloudFetch/LZ4 hints, and
volume operations.
query timeout, Direct Results `maxRows`, and per-statement CloudFetch/LZ4 hints.

Kernel-only capabilities include connection-pool sizing, proxy bypass hosts,
independent TLS hostname verification, optional U2M disk caching, and
per-statement `rowLimit`/`statementConf`.

The Kernel path remains internal and unstable; its behavior may change.

With the kernel backend, supplying `stagingAllowedLocalPath` completes Volume
transfers before `executeStatement` resolves and consumes instruction rows.
Omitting the option returns raw instructions without file effects. Unlike the
Thrift backend’s lexical path check, the kernel also rejects symlink targets
outside the configured roots while permitting symlinks within them.
2 changes: 1 addition & 1 deletion KERNEL_REV
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0b5d0edfed32c0745d5abc2468fc3070e2eb2a17
ab4f133620fce52f9932508f0b245db630a57c48
15 changes: 12 additions & 3 deletions lib/DBSQLSession.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,20 @@ export default class DBSQLSession implements IDBSQLSession {
// operation, everything will work as usual. In a case of staging operation, it will be processed like any
// other query - it will be possible to get data from it as usual, or use other operation methods.
if (options.stagingAllowedLocalPath !== undefined) {
const allowedLocalPath = Array.isArray(options.stagingAllowedLocalPath)
? options.stagingAllowedLocalPath
: [options.stagingAllowedLocalPath];
if (opBackend.completeStagingOperation) {
try {
await this.runBackend(() => opBackend.completeStagingOperation!(allowedLocalPath));
return operation;
} catch (error) {
await operation.close().catch(() => {});
throw error;
}
}
const metadata = await operation.getResultMetadata();
if (metadata.isStagingOperation) {
const allowedLocalPath = Array.isArray(options.stagingAllowedLocalPath)
? options.stagingAllowedLocalPath
: [options.stagingAllowedLocalPath];
return this.handleStagingOperation(operation, allowedLocalPath, metadata);
}
}
Expand Down
3 changes: 2 additions & 1 deletion lib/contracts/IDBSQLSession.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ export type ExecuteStatementOptions = {
* `false`/unset (default) runs the blocking direct-results path (faster,
* cancellable mid-compute); `true` submits and polls (returns a pending
* handle before completion). Default is sync, matching the python
* connector's `cursor.execute()`.
* connector's `cursor.execute()`. With `stagingAllowedLocalPath` configured,
* execution waits for staging completion regardless of this option.
*/
runAsync?: boolean;
maxRows?: number | bigint | Int64 | null;
Expand Down
2 changes: 2 additions & 0 deletions lib/contracts/IOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ export default interface IOperationBackend {
*/
getCachedResultMetadata?(): ResultMetadata | undefined;

completeStagingOperation?(allowedPaths: string[]): Promise<void>;

/** Cancel the operation. */
cancel(): Promise<Status>;

Expand Down
26 changes: 25 additions & 1 deletion lib/kernel/KernelErrorMapping.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { constants } from 'node:os';
import { getSystemErrorName } from 'node:util';
import HiveDriverError from '../errors/HiveDriverError';
import AuthenticationError from '../errors/AuthenticationError';
import OperationStateError, { OperationStateErrorCode } from '../errors/OperationStateError';
Expand Down Expand Up @@ -311,7 +313,29 @@ export function decodeNapiKernelError(err: unknown): Error {
const msg = envelope.message as string;
const sqlState = typeof envelope.sqlState === 'string' ? envelope.sqlState : undefined;

const jsErr = mapKernelErrorToJsError({ code, message: msg, sqlstate: sqlState });
let jsErr = mapKernelErrorToJsError({ code, message: msg, sqlstate: sqlState });
if (typeof envelope.localPath === 'string' && typeof envelope.localErrorKind === 'string') {
const kindCodes: Record<string, string> = {
NotFound: 'ENOENT',
PermissionDenied: 'EACCES',
AlreadyExists: 'EEXIST',
NotADirectory: 'ENOTDIR',
IsADirectory: 'EISDIR',
DirectoryNotEmpty: 'ENOTEMPTY',
};
const raw =
typeof envelope.localOsError === 'number' && Number.isSafeInteger(envelope.localOsError)
? envelope.localOsError
: undefined;
const systemCode = process.platform !== 'win32' && raw ? getSystemErrorName(-Math.abs(raw)) : undefined;
Comment thread
peco-review-bot[bot] marked this conversation as resolved.
Resolved
const localCode =
systemCode && /^E[A-Z0-9]+$/.test(systemCode) ? systemCode : kindCodes[envelope.localErrorKind] ?? 'EIO';
jsErr = Object.assign(new Error(msg), {
code: localCode,
errno: -((constants.errno as Record<string, number>)[localCode] ?? Math.abs(raw ?? constants.errno.EIO)),
path: envelope.localPath,
});
}

const meta = buildKernelMetadata(envelope);
// Skip the namespace attachment entirely when no fields validated
Expand Down
44 changes: 37 additions & 7 deletions lib/kernel/KernelOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,14 @@ import { LogLevel } from '../contracts/IDBSQLLogger';
import { RowMode } from '../contracts/IDBSQLSession';
import Status from '../dto/Status';
import HiveDriverError from '../errors/HiveDriverError';
import StagingError from '../errors/StagingError';
import ParameterError from '../errors/ParameterError';
import OperationStateError, { OperationStateErrorCode } from '../errors/OperationStateError';
import ArrowResultConverter from '../result/ArrowResultConverter';
import ResultSlicer from '../result/ResultSlicer';
import KernelResultsProvider from './KernelResultsProvider';
import { arrowSchemaToThriftSchema, decodeIpcSchema, patchIpcBytes } from './KernelArrowIpc';
import { decodeNapiKernelError } from './KernelErrorMapping';
import { decodeNapiKernelError, ErrorWithSqlState } from './KernelErrorMapping';
import {
KernelStatement,
KernelNativeAsyncStatement,
Expand Down Expand Up @@ -84,7 +86,10 @@ export type KernelOperationStatement = KernelStatementHandle & Partial<KernelSta
* (`KernelResultsProvider` → `ArrowResultConverter` → `ResultSlicer`) consumes
* either interchangeably.
*/
type KernelFetchHandle = Pick<KernelStatement, 'fetchNextBatch' | 'schema'>;
type KernelFetchHandle = Pick<
KernelStatement,
'fetchNextBatch' | 'schema' | 'isVolumeOperation' | 'completeVolumeOperation'
>;

/**
* The rich operation-status surface the kernel exposes on a terminal sync
Expand Down Expand Up @@ -220,6 +225,8 @@ export default class KernelOperationBackend implements IOperationBackend {
// already-terminal statement. Drives both fetch and result-metadata.
private fetchHandlePromise?: Promise<KernelFetchHandle>;

private volumeCompleted = false;

// Async path only: set once the kernel reported the statement Succeeded (a
// status poll, or `awaitResult()` resolving). The kernel exposes no
// synchronous terminal-state accessor, so the observed success is retained.
Expand Down Expand Up @@ -247,16 +254,15 @@ export default class KernelOperationBackend implements IOperationBackend {
this.cancellableExecution = cancellableExecution;
this.blockingStatement = statement;
// Lifecycle surface. The async/metadata handles expose both cancel/close.
// The sync-execute path uses a composite: `cancel()` always routes to the
// cancellable execution (lock-free, interrupts a running `result()`
// mid-compute and is a no-op once terminal); `close()` closes the resolved
// The sync-execute path uses a composite so SQL execution and subsequent
// transfer completion remain cancellable. `close()` closes the resolved
// terminal statement once `result()` produced it, OR — if `result()` is
// still in flight — proactively cancels the running execution so the server
// stops computing immediately rather than running on until the kernel's
// drop-guard fires whenever this handle is eventually GC'd.
this.lifecycleHandle = cancellableExecution
? {
cancel: () => cancellableExecution.cancel(),
cancel: () => (this.blockingStatement ? this.blockingStatement.cancel() : cancellableExecution.cancel()),
close: () => (this.blockingStatement ? this.blockingStatement.close() : cancellableExecution.cancel()),
}
: ((asyncStatement ?? statement) as KernelStatementHandle);
Expand Down Expand Up @@ -304,6 +310,28 @@ export default class KernelOperationBackend implements IOperationBackend {
// Fetch / metadata (owned by the kernel-results pipeline).
// ---------------------------------------------------------------------------

public async completeStagingOperation(allowedPaths: string[]): Promise<void> {
failIfNotActive(this.lifecycle);
const handle = await this.getFetchHandle();
try {
const outcome = await handle.completeVolumeOperation(allowedPaths, true);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — Nothing checks that the loaded native binding actually has completeVolumeOperation. The PR description says published native 1.4.0 doesn't have it. On that binding, any kernel executeStatement that passes stagingAllowedLocalPath will reject with TypeError: handle.completeVolumeOperation is not a function. That includes plain SELECTs, because DBSQLSession now calls completeStagingOperation for every statement with the option set. decodeNapiKernelError passes the TypeError through unchanged. Before this PR the same call logged a warning and continued.

If the binding can't be pinned to a compatible release when this merges, add a feature-detect: when typeof handle.completeVolumeOperation !== 'function', throw a clear StagingError/HiveDriverError stating that the native binding doesn't support Volume operations. That would replace the opaque TypeError.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: compatible published native packages are a merge prerequisite, including ordinary statements with stagingAllowedLocalPath. Source KERNEL_REV does not update installed platform dependencies. The matching release must supply completion, all platform pins/locks must advance, and clean installed queries plus Volume workflows must pass before merge. The accessible proxies currently expose 1.4.0, whose API is incompatible. Keeping the gate open; no old-native fallback or unavailable version is being introduced.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming. I agree that pinning a compatible published native release is the right fix, and that it has to land before merge. Until then, the code at this line still calls handle.completeVolumeOperation without checking it exists, so on 1.4.0 it would still fail with the opaque TypeError. I'm leaving this thread open as the merge gate. It can be resolved once the platform pins and lockfiles move to a release that includes completeVolumeOperation. If that release won't be ready by merge time, the typeof check plus a clear StagingError is still a cheap safeguard.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The release plan is coordinated native/connector version updates. Published 1.4.0 remains unsupported for these calls: before connector release, compatible platform pins and locks plus clean installed ordinary-query and Volume validation are required. Source validation selects kernel 855847bd with completeVolumeOperation. We are continuing review of that source pair while keeping package integration as a release prerequisite; no fallback, invented version or completed package proof is claimed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for laying out the release plan. Your reply doesn't challenge the finding. The code at line 318 still calls handle.completeVolumeOperation without checking that it exists, and the installed pins still resolve to 1.4.0. My last reply already said this thread stays open as the merge gate. It can be resolved once the platform pins and lockfiles move to a native release that has completeVolumeOperation, or once a typeof guard that throws a clear StagingError lands. Until one of those is pushed, I'm keeping it open.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — completeStagingOperation calls handle.completeVolumeOperation(...) without checking that it exists. The repo still pins the published native packages at 1.4.0, which the PR description says lack this API. On those binaries, every kernel statement run with stagingAllowedLocalPath would fail. That includes plain SELECTs, because DBSQLSession sends every such statement through completion whenever the backend has the method. The failure is TypeError: handle.completeVolumeOperation is not a function, it is rethrown unchanged by decodeNapiKernelError, and the operation is closed. Before this PR the option was a no-op with a warning, so this is a regression for anyone running a published build, while the CHANGELOG says the feature is supported.

If the pins can't be bumped in this PR, check typeof handle.completeVolumeOperation === 'function' first. When it is missing, either return the raw rows (the omitted-option behaviour) or throw a clear StagingError saying the native binary is too old.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed; this remains a merge prerequisite and also affects ordinary queries when the staging option is set. Published native 1.4.0 does not supply this completion API. The coordinated release must provide the compatible API, then all platform pins/locks must select it and clean installed-package ordinary-query/Volume workflows must pass. Source-built validation does not close that gate.

We are requiring the new native API rather than silently returning raw rows with an older binary or selecting an invented version. The PR description keeps this dependency unresolved; this comment is acknowledged, not claimed fixed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming this. Requiring the new native API and treating it as a merge gate makes sense, and so does not quietly falling back to raw rows. The current code still calls handle.completeVolumeOperation without checking that it exists, and the pins are still at the published 1.4.0, so I'm leaving this thread open until the coordinated release ships and every platform pin and lock moves to it. Consider adding a typeof check that throws a clear StagingError saying the native binary is too old. That stays within your no-fallback policy and gives users a readable error in place of a TypeError if a package mismatch slips through.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — KernelOperationBackend always defines completeStagingOperation, so the if (opBackend.completeStagingOperation) check in DBSQLSession passes for every kernel operation. That call then runs handle.completeVolumeOperation(...) unconditionally.

The installed native packages are pinned to 1.4.0, which has no completeVolumeOperation. With that binding, any kernel executeStatement that sets stagingAllowedLocalPath hits TypeError: handle.completeVolumeOperation is not a function. That includes a plain SELECT. decodeNapiKernelError passes the error through unchanged, and the operation is closed. Before this PR the option was a logged no-op, so code written for the Thrift backend that sets it on every call would start failing.

The merge plan relies on the pins being bumped later, but nothing in the code enforces it. A small guard would avoid the regression. For example, if typeof handle.completeVolumeOperation !== 'function', fall back to the old warn-and-ignore behaviour or throw a clear HiveDriverError saying the native binding is too old. A unit test with a handle that lacks the method would cover it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that published 1.4.0 is incompatible. This remains an explicit merge/release gate: select the real compatible native release, update all platform pins and locks, and validate a clean installed package on ordinary queries and Volume workflows before merging. The source pin and source-built tests do not complete that gate.

We require the new native APIs and a coordinated release; this change will not silently fall back to ignored staging or raw instructions. No fallback or invented version is being used to close the finding. It remains open until the compatible packages and installation proof exist.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming the 1.4.0 incompatibility. Requiring a coordinated native release instead of a silent fallback is a fair design choice. Right now, though, the gate exists only in the merge plan. At head, line 318 still calls handle.completeVolumeOperation with no check, and the native pins haven't changed, so this thread stays open until the compatible packages and pins land. If you'd like a cheap safety net that keeps your no-fallback stance, a typeof check that throws a clear HiveDriverError saying the native binding is too old would turn the raw TypeError into a useful message. It would also guard against a mismatched install after release.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — completeStagingOperation calls handle.completeVolumeOperation(...) with no check that the method exists. DBSQLSession sends every kernel executeStatement that has stagingAllowedLocalPath set through this path, including ordinary SELECTs, because KernelOperationBackend always defines completeStagingOperation.

The PR description says published native 1.4.0 has no completion API. With that binding (or any stale cached .node), these calls now fail with TypeError: handle.completeVolumeOperation is not a function, and the operation gets closed. Before this change, the option was logged and ignored, and the statement worked normally.

assertBindingShape was written to catch exactly this kind of drift, but it doesn't check the new surface. Two possible fixes:

  • check typeof handle.completeVolumeOperation !== 'function' here and throw a clear error, or skip completion;
  • or add the method to the binding shape check so a mismatched binding is rejected when it loads.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Published native 1.4.0 is incompatible; the compatible native versions, all eight platform pins/locks and clean installed-package ordinary-query/Volume checks remain required before merge. This is a coordinated release dependency, with no older-native fallback in the chosen contract. The source pin selects the current kernel, and source-built checks do not complete the package gate. Checking an old binding at load time could improve its diagnostic, but cannot make it compatible or replace that gate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the version pins are the real compatibility gate, and that a shape check can't make an old binding work. My concern is narrower: what happens when the pin gate is bypassed by a stale cached .node, a source build, or a hand-installed platform package. In that case every kernel executeStatement with stagingAllowedLocalPath set fails, including ordinary SELECTs, and the error is an opaque TypeError. Adding completeVolumeOperation to assertBindingShape is a one-line change. It would turn that into a clear error when the binding loads, which is the kind of drift that check exists to catch. The code at line 318 is unchanged, so I'm leaving this open for the maintainers to decide on. It's still low severity and doesn't block the coordinated release.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — completeStagingOperation calls handle.completeVolumeOperation(...) without checking that the loaded binding has it. The optional dependencies are still pinned to native 1.4.0, which has no completion API, so on the kernel backend every executeStatement with stagingAllowedLocalPath now fails, including a plain SELECT. The error is TypeError: handle.completeVolumeOperation is not a function. decodeNapiKernelError passes it through unchanged, and the facade closes the operation.

Before this PR the option was logged and ignored, so callers that set it globally and run ordinary queries break.

The facade check if (opBackend.completeStagingOperation) doesn't prevent this. That method exists on every KernelOperationBackend instance, so the check is always true. The check needs to be on the native handle. If typeof handle.completeVolumeOperation !== 'function', either keep the earlier behaviour (do nothing and return the raw instruction rows) or throw a clear HiveDriverError that names the minimum native version. Alternatively, bump the dependency pins in this PR.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Published native 1.4.0 is incompatible. The actual compatible versions, all eight platform dependency pins/locks, and clean installed ordinary-query/metadata/Volume proof remain required before merge; a source SHA pin does not satisfy those requirements. This is a coordinated-release dependency, with no older-native fallback in the selected contract. A stale-binding method check could improve the load/runtime diagnostic, but cannot make the missing API compatible or replace the package gate. The PR remains blocked on that gate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, a method check on the handle can't make native 1.4.0 compatible, and bumping the pins is the real fix. That's still outstanding, though: line 318 calls handle.completeVolumeOperation with no guard, and the optional dependencies still point to 1.4.0. I'm leaving this thread open until the new versions are published and all eight platform pins and locks are bumped. That way it stays in the unresolved queue as the visible merge gate. If the release slips and this branch gets installed against 1.4.0, a typeof guard that throws a HiveDriverError naming the minimum native version would still be cheap. It would replace an unclear TypeError on a plain SELECT.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — completeVolumeOperation is called without checking that it exists. The PR says published native 1.4.0 lacks the completion APIs, and the backend's own statement type is Partial<KernelStatement>. On a binding without the method, every kernel executeStatement that sets stagingAllowedLocalPath fails with TypeError: handle.completeVolumeOperation is not a function, and DBSQLSession then closes the operation. That includes ordinary SELECT/DML statements that aren't Volume operations; before this PR they ran with a warning. A typeof handle.completeVolumeOperation !== 'function' guard would fix it. It could throw a clear HiveDriverError/StagingError saying the native binding is too old, or fall back to the previous behaviour for non-volume statements (isVolumeOperation falsy).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The incompatible-package concern is valid and remains a required merge gate: compatible native platform versions, all dependency pins/locks, and clean installed SELECT/metadata/Volume proof. Native 1.4.0 lacks these APIs. The source build selects kernel30c8d8e7; both the rebuilt sync/async lifecycle comparisons and ordinary-query controls pass. Coordinated native uptake is required, so we are retaining direct calls rather than adding an older-native compatibility path. This PR is not ready to merge until that release gate passes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming. Requiring a coordinated native release instead of adding a compatibility path is a reasonable choice. The code at this line still calls completeVolumeOperation directly, though. Until the published native dependency is pinned to a version that has the completion APIs, an install that pulls native 1.4.0 will still fail ordinary statements that set stagingAllowedLocalPath. I'll leave this thread open so the release gate stays in the unresolved queue. It can be closed once the pin and lockfile point to a compatible native release, or once a guard is added.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The needs_input_stream branch has no unit test. Its StagingError is thrown inside the try, so it also goes through decodeNapiKernelError and the classification logic, and no test confirms it comes out unchanged. Any other unrecognised outcome string from a future native version is also treated silently as 'not completed', and the raw instruction rows are returned. Add a test case for needs_input_stream. Also consider throwing or logging when the outcome is not one of the known values (completed / not_volume / needs_input_stream).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sentinel instruction can reach this branch; it is not unreachable. Node has no input-stream upload API, so needs_input_stream produces the explicit StagingError. decodeNapiKernelError returns that ordinary Error unchanged, and it cannot satisfy either remapping condition. No incorrect branch behavior is established. An additional test of this direct rejection remains optional; core handshake/owned-byte coverage and distinct Node storage/instruction/error/lifecycle coverage remain. Unknown future outcomes retain the earlier optional disposition; supported native inputs require coordinated platform pins/locks.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that this branch is reachable, and your explanation of why decodeNapiKernelError should pass the StagingError through unchanged makes sense. My concern was never that the branch behaves wrongly today. It was that nothing locks that behavior in. If someone later changes the catch classification, this rejection could get wrapped or remapped without any test failing, and a single mocked completeVolumeOperation returning 'needs_input_stream' would cover that cheaply. For unknown future outcome strings, coordinated platform pins lower the risk but don't remove it: a version mismatch would still silently report "not completed". I'm leaving this open at Low severity for a human to decide whether to add the test and an else throw or log. No code change is needed to keep the current behavior correct.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — handle.completeVolumeOperation(...) is called without checking that the loaded binding has it. The packages declared in package.json are still 1.4.0, and per the PR description that version has no completion API. With that binary installed, any kernel executeStatement that passes stagingAllowedLocalPath fails with TypeError: handle.completeVolumeOperation is not a function. That includes a plain SELECT. Before this change the option was logged and ignored. decodeNapiKernelError returns non-envelope errors unchanged, and DBSQLSession then closes the operation and rethrows, so callers see the raw TypeError.

The PR does say merge depends on updated pins. Until that happens, though, every environment that installs from npm gets this regression. Either guard with typeof handle.completeVolumeOperation === 'function' and throw a clear StagingError or HiveDriverError (or keep the old no-op), as readStatusFieldsFrom does for older bindings, or block the merge on the pin bump.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: compatible native packages, all eight platform pins/locks and clean installed SELECT, metadata and Volume proof are required before this PR can merge. The declared 1.4.0 inputs do not satisfy the new API contract, and the source SHA pin is not released-package proof. This gate remains open in the PR description. Coordinated native uptake is required; an older-binding no-op/fallback would conceal unsupported Volume behavior and is not the selected contract. No merge-ready or complete-parity claim is made.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, blocking the merge on the native package bump is one of the resolutions I suggested, and choosing it over a silent no-op makes sense. Line 317 still calls handle.completeVolumeOperation with no guard, though, and package.json still pins 1.4.0. So the regression is real at current head. I'll leave this thread open as the visible merge gate. It can be resolved once the eight platform pins and lockfile point at a released binding that ships the completion API.

if (outcome === 'needs_input_stream') {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — No test covers the needs_input_stream outcome. The suite only exercises completed, not_volume and thrown errors. This branch throws inside the try, gets passed through decodeNapiKernelError, and must still come out as a StagingError. A single test case pinning that would stop a later refactor of the catch-side classification from silently turning it into a different error class.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a useful optional branch test, but source inspection does not establish a classification defect: the needs_input_stream branch creates a StagingError, and decodeNapiKernelError returns a nonsentinel Error unchanged (KernelErrorMapping.ts:270–274). The class therefore survives this catch. Keeping the existing focused tests and recording the uncovered branch; no behavior change is needed for this finding.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that today's code keeps the class: decodeNapiKernelError returns a non-sentinel Error unchanged, so the StagingError gets through. I wasn't claiming a current defect, though. This finding is only about missing test coverage. Nothing pins that pass-through, so if someone later changes the catch-side mapping (for example, wrapping or reclassifying every non-sentinel error), the needs_input_stream path could start returning a different error class and no test would fail. The fix is one test that mocks completeVolumeOperation to resolve 'needs_input_stream' and asserts StagingError. Since there's still no test, I'm leaving this open as a low-priority item for a human to decide.

Comment thread
peco-review-bot[bot] marked this conversation as resolved.
Resolved

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — No test covers the 'needs_input_stream' outcome, which throws StagingError('Volume PUT requires a local file path'). The Volume completion tests only exercise 'completed', 'not_volume' and thrown errors. Please add a case where the stub returns 'needs_input_stream'. It should assert that a StagingError is thrown and that volumeCompleted stays false, so later fetches still return the raw rows.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is unchanged in the source-pin update. volumeCompleted starts as false (KernelOperationBackend.ts:228), and the needs_input_stream branch throws before the only assignment at line 322, so it stays false and does not select the completed-result fetch shortcut. The non-enveloped StagingError is returned unchanged by decodeNapiKernelError, preserving its class and message.

The same branch was source-reviewed and adjudicated as optional test hardening in the preceding thread (#discussion_r4576690182); no runtime defect or required missing workflow was established. That disposition still applies to the unchanged code. Retaining the existing focused completion/error-provenance tests without another code/test push for this optional case.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your reading of the code: volumeCompleted stays false on that branch and the StagingError passes through decodeNapiKernelError unchanged, so I don't think there's a runtime bug today. The finding asked for a test, though, and nothing in the current code adds one. Right now only a source read guarantees this behaviour, so a later refactor could break it without any test failing. For example, it could move the volumeCompleted assignment above the check, or wrap the error in decodeNapiKernelError. It's still low priority and non-blocking. I'll leave the thread open so a maintainer can decide whether to add the test.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — No unit test covers the 'needs_input_stream' outcome. That branch throws a StagingError inside the try, which is then decoded again and run through the reclassification check. The Volume completion tests only exercise 'completed', 'not_volume' and thrown errors. Please add a case asserting that 'needs_input_stream' rejects with StagingError('Volume PUT requires a local file path').

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sentinel branch is reachable and intentionally rejects with StagingError('Volume PUT requires a local file path'). decodeNapiKernelError returns an ordinary non-envelope Error unchanged, and this StagingError satisfies neither remapping condition, so its class/message survive. This repeats the existing branch-test disposition. The extra mocked assertion remains optional; the distinct storage/instruction/error/lifecycle tests remain. No behavior defect is demonstrated.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your reasoning about the runtime behaviour. decodeNapiKernelError passes a plain non-envelope Error through unchanged, and this StagingError matches neither remapping condition, so it should reach the caller as is. My finding wasn't about a bug, though. It was about the missing regression test, and there's still no test under tests/ that mentions needs_input_stream or the message "Volume PUT requires a local file path". That makes the class and message you've described an unenforced contract: if the catch block's decode or reclassification logic changes later, nothing will catch it. One mocked completeVolumeOperation returning 'needs_input_stream' would cover it. It's low priority, so I'll leave the thread open for you to decide whether to add it before merge.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — No test covers the 'needs_input_stream' branch. In particular, nothing checks that the StagingError thrown inside the try passes through decodeNapiKernelError and the reclassification logic unchanged. The Volume completion tests cover only 'completed' and 'not_volume'.

There is also no tests/e2e/ coverage of kernel Volume PUT/GET/REMOVE, even though the change has visible effects on files and storage. The PR description mentions out-of-tree comparisons only.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sentinel branch is reachable, and the plain StagingError is passed through by decodeNapiKernelError; it matches neither remapping condition in this catch. The additional mocked assertion remains optional, with no incorrect runtime outcome established. Distinct native owned-input, adapter error/cancellation/lifecycle and real-driver transfer groups remain.

The shared end-to-end workflows are in databricks/databricks-driver-test#2122, tests/comparator-tests/node/config/thrift-vs-sea.yaml. The retained source-built sync/async DBSQL comparisons at kernel029c59cc have two matches, 20 successful positive steps and zero differences, including initial/changed bytes and storage-origin removal. Compatible installed-native and named scheduled proof remain open; duplicating that shared journey in another connector matrix would not close either gate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointing to the shared comparator workflows in databricks-driver-test#2122 is a fair answer to the e2e half of this, and I'm not claiming the current code behaves incorrectly. The unit-test gap is still open, though. The 'needs_input_stream' branch is reachable, but no test exercises it. That means nothing would catch a future change to decodeNapiKernelError or to the reclassification conditions that starts remapping or wrapping this StagingError. A small mocked test would cover it: stub completeVolumeOperation to resolve 'needs_input_stream', then assert that a StagingError with this message is rejected and that volumeCompleted stays false. This is Low severity, so I'll leave it to the maintainers whether it blocks the PR.

throw new StagingError('Volume PUT requires a local file path');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The needs_input_stream outcome, which should throw StagingError('Volume PUT requires a local file path'), has no test. The new suite covers completed and not_volume, plus thrown errors. It doesn't cover this branch, or the fact that the StagingError thrown inside the try goes through decodeNapiKernelError and the reclassification check before reaching the caller. A one-case addition to the existing for (const outcome of [...]) loop would cover it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed that this rejection branch has no dedicated test. Node exposes file PUT rather than a BinaryIO API; reaching it requires a server instruction for the stream sentinel. The branch throws an existing StagingError, and decodeNapiKernelError returns that non-envelope error unchanged. No runtime defect is established. A third row in the existing test table remains optional branch hardening; the required file lifecycle, completion/raw modes and error-origin paths retain unit and actual-driver coverage.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming. I agree the branch is hard to reach from Node's file-PUT API and that decodeNapiKernelError should pass a non-envelope StagingError through unchanged, and I'm not claiming a runtime defect. The finding is about the missing test, though, and that test is still missing. Your claim that decodeNapiKernelError and the reclassification check leave this error alone is exactly what a third needs_input_stream row in the existing outcome loop would prove. Without it, a later change to either one could quietly turn this rejection into a different error type. The fix is small and low priority, so I'll leave the thread open for a maintainer to decide whether to add it.

}
this.volumeCompleted = outcome === 'completed';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The native binding returns completeVolumeOperation outcomes as a bare string. This code handles 'needs_input_stream' and 'completed'. Every other value, including one a future kernel adds or one that is misspelled, gets the same treatment as 'not_volume': executeStatement resolves without an error and the caller gets the raw instruction rows. The user configured stagingAllowedLocalPath, so a transfer can silently not happen. A failure-loud check would fix this: accept only 'completed' and 'not_volume', and throw a StagingError such as Unexpected Volume completion outcome: ${outcome} for anything else.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compatible pinned NAPI producer maps the closed VolumeCompletion enum directly through as_str: its only outputs are not_volume, needs_input_stream and completed (src/volume.rs, napi/src/statement.rs, napi/src/async_statement.rs). This adapter handles all three, so no current native path produces the proposed unexpected value. A defensive check for a future or mismatched binding remains optional; compatible platform package pins/locks and clean installation are still required before merge.

The review-body nits remain acknowledged: preserving cause/extra metadata could improve StagingError diagnostics, while the current shape follows the audited Thrift error; the changelog blank line is cosmetic.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the pinned NAPI producer emits only not_volume, needs_input_stream and completed, and that all three are handled today. The concern was about a mismatched or future binding. Line 322 still treats any unrecognized string as not_volume and resolves without an error, and nothing in this package enforces the native enum, since the binding is typed as string and the platform packages are versioned separately. Throwing a StagingError when the outcome isn't completed or not_volume costs two lines and turns a silent skipped transfer into a visible error. It's still Low and optional, so I'm leaving the thread open for the maintainers to decide.

Comment thread
peco-review-bot[bot] marked this conversation as resolved.
Resolved
} catch (err) {
const mapped = decodeNapiKernelError(err);
const metadata = (mapped as ErrorWithSqlState).kernelMetadata;
if (
Comment thread
peco-review-bot[bot] marked this conversation as resolved.
Resolved
(mapped instanceof ParameterError && !metadata?.httpStatus) ||
metadata?.errorCode === 'VOLUME_STORAGE_HTTP_ERROR'
) {
throw new StagingError(mapped.message);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — When a transfer fails with a ParameterError or an error that has kernelMetadata.httpStatus, this code rethrows it as new StagingError(mapped.message). That has two problems:

  1. Kernel diagnostics are lost. The new error has no sqlState, kernelMetadata (HTTP status, queryId, retryable) or cause. The thrown error is the only failure signal callers get, and nothing else exposes these fields.
  2. The storage error's message reaches the caller unchanged. Presigned Volume URLs carry credentials in the query string. I could not check the kernel PR (Add disableRowMaterialization option to skip per-cell decode #465) to confirm whether its storage HTTP error messages include the request URL. The Thrift path avoids this by building its own message (HTTP error ${response.status} ${response.statusText}, lib/DBSQLSession.ts handleStagingGet/Put/Remove).

Possible fix: for HTTP failures, build the message from kernelMetadata.httpStatus instead of the kernel's text. In either case, copy kernelMetadata/sqlState/cause onto the StagingError, or set the mapped error as its cause.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against 8e93f23a and native 08a92137. The storage URL-exposure premise does not hold for these sources: non-success HTTP messages contain only the status, and reqwest transport/body failures use without_url() before formatting. No raw presigned URL is passed through this native path.

Preserving additional diagnostics on legacy StagingError remains the optional enhancement identified in the earlier same-source follow-up; it is not a new failure of the staging contract. No message rewrite or new diagnostic layer is needed for this finding.

The portable local-error branch now has four cases in the existing test: raw errno, absent/zero errno with PermissionDenied→EACCES, and unknown kind→EIO. They check negative errno, path, metadata/cause and absent SQLSTATE. The four cases, broader affected tests and typecheck pass.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking the native source at 08a92137. If non-success messages carry only the status and reqwest errors go through without_url(), then the kernel doesn't leak today. I can't confirm that from here, though, because the native code isn't in this repo. Line 329 is unchanged and still passes the kernel's message straight through, so this path's safety depends on how a separate codebase formats its messages. .claude/knowledge/driver-pitfalls.md is explicit: "Never copy a network error's message into an error the caller sees" (#567). The Thrift handlers follow that by building HTTP error ${status}. Building the message from kernelMetadata.httpStatus here would be a one-line change, and it would keep this path safe if the kernel's formatting changes later. Separately, the StagingError still has no cause or kernelMetadata, and that was the first point in the finding. Those two fixes make one small change, so I'm leaving this thread open for a maintainer to decide whether to accept the reliance on the native code.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — When a transfer fails with ParameterError or an HTTP status, the error is replaced by new StagingError(mapped.message). The new error has no kernelMetadata (including httpStatus and queryId), no sqlState and no cause, even though decodeNapiKernelError just attached them.

That means a caller can't tell a storage 403 from a 404 or other failure except by parsing the message. It also can't correlate the failure with a query id. The local-filesystem branch keeps that metadata, so transfer failures end up less diagnosable than the other error classes this PR separates out.

Fix: copy the metadata onto the wrapper, e.g. Object.assign(new StagingError(mapped.message), { kernelMetadata: mapped.kernelMetadata }), or set cause: mapped. The class change still happens, but the diagnostics survive.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8f800ba narrows conversion to the tagged VOLUME_STORAGE_HTTP_ERROR case and local ParameterError without HTTP metadata. Deferred instruction-fetch 401 and 400 now retain their mapped error identity, SQLSTATE and metadata; the existing backend fixture asserts those controls. Extra structured metadata on legacy storage StagingError remains an optional API enhancement: its existing Thrift surface is the staging class/message, which this adapter preserves.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Narrowing the conversion in 8f800ba fixes part of this: instruction-fetch 400 and 401 errors now keep their class, SQLSTATE and metadata. But line 330 still throws new StagingError(mapped.message) for the VOLUME_STORAGE_HTTP_ERROR path. Those are exactly the storage failures where httpStatus and queryId matter most, so callers still can't tell a 403 from a 404 without parsing the message. Matching the legacy Thrift class and message makes sense, but adding { cause: mapped } (or copying kernelMetadata onto the wrapper) keeps that and only adds information. It's a small additive change, so I'd still like to see it here rather than deferred. Severity stays low.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — When a policy or storage-HTTP failure is re-thrown as new StagingError(mapped.message), the decoded error's kernelMetadata, including httpStatus, errorCode (VOLUME_STORAGE_HTTP_ERROR) and queryId, is thrown away, along with its sqlState and cause. Callers then can't tell a storage 403 from a 404 or correlate the failure with a query, even though the PR keeps this metadata on the other error paths. Copying kernelMetadata and cause onto the StagingError (for example with Object.assign or Object.defineProperty, matching how KernelErrorMapping attaches them) keeps the established error class and the diagnostics.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional metadata/cause on StagingError would improve diagnostics. The current mapping deliberately retains the established Thrift staging error class/shape, while SQL instruction errors keep their decoded metadata. Native storage HTTP failures retain their specific Volume storage request failed: HTTP ... diagnostic, so the storage-origin deletion check does not depend on discarded metadata or accept a generic statement 404. This remains an acknowledged optional diagnostic enhancement.

The review-body constructor comment is also a valid optional wording improvement: cancellation uses the cancellable execution while pending and the resolved statement afterward. Existing cancellation/lifetime behavior is preserved.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood that keeping the established StagingError class is deliberate, and I'm fine with that part. My suggestion was never to change the class, only to attach the diagnostics to it. The code at line 330 still builds a new StagingError from the message alone. The HTTP ... text in the message helps a human reading it, but queryId, errorCode, sqlState and cause are still lost, so callers have nothing structured to branch on or use for correlation. This is Low severity and fine to defer. I'm leaving the thread open so the follow-up stays visible: one Object.defineProperty for kernelMetadata/cause on the new error would fix it without changing its class or shape.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — When a transfer failure is re-wrapped as new StagingError(mapped.message), the error loses kernelMetadata (including queryId and httpStatus/errorCode) and the non-enumerable cause that decodeNapiKernelError attaches. Other paths now keep these fields; the PR description says deferred instruction errors keep their metadata, and the tests check that too. For storage HTTP failures (VOLUME_STORAGE_HTTP_ERROR), callers can no longer see the HTTP status or the query id needed to match the failure in server logs. Consider copying kernelMetadata and cause onto the StagingError, for example with Object.defineProperty as the mapper already does. This keeps the error class callers expect and still preserves the diagnostics.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnostic loss is real and remains the previously acknowledged optional enhancement. Keeping kernelMetadata/cause would improve storage-error correlation without changing the StagingError class. The current class/message follows the audited Thrift staging contract; deferred SQL instruction failures retain their decoded metadata. The specific storage diagnostic is retained, and the strict deletion oracle rejects generic statement/instruction 404s without relying on dropped metadata. This finding remains visible; no required baseline behavior or newly demonstrated failure calls for another adapter change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming the diagnostic loss is real. Keeping the StagingError class and message for the Thrift staging contract makes sense, and the fix I suggested doesn't change that. It only adds the non-enumerable kernelMetadata/cause properties to the same error. The code at line 329 still drops them, so storage HTTP failures can't be matched to server logs by queryId or httpStatus. This is Low severity and isn't blocking. I'm leaving the thread open so the optional fix stays in the queue for someone to pick up later.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — When a transfer failure is reclassified, new StagingError(mapped.message) keeps only the message. The decoded error's kernelMetadata is dropped, including httpStatus, errorCode: 'VOLUME_STORAGE_HTTP_ERROR' and queryId, and so is its non-enumerable cause. Every other kernel error keeps that metadata, and the PR description says HTTP errors and their metadata stay intact. A caller that hits a storage 403/404 therefore gets a StagingError whose status can only be recovered by parsing the message, and there's no query id to correlate with the server.

A contained fix is to copy kernelMetadata (and cause) from mapped onto the new StagingError before throwing it. The tests could then assert that kernelMetadata.httpStatus survives in the VOLUME_STORAGE_HTTP_ERROR case.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the acknowledged optional storage-diagnostic enhancement. Storage failures retain the audited StagingError class and the specific storage diagnostic; deferred SQL instruction HTTP failures retain decoded metadata as stated in the description. The comparator’s REMOVE verification in databricks/databricks-driver-test#2122 requires storage-origin evidence and rejects generic instruction/statement 404s independently of these dropped properties. Copying kernelMetadata/cause would improve correlation, but no required Thrift behavior or newly demonstrated incorrect transfer/error outcome requires an adapter change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying. I accept that the PR description's metadata promise covers deferred SQL instruction HTTP failures rather than storage failures, and that the comparator in databricks-driver-test#2122 checks for storage-origin evidence without needing these properties. The code at line 329 still rebuilds the error from mapped.message alone, though, so a caller who catches the StagingError loses httpStatus, errorCode and queryId. The only way to recover the status is to parse the message, and there's no query id to correlate with the server. This stays Low and doesn't need to block the merge. I'll leave the thread open so a human can decide whether to copy kernelMetadata and cause over now or in a follow-up.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — Converting a tagged storage HTTP error or a local-policy ParameterError into new StagingError(mapped.message) drops everything the decoder attached: kernelMetadata (including httpStatus, errorCode: 'VOLUME_STORAGE_HTTP_ERROR' and queryId) and the non-enumerable cause.

After the rewrap, a caller cannot tell a 403 from a 404 except by parsing the message, and cannot link the failure to its statement. The deferred instruction errors next to this path keep their metadata intact (the tests check this), so these two error kinds now behave differently.

Consider copying kernelMetadata and cause onto the new StagingError, for example with Object.assign or Object.defineProperty, so the error type changes but the diagnostics survive.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the acknowledged optional storage-diagnostic enhancement. Storage failures retain the audited StagingError class and the specific storage diagnostic; deferred SQL instruction HTTP failures retain decoded metadata as stated in the description. The comparator’s REMOVE verification in databricks/databricks-driver-test#2122 requires storage-origin evidence and rejects generic instruction/statement 404s independently of these dropped properties. Copying kernelMetadata/cause would improve correlation, but no required Thrift behavior or newly demonstrated incorrect transfer/error outcome requires an adapter change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, and I agree this is Low severity and doesn't change Thrift behavior or transfer outcomes. Your reply also confirms the diagnostics are lost: line 329 still builds a bare new StagingError(mapped.message), so httpStatus, errorCode and queryId are gone for storage failures but kept for deferred instruction failures. The comparator in databricks-driver-test#2122 can work without them, but driver callers outside that harness can't. This is a two-line change that keeps the StagingError class: Object.assign(err, { kernelMetadata: metadata }) and define cause on the new error. I'll leave the thread open so a maintainer can decide whether to take it now or track it as a follow-up.

}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — When a transfer error is reclassified as StagingError, only message is kept. The decoded error's kernelMetadata (queryId, httpStatus, errorCode) and its non-enumerable cause are lost. A storage VOLUME_STORAGE_HTTP_ERROR then reaches the caller with no status code or query id to correlate, even though decodeNapiKernelError worked to attach them. Copying kernelMetadata and cause onto the new StagingError, for example with Object.defineProperty as the mapping module does, would keep the type change without losing diagnostics.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata/cause loss is real and remains the previously acknowledged optional diagnostic enhancement. The StagingError class and message follow the audited Thrift staging contract; SQL/instruction-side failures keep their decoded metadata. Genuine storage failures retain the specific storage diagnostic, and deletion verification rejects generic instruction/statement 404s without depending on discarded metadata. This repeats the earlier disposition; no new required behavior failure is established.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. We agree on the facts: the loss of kernelMetadata and cause is real, and this is a Low-severity diagnostic issue, not a functional failure. Line 329 still builds new StagingError(mapped.message) with nothing else attached. So for a VOLUME_STORAGE_HTTP_ERROR, the caller never sees the httpStatus, queryId or errorCode that the decoder already extracted. Copying those fields onto the new error wouldn't change its class or message, so it wouldn't conflict with the Thrift staging contract. I'm leaving this open so a maintainer can decide whether to add the fields in this PR or track it separately.

throw mapped;
}
}

public async fetchChunk({
limit,
disableBuffering,
Expand All @@ -316,6 +344,7 @@ export default class KernelOperationBackend implements IOperationBackend {
// Cancel-mid-fetch propagation: if cancel() has flipped the
// lifecycle flag, fail locally without a wire round-trip.
failIfNotActive(this.lifecycle);
if (this.volumeCompleted) return [];
// Cooperative cancel (parity with ThriftOperationBackend): the facade
// supplies `isClosed` and re-checks `failIfClosed()` after we return, so
// bailing with `[]` at a yield point is the contract-correct way for a
Expand All @@ -336,6 +365,7 @@ export default class KernelOperationBackend implements IOperationBackend {

public async hasMore(): Promise<boolean> {
failIfNotActive(this.lifecycle);
if (this.volumeCompleted) return false;
try {
// `hasMore()` prefetches the next batch, so it fails like `fetchChunk()`.
const slicer = await this.getResultSlicer();
Expand Down Expand Up @@ -379,7 +409,7 @@ export default class KernelOperationBackend implements IOperationBackend {
// `arrowSchema` doesn't hit apache-arrow@13's `Unrecognized type
// "Duration" (18)`. Matches what the per-batch fetch path already does.
arrowSchema: patchIpcBytes(arrowSchemaIpc.ipcBytes),
isStagingOperation: false,
isStagingOperation: Boolean(handle.isVolumeOperation),
};
this.metadata = meta;
return meta;
Expand Down
23 changes: 7 additions & 16 deletions lib/kernel/KernelSessionBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,10 +148,9 @@ export default class KernelSessionBackend implements ISessionBackend {
* `query_tags` key (the same wire shape Thrift's `serializeQueryTags`
* produces), merged with any explicit `statementConf`.
*
* Still rejected (genuinely unsupported on kernel, rather than silently
* dropped): `useCloudFetch` (governed by the kernel `ResultConfig`, not a
* per-statement knob), `useLZ4Compression` (kernel owns result compression),
* and `stagingAllowedLocalPath` (volume operations). `maxRows` is applied by
* `useCloudFetch` and `useLZ4Compression` are logged and ignored because the
* kernel owns result fetching and compression. `stagingAllowedLocalPath` is
* applied by the facade. `maxRows` is applied by
* the facade at fetch time, so it is intentionally not handled here.
*/
public async executeStatement(statement: string, options: ExecuteStatementOptions): Promise<IOperationBackend> {
Expand All @@ -160,8 +159,7 @@ export default class KernelSessionBackend implements ISessionBackend {
// Per-statement options the kernel backend doesn't honour are treated as
// NO-OPs (logged at warn), not errors — so call sites written for the Thrift
// backend are drop-in on the kernel path. These are perf/format hints the
// kernel governs internally (useCloudFetch / useLZ4Compression) or a feature
// it doesn't expose yet (staging); ignoring them cannot change query results.
// kernel governs internally (useCloudFetch / useLZ4Compression).
// NB: parameter binding (compound/BINARY) is deliberately NOT no-op'd — a
// dropped param would silently change results, so it still throws.
if (options.useCloudFetch !== undefined) {
Expand All @@ -180,15 +178,6 @@ export default class KernelSessionBackend implements ISessionBackend {
'kernel executeStatement: ignoring per-statement `useLZ4Compression` — result compression is governed by the kernel (no-op on kernel).',
);
}
if (options.stagingAllowedLocalPath !== undefined) {
this.context
.getLogger()
.log(
LogLevel.warn,
'kernel executeStatement: ignoring `stagingAllowedLocalPath` — volume/staging operations are not yet supported on the kernel backend (no-op).',
);
}

// `runAsync` selects the kernel execution path. NOTE: this is a kernel-
// specific use of the option — the Thrift backend hardcodes `runAsync: true`
// on the wire and never reads `options.runAsync`, so the field is a no-op
Expand All @@ -212,7 +201,9 @@ export default class KernelSessionBackend implements ISessionBackend {
// the backend polls `status()` to terminal in `waitUntilReady()` and
// materialises results via `awaitResult()`. `queryTimeout` is a no-op
// here too.
const runAsync = options.runAsync ?? false;
// Configured staging completes before the facade returns, so retain inline
// results even when the caller requests asynchronous execution.
const runAsync = (options.runAsync ?? false) && options.stagingAllowedLocalPath === undefined;
// `queryTimeout` is a NO-OP on the kernel (kernel) backend. It is the JDBC
// `setQueryTimeout` knob which — per the option's JSDoc — is effective only
// on Compute clusters; SQL Warehouses (what kernel targets) use the
Expand Down
6 changes: 6 additions & 0 deletions native/kernel/index.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading