Skip to content

fix(audits): apply condition to the rows checked by z_score and valid_http_method - #6156

Open
DG47 wants to merge 2 commits into
SQLMesh:mainfrom
DG47:fix/audit-condition-z-score
Open

DG47 wants to merge 2 commits into
SQLMesh:mainfrom
DG47:fix/audit-condition-z-score

Conversation

@DG47

@DG47 DG47 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #6155

Two built-in audits did not apply the optional condition argument to the rows they check:

  • z_score applied condition only in the stats CTE that computes the mean and standard deviation. The outer query read every row of the model, so rows excluded by the condition were compared against statistics from a different subset and reported as failures. It also scanned the full model instead of the filtered rows.
  • valid_http_method had no condition default and did not reference @condition, so a condition passed to it was silently ignored. Every other built-in audit supports it.

This change adds @condition to the outer WHERE clause of z_score. It also gives valid_http_method the same condition default (TRUE) and @AND(..., @condition) filter that the other built-in audits use. Without a condition, both queries return the same rows as before. The _non_blocking variants are copies of these audits, so they get the fix too.

Test Plan

  • Added test_z_score_audit and test_valid_http_method_audit in tests/core/test_audit.py. They check the rendered SQL with and without a condition.
  • Added test_builtin_audits_condition_filters_audited_rows, which runs both audits against an in-memory DuckDB table where the condition excludes some rows. Before this change, z_score flagged the excluded rows and valid_http_method flagged rows outside the condition.
  • All three tests fail on main and pass with this change.
  • pytest tests/core/test_audit.py: 46 passed.
  • pytest -n 8 -m "fast and not cicdonly and not isolated" tests/core: 2105 passed, 2 skipped. 6 failures in tests/core/test_connection_config.py (mssql/fabric pyodbc) come from pyodbc not being installed locally, and they fail the same way on main.
  • SKIP=prettier,eslint pre-commit run --all-files: ruff and ruff-format pass. mypy passes on the changed files; across the tree it only reports import-not-found for optional extras (web, lsp) that are not installed in my local environment.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test) (ran the tests/core fast subset locally, see above)
  • My commits are signed off (git commit -s) per the DCO

Summary by CodeRabbit

  • Bug Fixes
    • The built-in HTTP method and z-score audits now apply configured conditions when identifying validation failures, so rows outside the condition are not reported as failures.
  • Tests
    • Added coverage confirming that both audits apply conditions when evaluating rows, including during SQL execution.

…_http_method

The z_score audit only applied `condition` to the CTE that computes the
mean and standard deviation. The outer query scanned every row of the
model, so rows excluded by the condition were still compared against
statistics computed from a different subset and reported as failures.

The valid_http_method audit had no `condition` default and did not
reference it in its query, so a `condition` passed to it was silently
ignored, unlike every other built-in audit.

Apply `condition` in the WHERE clause of both queries.

Signed-off-by: Dhruv Gupta <todhruvg@gmail.com>
@DG47

DG47 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@cmgoffena13 @mday-io would one of you be able to review this one? It fixes #6155: the z_score audit applied condition only in its stats CTE and still checked every row, and valid_http_method ignored condition entirely. The change adds @condition to the outer WHERE of z_score and gives valid_http_method the same condition default and filter the other built-in audits use, with rendered-SQL and DuckDB tests that fail on main. Happy to rebase or adjust if you'd like anything done differently.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 246742f1-9b20-4a56-a08a-e16cd271ed4f
Commits

Reviewing files that changed from the base of the PR and between f6527ff and fafe4c6.

Files selected for processing: 2
  • sqlmesh/core/audit/builtin.py
  • tests/core/test_audit.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The z_score and valid_http_method audits now apply condition when selecting reported failures. Added SQL-rendering and DuckDB tests verify the filtered audit results.

Changes: Audit condition filtering

Layer / File(s) Summary
Apply conditions to audit failures
sqlmesh/core/audit/builtin.py
The HTTP-method audit now includes condition in its failure predicate. The z-score audit applies condition to the rows checked for threshold violations.
Verify condition filtering
tests/core/test_audit.py
SQL-rendering tests cover both audits with and without conditions. A DuckDB test checks their conditioned results.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to fafe4

The conditioned audits restrict reported failures as intended. No actionable merge-blocking issue was identified.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the primary change: applying the condition to rows checked by the z_score and valid_http_method audits.
Description check Passed The description includes the required Description, Test Plan, and Checklist sections. It explains the behavior change, implementation, tests, results, and known local test limitations. The unchecked f…
Linked Issues check Passed Issue #6155 requires z_score to apply condition when computing statistics and when selecting rows, and requires valid_http_method to accept and apply condition. The reviewed diff adds `@condit…
Out of Scope Changes check Passed The changed production code is limited to the two audits named in issue #6155. The added tests verify their rendered SQL and condition-filtered results. These changes directly support the issue object…
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Full details: Description check

Explanation

The description includes the required Description, Test Plan, and Checklist sections. It explains the behavior change, implementation, tests, results, and known local test limitations. The unchecked full-test item is explicitly addressed with the fast-subset results and pyodbc limitation.

Full details: Linked Issues check

Explanation

Issue #6155 requires z_score to apply condition when computing statistics and when selecting rows, and requires valid_http_method to accept and apply condition. The reviewed diff adds @condition to the outer z_score filter. It adds the TRUE default and @AND(..., @condition) filter to valid_http_method. The supplied change summary reports rendered-SQL tests and a DuckDB execution test for both audits. The PR description also states that the _non_blocking variants are copies and receive these fixes.

Full details: Out of Scope Changes check

Explanation

The changed production code is limited to the two audits named in issue #6155. The added tests verify their rendered SQL and condition-filtered results. These changes directly support the issue objectives. No unrelated change is shown.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 2 files.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mday-io

mday-io commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Simple patch. Should be an easy merged. Kicked off the checks and we'll see what happens

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Built-in z_score audit ignores condition for the rows it checks, and valid_http_method ignores condition entirely

2 participants