Skip to content

Live staging household parity checks fail: a US variable on the wrong entity, and a 90 s timeout on /calculate-full #4287

Open

Description

@anth-volk

What fails

The live staging household checks added in #4274 failed in the release run on cd513dacc (push.yml run 43075278725, job "Run live simulations against the staging candidate", step "Run live staging tests, one pytest worker per test"): 2 failed, 7 passed. Staging promotion depends on that job, so no release of the Public API reaches production until it passes.

1. test_a_v2_household_calculation_matches_api_v1[us-single-california]

The Public API's /us/calculate refuses the check's household with 400:

Household variable income_tax belongs on tax_units, not people, at household.people.adult.income_tax.

The household in tests/integration/test_live_v2_household_parity.py puts income_tax on the person. policyengine_us defines income_tax with entity = TaxUnit, and /calculate has refused a variable on the wrong entity since 3a59b4d14 (May 2026). The simulation service's household function has the same check (household_variable_wrong_entity). The household is wrong, not either calculation. The UK households put income_tax on the person correctly, since policyengine_uk defines it there.

2. test_a_v2_household_calculation_matches_api_v1[us-single-reform-added-variables]

httpx.ReadTimeout on the check's POST /us/calculate-full, after its v2 calculation had finished. The check's client allows 90 s per read. The candidate's request log shows the same request, received at 18:44:19.6 UTC, answered 200 after 223.8 s. The client gave up at 18:45:49.9, 90.3 s in.

That time is normal for /calculate-full, which calculates every variable the household leaves out. Production's /calculate-full requests from the past week took 50 s, 84 s, 248 s and 296 s, and some ended with Cloud Run's 504 at its 300 s request timeout. That timeout is CLOUD_RUN_TIMEOUT in .github/scripts/cloud_run_env.sh, the same for staging and production.

Fix

  • Put the US household's income_tax on its tax unit.
  • Give the parity check's calls to the Public API's own /calculate and /calculate-full a read timeout longer than the candidate's 300 s request timeout. The check then gets the candidate's own answer.
  • Add unit tests that would have caught both failures before a release:
    • each parity household passes the Public API's real calculation routes, input checks and situation parsing included, with the installed country models;
    • the check's request waits longer than CLOUD_RUN_TIMEOUT.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions