[anth-volk]

Fixes #4273

Summary

Adds POST /v2/household-calculations and GET /v2/household-calculations/{calculation_id}, which calculate a household stored in the Public API through the simulation service's household jobs. Replaces #4092, whose migration chain, reads through the simulation service and module layout no longer fit master.

POST needs no account. In order, it:

1. refuses a country without household_enabled (422);
2. answers 404 for a household, or a named policy, not stored in that country;
3. refuses a household with no default_year (422);
4. refuses a policy that sets parameters the running catalog version no longer has, with 422 code: "missing_parameters" and the list in missing_parameters, calling nothing (the check is require_policy_parameters_present, reusable by the run routes);
5. submits the household, the policy, add_missing_variables, optional axes and optional spm settings to the simulation service.

The simulation service's answers map as follows:

Simulation service	POST answers
202 {job_id, status}, for any of its four statuses	202 with calculation_id, status, country_id, household_id, policy_id and year
422	422, with its reasons passed through
400, with its reason	503: "The simulation service refused the calculation (): "
401, 403, 404, 405, 413 or 429 (answered before its route runs), its own 503 naming its code, a connection never made, or a gateway token that can't be fetched	503: "The simulation service is unavailable ( )." No job of the submission will run.
A 500, an unreadable 202, or any other status (such as 200, a redirect, 408, 409 or 501)	502: "The simulation service may have created the household job, but it failed ( ). Submit the household calculation again; if the first request went through, this creates a second job."
A lost answer (read timeout, dropped connection), or a proxy's 502, 504, or 503 without the simulation service's code	502: "The simulation service may have received the request, but its answer was lost (<transport error, or a proxy answered >). Submit the household calculation again; if the first request went through, this creates a second job."

No route finds a household job without its ID, so a 502 asks the caller to submit again rather than check first. The simulation service's own text may hold stored data, and the route needs no account, so only a 400 or 422 passes its reasons on; every other message gives only its status and a code that is a fixed identifier (lower-case letters, digits and underscores, starting with a letter, at most 64 characters).

Every answer given after the simulation service was called carries X-PolicyEngine-Observability-Id with the ID of that attempt. Refusals before it carry none.

GET reads household_jobs through the v2 layers, as the runtime role, and never calls the simulation service. It answers:

• the status, the IDs, the year, the catalog version and both package versions, and the timestamps;
• on success, the calculated household in its stored key order, warnings, spm_config and spm_provenance, from the simulation service's single-policy result document (contract version 1);
• on failure, code, message, input_errors and spm_error from the failure document, or the job's error_message when it has none;
• 404 for an unknown ID, 500 for a succeeded job whose document can't be read, and 503 when storage is down.

SPM availability in the catalog. A new nullable JSON column, tax_benefit_model_versions.spm_availability (autogenerated revision 0c136a278f29 on 970be5d29283), is recorded at catalog publication from the simulation service's GET /versions. /v2/model-catalog serves it as spm: available: true or false once recorded, null while unrecorded. When /versions can't be read, publication still succeeds, and its evidence lists the versions still unrecorded.

Also:

• the route group in the migration registry, request logging and the regenerated migration contract documents;
• live staging checks, with a US and UK parity set compared with /calculate and /calculate-full, under a 30-minute job limit;
• SIMULATION_ENTRYPOINT_URL added to the publication step.

The call to the simulation service sits behind the HouseholdCalculationSubmitter protocol. The adapter, policyengine_api/libs/simulation_household_calculations.py, reuses SimulationEntrypointClient's HTTP client, so the gateway credentials and the request and observability ID headers come from there. Wiring the released generated client is one change, once PolicyEngine/policyengine-sim-api#1046 merges and the simulation service publishes a client release with the household routes: add policyengine-api-simulation-client>=<that release> and relock, then replace post_household_calculation's body with the generated submit_household_calculation.sync_detailed(...), as its docstring shows.

The request fields and the answers the adapter reads were checked against PolicyEngine/policyengine-sim-api#1046 at 5fe2a584: the request (country, household_id, policy_id, add_missing_variables, and axes and spm only when given), the 202 {job_id, status}, the 422 field list, the 400 and 503 {code, detail} (the adapter keeps the 400's reason, and of a 503 or 500 only its code, ignoring a 503's household_job_id), and the JSON 500 internal_error. When the simulation service cannot tell whether a job's call started, it now answers 202 pending rather than 503, which the adapter already reads as an accepted job.

Merge order

Merge only after the simulation service's /household-calculations route, PolicyEngine/policyengine-sim-api#1046, is deployed to staging. Until then, the live staging checks fail, and with them every Public API staging promotion and release.

Tests

• Each test was run individually, by node ID, and seen failing for the expected reason before its change: 124 node IDs across the build and three review rounds, 191 passed and 9 skipped (the live tests, which need API_BASE_URL).
• Route and service tests run through the real country, household and policy services on SQLite, including the catalog failure path.
• Adapter tests use the real HTTP client over a mock transport, covering each class of transport error and the ID headers.
• Integration tests on PostgreSQL 17 read a job as the runtime role and record SPM availability as the catalog writer.
• alembic -c alembic-v2.ini check: no new operations; one head, 0c136a278f29.
• lint-imports, the rules registry, the quality guards, the migration contract export, ruff, ruff format, shellcheck, uv lock --check and mypy (including the changed data/v2 files) pass.
• Review: three rounds and a follow-up of the fixes. Round 1 found two bugs: a catalog failure answered 500 instead of 503, and the live checks' merge order, now handled by merge order. Round 3 found one: a lost answer was answered 503, inviting a resubmission. All are fixed test-first, and the follow-up's findings (a stray warning, a proxy's 502 or 504, docs) are fixed.
• The 502 messages were reworded to ask the caller to submit again, test-first: 25 test cases across the adapter's four 502 cases and the OpenAPI description, seen failing on the old wording, then passing. One more review round found no bugs; its two docstring findings are fixed.
• After merging Send the CI server credential from push.yml's deployed-API tests #4265, which makes the deployed-API tests send the CI server credential, the parallel live check's own clients send it too. They come from deployed_api_client in tests/integration/deployed_api_credentials.py, which the api_client fixture now builds its client with as well, so both send the same credential and settings. Unit tests were seen failing first; two review rounds of this change found no bugs.
• Answers the simulation service may have given after creating the job (any status not listed above, and a 503 without its code) answer 502, and only a 400 or 422 passes on its text, test-first: the new cases were seen failing, then passing. Two review rounds found three bugs, all fixed test-first: a front-end 503 answered as "no job", the OpenAPI 503 still promising the simulation service's reason, and a deeply nested body answered 500 (each answer is now parsed once, by pydantic's parser).

Environment variables and secrets

Value	Provider	How it reaches the code
SIMULATION_ENTRYPOINT_URL	GitHub environment secret in staging and production	Runtime: the deploy job's env:, then deploy_cloud_run_candidate.sh, then the Cloud Run revision. Publication: the publish step's env: in seed-v2-database.yml (new), whose job declares environment: and is called with secrets: inherit.
GATEWAY_AUTH_ISSUER, GATEWAY_AUTH_AUDIENCE, GATEWAY_AUTH_CLIENT_ID, GATEWAY_AUTH_CLIENT_SECRET_RESOURCE	Organization secrets	The deploy step's env:, then the deploy script with GATEWAY_AUTH_REQUIRED=1. Unchanged.
V2_RUNTIME_DATABASE_URL_SECRET_RESOURCE, V2_DATA_WRITE_DATABASE_URL	Environment secrets	Unchanged wiring.
API_BASE_URL, STAGING_API_TEST_PROBE_ID	The deploy job's outputs	The live test step. STAGING_HOUSEHOLD_* and the poll interval are optional and use their defaults.
CI_SERVER_ACCESS_TOKEN	Requested in the live staging job by request_ci_server_token.sh (from #4265) from the staging environment's CI_SERVER_AUTH0_DOMAIN, CI_SERVER_AUTH0_CLIENT_ID and CI_SERVER_AUTH0_AUDIENCE variables and CI_SERVER_AUTH0_CLIENT_SECRET secret	The live test step's env:. The household checks send it through the api_client fixture, and the parallel check through deployed_api_client, one client per thread.

Not verified:

• that each environment's SIMULATION_ENTRYPOINT_URL names its own simulation service;
• that the runtime can read the gateway client secret from Secret Manager;
• that the Auth0 client holds the simulation service's simulation:submit permission;
• that GET /versions describes the v2 household functions' SPM support.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.61017% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.66%. Comparing base (8b906f8) to head (3dc1f8b).
⚠️ Report is 61 commits behind head on master.

Files with missing lines	Patch %	Lines
...licyengine_api/data/v2/catalog/spm_availability.py	89.65%	4 Missing and 5 partials ⚠️
policyengine_api/fastapi_routes/dependencies.py	28.57%	5 Missing ⚠️
...gine_api/libs/simulation_household_calculations.py	97.20%	1 Missing and 3 partials ⚠️
...ne_api/services/v2/household_calculations/types.py	97.65%	1 Missing and 2 partials ⚠️
...icyengine_api/data/v2/catalog/publication_types.py	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4274      +/-   ##
==========================================
+ Coverage   92.51%   92.66%   +0.15%     
==========================================
  Files         318      329      +11     
  Lines       18137    18783     +646     
  Branches     3002     3114     +112     
==========================================
+ Hits        16779    17405     +626     
- Misses        814      824      +10     
- Partials      544      554      +10     

Flag	Coverage Δ
integration	53.77% <57.16%> (+0.12%)	⬆️
unit	90.42% <94.45%> (+0.16%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.