[anth-volk]

Fixes #1045

Summary

Adds POST /household-calculations and GET /household-calculations/{household_job_id} to the Simulation Entrypoint. They create and read household jobs, rows of the Public API's household_jobs table, and start the v2 Modal household functions (#903), which calculate one household for one year under one policy and record the result on the job. Replaces #957, which was built on a schema that is no longer used.

POST (permission simulation:submit) takes country, household_id, policy_id (omitted or null for current law), add_missing_variables, optional axes and optional spm settings. In order, it:

1. finds the household function of the v2 manifest's default worker for the country, and the catalog version whose .py version matches that worker;
2. reads the household, the policy and the policy's parameter values in one read transaction;
3. builds the household contract's request for the household's default_year, with calculation_id set to the new job's ID;
4. inserts the job pending, with no Modal call ID, dynamic_id null and request_data holding the request as received;
5. starts the household function's call with spawn, passing the job's ID, and waits up to 10 seconds for Modal to accept it;
6. records the call ID only while the job is still pending with none, and answers 202 {job_id, status}.

If Modal refuses the call, the job is failed at once under the same condition, with error_message and completed_at, and the answer is 503 modal_call_refused. If Modal accepts it late and the function has already claimed the job, the answer is 202 with the job's current status. If the job's state can't be recorded or read back, the answer is 202 pending: poll the job rather than submitting again. The stuck-run sweep fails a job no call ever started after about 10 minutes. Each submission creates a new job.

GET (permission simulation:read) answers 200 with job_id, status, country, household_id, policy_id, year, both package versions, result (the household function's result or failure document), error_message and the timestamps; 404 unknown_household_job.

Error answers

Status	When
400 {code, detail}	An unknown household, policy or catalog version; no household function for the country on the default worker; a foreign-key violation (SQLSTATE 23503) on insert, naming the missing row. Every referenced row is checked before the insert.
422	A malformed request; a household with no default_year or one that can't be read; a policy parameter missing from the catalog version; an explicit null spm; a request the household contract refuses, traced to the request field that set it. FastAPI's detail field list.
500 internal_error	Any unexpected failure, as JSON, including a contract refusal of a field the service builds itself. The log is the fixed event household_route_failed with the route, the error's type and its stack frames as one string, innermost first, within the observability runtime's string length limit (stack_frames_omitted counts the frames left out). It never carries the error's message or any value.
503 {code, detail}, Retry-After: 10	not_configured, manifest_unavailable, database_unavailable, modal_call_refused (with household_job_id).

Policy periods. The policy reader in libs/policyengine-simulation-persistence (policy_reform.py, reusable by the economy routes) gives each stored parameter value the period .py's reform_dict_from_parameter_values gives it, in the same order, with one difference: the last open-ended value stops at 2200-12-31, where .py stops at 2100-12-31, because households can be calculated up to 2200. An open-ended value that .py would give a range ending before it starts (one followed the same day by another value, or one starting after 2200) is left out. policyengine-core applies such a range on no day, but policyengine-uk refuses it with "ends before it starts", so leaving it out also keeps those UK calculations from failing.

Also:

• read-only models of the input tables' columns the service reads, compared column by column with the Public API's tables in the existing schema job;
• every deploy now gives the entrypoint the Modal token and the v2 manifest's Modal environment, not only deploys with DEPLOY_STAGE12_V2;
• a household calculation on every staging deploy, of the household the beta variable SIMULATION_STAGING_HOUSEHOLD_ID names;
• the OpenAPI document declares every answer, and the generated clients are regenerated.

Operator steps before merging (done 2026-11-11)

Without them, every staging deploy fails at the new household check, and production doesn't deploy. Both were done and read back on 2026-11-11:

1. A US household (one adult aged 40 with $30,000 of employment income in Harris County, Texas, default year 2026) was stored through the staging Public API's POST /v2/households?country_id=us, which answered 201 with ID b6604e4a-a531-404a-b839-42a3f78fc873. The row is deduplicated by content, so a repeat returns the same ID.
2. The beta environment variable SIMULATION_STAGING_HOUSEHOLD_ID is set to that ID.

The staging catalog holds the default worker's .py version, 6.17.1 (checked through GET /v2/model-catalog?country_id=us&policyengine_version=6.17.1), so the check won't fail with unknown_catalog_version. Not verified: that staging sim's database URL names the same v2 database as the staging Public API; the first staging deploy's check confirms it.

The Public API's matching routes, PolicyEngine/policyengine-api#4274, merge after this is on staging.

Tests

• Each test was run individually by node ID, and seen failing for the expected reason before its change, except the executor tests that pin .py's existing behaviour, which passed on their first run.

• Persistence library, on PostgreSQL 17 built from the Public API's master with its privileges applied and verified: the policy reader, the household job records and store, the read-only input models and their comparison, and the bounded transactions.

• Simulation Entrypoint: the service, routes, contract-field tracing, workers, household conversion, deploy wiring, the staging check, authorization and the OpenAPI document. The 500 log tests read the record the service's own observability runtime emits.

• Executor: reform periods applied in 2200 with the real US and UK models, and .py's own reform_dict_from_parameter_values on same-day, bounded and open-ended values, applied with policyengine-core, including the case policyengine-uk refuses.

• Generated client: submit and read, including the JSON 500.

• Checks: CI's Ruff, the rules registry, the quality guards including file length, the lock checks and Pyright passed before each commit.

• CI fixes:

  • test_deployment_assets.py::test_deployment_uses_gcloud_workflow_without_terraform counts the reusable deploy workflow's jobs that declare environment: ${{ inputs.release_environment }}. The staging household check is one of them, because it reads that environment's secrets and variable, so the expected count is now 18, not 17.
  • The two staging household scripts now start with #!/bin/bash, which the executor's test_modal_scripts.py::TestAllScriptsHaveShebang::test_all_scripts_have_shebang requires of every modal-*.sh and simulation-*.sh script.

• Review: three rounds, and four more of the fix code.

  • Round 1: a call Modal accepted after the 10-second wait was answered as failed.
  • Round 2: a 500 declared on the POST broke the generated client for other errors.
  • Round 3: the same-day rule differed from .py when the earlier value had an end date.
  • Round 4: the observability runtime dropped the 500 log's stack frames.
  • Round 5: no bugs. Its reader comparison matched .py on 20,014 generated policies.
  • Rounds 6 and 7, of the two CI fixes: no bugs.

  All are fixed test-first, with each round's code-quality findings.

Environment variables and secrets

Value	Provider	How it reaches the code
MODAL_TOKEN_ID, MODAL_TOKEN_SECRET	Secret Manager secrets named by the beta/prod variables MODAL_TOKEN_ID_SECRET_NAME, MODAL_TOKEN_SECRET_SECRET_NAME	The deploy step's env: (job environment: inputs.release_environment), then the deploy script's --set-secrets with the newest version. Now mounted on every deploy; the Stage 12 block no longer mounts them separately.
SIMULATION_V2_MODAL_ENVIRONMENT	The workflow input modal_environment	The deploy script's environment file.
SIMULATION_V2_MANIFEST_NAME	A literal in the deploy script	The environment file.
SIMULATION_SERVICE_DATABASE_URL	Secret Manager	Unchanged. /ready already checks every grant these routes use.
SIMULATION_STAGING_HOUSEHOLD_ID	New beta environment variable (not set yet)	The staging check's STAGING_HOUSEHOLD_ID.
SIMULATION_ENTRYPOINT_AUTH_ISSUER, _AUDIENCE, SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_ID, _SECRET	beta and prod environment secrets, with no repository-level copies	The staging check job's environment: inputs.release_environment (beta on staging), then its step's env:. The callers' secrets: inherit passes only repository and organization secrets, so the job's environment: is what makes these available. They exist; authenticated_test already reads them.

Not verified:

• the Modal token's permission to call the household functions (the same token already starts coordinators);
• that the staging test client holds simulation:submit and simulation:read (this matters once enforcement is on);
• that the staging Public API and the staging Simulation Entrypoint use the same database;
• nothing ran against real Modal or staging.

🤖 Generated with Claude Code