From 769a3e6c7a97b5953ba7664253e629aefe2b5ed9 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 12:27:01 -0500 Subject: [PATCH 01/18] docs: design for rail 17 reading Coinbase and DCA catch-up (#1010) Co-Authored-By: Claude Opus 5.5 --- ...il-17-venue-read-and-dca-catchup-design.md | 165 ++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md diff --git a/docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md b/docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md new file mode 100644 index 00000000..bdeeea65 --- /dev/null +++ b/docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md @@ -0,0 +1,165 @@ +# Rail 17 reads Coinbase, and a skipped DCA buy is retried until the next window + +Status: design, 2026-11-06. Supersedes the open options in #1010. + +## Why + +Rail 17 (withdrawal capability, KB §65.4 *qabd*) blocks every live BUY unless the operator typed +`keel withdrawals attest --enabled` at a terminal within the last 7 days. Since August it has +lapsed at least five times (2026-08-14, 08-25, 10-09, 10-17, 11-02). Each time the operator found +out days later, and each lapse cost DCA windows outright, because `Dca.detect` fires only on +`(bar_ts // 86400) % cadence_days == 0` and a vetoed window is never retried. As of 2026-11-06 the +five 14-day rules (PAXG, ADA, XLM, DOGE, FET; live rules 9-13) have lost all three of their +windows (10-09, 10-23, 11-06) and have never placed a live order. + +The operator's words: the problem is "the silent failure and losing the buy". The rule they want +is: buy when the Coinbase balance covers the order, never when it is short or empty. Rail 13 +already enforces exactly that and stays unchanged. The §65.4 possession test stays too, but its +input changes from a human assertion to the venue's own answer. + +## What the probe established (2026-11-06, live key, read-only) + +- `GET /v2/accounts` (Coinbase App API) returns `allow_withdrawals` and `allow_deposits` on every + wallet. With the existing CDP key it returned `true` for BTC, ETH, PAXG, XLM, USDC, USD. +- `GET /api/v3/brokerage/key_permissions` -> `can_view: true, can_trade: true, + can_transfer: false`. The read needs no new permission, and keel still cannot move funds. +- ADA, DOGE and FET have no wallet in either `/v2/accounts` or Advanced Trade `get_accounts` + (never held). There is no flag to read for them until a first buy creates the wallet. +- The public `GET https://api.exchange.coinbase.com/currencies` reports `status: online` per + currency and per network. Recorded here; not used by this design (see Non-goals). +- Not established: that `allow_withdrawals` actually turns `false` during a real account + restriction. It has only been observed `true`. The design treats it as the venue's statement + and keeps the manual `--suspended` override for the case where it does not flip. + +The probe script is not committed; its calls are the three above. + +## Goals + +1. Rail 17's input on a live BUY is read from Coinbase on every intent, not from a dated + attestation. No TTL, no weekly ceremony, no expiry warnings. +2. The decision: + - the venue reports `allow_withdrawals: true` for the **quote wallet the order spends** + (USDC by default, whatever rail 13 checks) **and** for the **base wallet being bought** + when that wallet exists -> rail 17 passes; + - either reports `false` -> veto, reason names the wallet; + - the base wallet does not exist yet -> decided on the quote wallet alone (operator-accepted + guess: a coin never held has nothing to be frozen, and the order's funds are the USDC); + - the read fails, times out, or the quote wallet is missing from the response -> veto + (fails closed, operator's decision 2026-11-06: "skip the buy"). +3. A manual `keel withdrawals attest --suspended` still halts entries, with no expiry, until + cleared. Clearing it keeps the typed-`yes` TTY gate (it releases a halt). A stored + suspension beats a venue `true`. +4. Every skipped live DCA buy (rail-17 veto or any other reason) raises a **macOS + notification posted by keel itself** (operator's decision 2026-11-06), the same cycle: which + product, and why (venue `false` naming the wallet, venue unreadable, manual suspension, or + the other rail's name). Once per rule per day, repeating every day the buy stays skipped; a + later fill ends it. This is the fix for "found out very late". + - The existing `notify_after_cycle` webhook path sends nothing on the live deployment (no + `notifications:` block, no `KEEL_ALERT_WEBHOOK`), and the wrapper's `osascript` alerts fire + only on non-zero exits. A vetoed buy is a zero-exit cycle, so neither reaches the operator. + - Do not signal through a new wrapper exit code: non-zero exits drive the wrapper's same-day + retries and failure counter, and `keel-live-run.sh` is not in the repo. + - Do not reuse #793's `unreported()` once-per-window dedupe; it would silence the daily + repeats. + - The sink must be a no-op off macOS and in tests, and a failure to post must never fail the + cycle. +5. **DCA catch-up.** A DCA buy that was due and did not fill, for any reason (any rail veto, + a failed cycle, a venue error, the machine being off), is attempted again on every later + daily cycle until it fills or the rule's next window opens. At most one fill per rule per + window, ever. +6. Rails 13, 14 and every other rail are unchanged. SELLs are still never touched by rail 17. + +## Non-goals + +- Using the public `/currencies` network status as a rail input. A network outage is not an + account freeze; it could be added later as its own signal. +- Changing DCA cadence, budgets, or the epoch-day anchoring of windows. +- Catch-up for non-DCA rules (Turtle etc.). Their signals are price-dependent and a late entry is + a different trade. +- Paper and backtest behaviour of rail 17: it stays "skipped, recorded as skipped" as today. + +## Anti-goals (would be a failure even if it works) + +- A retry that double-buys a window. The ledger, not the candle, decides "already bought". +- A catch-up buy that fires on the day after a window was already filled (e.g. a cycle that + re-runs after a crash). +- A venue read failure that is indistinguishable, in the notification, from a real freeze. +- Any path where a cron line or unattended process can clear a manual suspension. +- Silence: a skipped buy with no notification. + +## The DCA window and catch-up + +Define for a DCA rule with `cadence_days = N` and a completed daily bar at epoch day `d`: + +- the rule's current window starts at `w = d - (d % N)` and ends before `w + N`; +- the rule is **due** on bar `d` if no BUY order for this rule (`orders.rule_id`) with status + `filled` has `created_at` within `[w * 86400 + the cycle offset, ...)` — exact boundary is the + builder's call, but it must be keyed to the window, not to "today"; +- today's behaviour (`d % N == 0`) is the special case where the window opens. + +Constraints the builder must respect: + +- `Dca.detect` is pure and is called by the backtester, the account sim, paper, and + `_dca_fires_today` (`keel/agent.py`). Keep the window arithmetic pure and put the ledger + lookup in the live/paper engine path, so backtests keep their current semantics unless the + builder shows they should change. The sizing (`size_usd`, dip bonus) on a catch-up day is + computed from that day's bar, not the window-opening bar. +- `_dca_fires_today` drives spec §3.4's same-day-DCA exclusion for sleeve sells. A catch-up day + is a buy day for that exclusion. +- A partially filled or `open` DCA order counts as "attempted, not filled": the builder decides + whether that blocks a retry. Recommended: an order in a non-terminal state blocks retry until + it resolves, so two buys are never in flight. +- Rail 14 (monthly buy cap) and rail 2/3 (per-order, per-day) may legitimately veto a catch-up; + it just retries next day within the window. *Designer's call:* a catch-up that spills a buy + into the next calendar month is accepted, and rail 14 remains the bound. `keel dca-plan`'s + worst-month figure (`_worst_month_buy_days`) is left alone, but its output must say that + catch-up can add up to one buy per rule to a month. For scale: the live cap is $500/month and + the seven live rules already plan up to ~$580 in a month with five Thursdays and three + biweekly windows, so rail 14 can bind today, independent of this change. +- *Designer's call:* a DCA order still `pending`/`open` blocks a retry for that rule, so two buys + are never in flight. If it is still unresolved on the next daily cycle, that is a rail-17-class + "buy skipped" notification for that day (never silent). The builder must verify that + reconcile settles a `pending` row with no venue order id (the `place_order` raised path in + `executor._run_order`); if it does not, looking it up at the venue by `client_order_id` is in + scope. + +## Surfaces that change + +- `keel/execution/guards.py` rail 17 block and its message (drop "attest with" advice for the + venue case; drop "DCA-exempt paths are unaffected", #1010 notes it is misleading). +- `keel/execution/executor.py`: replace `_withdrawals_enabled` / `WITHDRAWAL_ATTESTATION_TTL_SEC` + with a venue read through the broker adapter. The adapter port (`keel_broker_api`) gets a + method for withdrawal capability per currency; Coinbase implements it via `/v2/accounts`; + Alpaca and any other venue must answer explicitly (the builder decides; it must not silently + pass). +- `keel/commands/withdrawals.py`: `attest --enabled` becomes "clear manual suspension"; + `--suspended` sets it with no TTL; `show` prints the venue's live answer per wallet plus any + manual suspension. +- Expiry machinery that only exists for this TTL: `keel/attestations.py` (rail 17 entries), + `keel/commands/doctor.py` rail-17 check, `keel/notifications.py` expiry banner for rail 17, + `keel/commands/setup.py` wizard step, `keel status` rail-17 line, the web UI's attestation + display if it reads the same keys. +- Docs: `docs/rails/rail-17-withdrawal-attestation.md` (rename/rewrite), `docs/operator-runbook.md` + item 3 (weekly attest), `docs/go-live-runbook.md`, `docs/glossary.md`, `docs/fiqh-basis.md` + only where it describes the mechanism. +- Existing stored state `withdrawals_enabled` / `withdrawals_attested_at`: a stored + `withdrawals_enabled = false` migrates to a manual suspension; a stored `true` is ignored. + +## Decided by the operator + +- Remove the dated attestation as a buy blocker. +- Keel reads withdrawability from Coinbase itself. +- Venue unreadable -> skip the buy and notify. +- A skipped DCA buy is retried daily until the next window. +- Keep rail 13's balance check as is. +- Alerts are macOS notifications posted by keel. + +## Left to the builder + +- Where the window/ledger logic lives, and the exact window boundary arithmetic. +- The adapter port method's name and shape, and what non-Coinbase venues answer. +- How the notification is worded and deduplicated across retry days (one per day per rule is + fine; it must not go silent after the first). +- Whether `withdrawals show` is renamed. + +Closes #1010 when shipped. From 184584c90c33cb445894b5d2dea07f3fd558a563 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 12:31:18 -0500 Subject: [PATCH 02/18] docs: implementation plan for rail 17 venue read and DCA catch-up (#1010) Co-Authored-By: Claude Opus 5.5 --- ...1-06-rail-17-venue-read-and-dca-catchup.md | 363 ++++++++++++++++++ 1 file changed, 363 insertions(+) create mode 100644 docs/superpowers/plans/2026-11-06-rail-17-venue-read-and-dca-catchup.md diff --git a/docs/superpowers/plans/2026-11-06-rail-17-venue-read-and-dca-catchup.md b/docs/superpowers/plans/2026-11-06-rail-17-venue-read-and-dca-catchup.md new file mode 100644 index 00000000..5de88e02 --- /dev/null +++ b/docs/superpowers/plans/2026-11-06-rail-17-venue-read-and-dca-catchup.md @@ -0,0 +1,363 @@ +# Rail 17 reads Coinbase, skipped DCA buys retry, skips alert on macOS — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Live buys stop depending on a 7-day human attestation: rail 17 asks Coinbase per buy, a +skipped DCA buy is retried daily until its rule's next window, and every skipped live DCA buy +posts a macOS notification. + +**Architecture:** Coinbase's `/v2/accounts` `allow_withdrawals` flag becomes rail 17's input, +read by the executor through an optional adapter method (absent method = unreadable = veto). +The stored `withdrawals_enabled` key keeps one meaning only: `False` is a manual suspension with +no expiry. DCA catch-up is a ledger-fed attribute on `Dca` that the agent sets before +`engine.evaluate`, so backtests and the account sim (which never set it) are unchanged. The +agent collects skipped DCA buys on the cycle result and posts them through a new macOS notifier +in `keel_core`. + +**Tech Stack:** Python 3.14, click, sqlite, pytest, mypy, ruff; `coinbase-advanced-py` +`RESTClient.get`; `/usr/bin/osascript`. + +**Spec:** `docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md` + +## Global Constraints + +- TDD: every task writes its failing test first and runs it red before any implementation. +- Work in a worktree. Tests that touch `packages/` run with + `PYTHONPATH=packages/keel-core:packages/keel-broker-api:packages/keel-broker-coinbase:packages/keel-broker-fake:packages/keel-broker-alpaca:packages/keel-broker-kraken:packages/keel-broker-robinhood` + (the worktree otherwise resolves packages from the main checkout). Type-check with bare `mypy`. +- Rail 17 still applies only to live BUYs (`is_buy and not offline`); SELLs and paper are untouched. +- Fails closed: an unreadable venue, a missing quote wallet, or an adapter without the method + vetoes the BUY. +- `keel withdrawals attest --enabled` keeps its typed-`yes` TTY gate. `--suspended` stays ungated. +- One fill per DCA rule per window, ever. The ledger (`orders`), not the candle, decides. +- A notification failure never fails the cycle. The notifier is a no-op off macOS + (`sys.platform != "darwin"`), when `KEEL_DISABLE_MACOS_NOTIFY` is set, and under pytest. +- After touching `keel/web/static/js/*`: `node --check` each changed file and load the console + in a real browser with the service worker unregistered. +- Commits end with `Co-Authored-By: Claude Opus 5.5 `. + +## Review Focus + +1. **Paginated `/v2/accounts`.** An account with more than one page of wallets must still find + USDC/BTC; a wallet on page 2 is not "missing". Test in Task 1. +2. **Deploy-day catch-up.** On the first cycle after deploy, rules whose current window was + vetoed before deploy (all seven, window opened on the 2026-11-05 bar) must buy once, and not + a second time on the next day. Test in Task 4. +3. **Rule promoted mid-window.** A rule promoted to live after its current window opened must + not buy immediately on catch-up. Test in Task 4. +4. **Same-day rerun after a crash.** The wrapper retries a failed cycle hourly the same day; a + rerun after a fill in the same window must not buy again. Test in Task 4. +5. **Notification storm.** Seven skipped rules, every hourly retry of a failed day: at most one + notification per rule per UTC day. Test in Task 5. + +--- + +### Task 1: Coinbase adapter reads `allow_withdrawals` + +**Files:** +- Modify: `packages/keel-broker-api/keel_broker_api/port.py` (add `WithdrawalReader` Protocol after `Broker`) +- Modify: `packages/keel-broker-coinbase/keel_broker_coinbase/transport.py` (declare `get` on `Transport`) +- Modify: `packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py` (method on `CoinbaseAdapter`, beside `verify_cash_account`) +- Modify: `packages/keel-broker-fake/keel_broker_fake/adapter.py` (configurable answer, default all `True`) +- Test: `tests/broker_coinbase/test_adapter.py` + +**Interfaces:** +- Produces: + - `keel_broker_api.port.WithdrawalReader` — `Protocol` with + `def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: ...` + Each requested currency maps to `True`/`False` from the venue, or `None` when the account + has no wallet for it. Raises on any failure to read (network, auth, unparseable). + - `CoinbaseAdapter.withdrawals_allowed(currencies)` implementing it. + - `Transport.get(self, url_path: str, params: dict[str, Any] | None = None, **kwargs: Any) -> Any`. + +- [ ] **Step 1: Write the failing tests** + +```python +def test_withdrawals_allowed_reads_v2_accounts_flags(): + transport = FakeTransport(v2_pages=[{"data": [ + {"currency": {"code": "USDC"}, "allow_withdrawals": True}, + {"currency": {"code": "BTC"}, "allow_withdrawals": False}, + ], "pagination": {"next_starting_after": None}}]) + got = CoinbaseAdapter(transport).withdrawals_allowed(["USDC", "BTC", "FET"]) + assert got == {"USDC": True, "BTC": False, "FET": None} + assert transport.get_calls[0][0] == "/v2/accounts" + +def test_withdrawals_allowed_follows_pagination(): + # page 1 has BTC, page 2 has USDC; both must be found + ... + assert got == {"USDC": True, "BTC": True} + assert len(transport.get_calls) == 2 + assert transport.get_calls[1][1]["starting_after"] == "" + +def test_withdrawals_allowed_raises_when_unreadable(): + transport = FakeTransport(v2_raises=ConnectionError("reset")) + with pytest.raises(ConnectionError): + CoinbaseAdapter(transport).withdrawals_allowed(["USDC"]) + +def test_withdrawals_allowed_treats_non_bool_flag_as_unreadable(): + # {"allow_withdrawals": "yes"} or a missing key on a wallet that exists -> raises ValueError + ... + +def test_coinbase_adapter_satisfies_withdrawal_reader(): + assert isinstance(CoinbaseAdapter(FakeTransport()), WithdrawalReader) # runtime_checkable +``` + +Extend the file's existing transport fake rather than adding a second one. Currency codes are +compared upper-cased; `currency` may be a dict (`{"code": ...}`) or a bare string — accept both. + +- [ ] **Step 2: Run, expect FAIL** — `pytest tests/broker_coinbase/test_adapter.py -k withdrawals -v` → `AttributeError: ... withdrawals_allowed`. +- [ ] **Step 3: Implement.** `limit=100`, follow `pagination.next_starting_after` via `starting_after`, stop after 20 pages and raise if still paginating. Docstring records the 2026-11-06 probe (flag observed `true` only; never observed flipping) and that `can_transfer: false` is unaffected. Mark the Protocol `@runtime_checkable`. Fake adapter: `withdrawals_allowed` answers from a constructor dict, defaulting every currency to `True`. +- [ ] **Step 4: Run, expect PASS**, then `mypy` clean. +- [ ] **Step 5: Commit** — `feat(coinbase): read per-wallet allow_withdrawals for rail 17 (#1010)`. + +--- + +### Task 2: Rail 17 decides from the venue; the TTL is gone + +**Files:** +- Modify: `keel/execution/executor.py` (replace `_withdrawals_enabled`, delete `WITHDRAWAL_ATTESTATION_TTL_SEC`; `BASE_INCREMENT_TTL_SEC` keeps its own literal `7 * 24 * 60 * 60` and its comment stops citing rail 17) +- Modify: `keel/execution/guards.py` (rail 17 block, `OrderIntent`) +- Test: `tests/execution/test_withdrawal_attestation.py` → rename `tests/execution/test_withdrawal_capability.py`; `tests/execution/test_guards.py` +- Modify test fakes: `tests/execution/test_executor.py::FakeBroker`, `tests/test_agent.py::FakeBroker`, `tests/test_cli.py::FakeBroker` gain `withdrawals_allowed` answering `True` for everything (drop the now-meaningless `withdrawals_attested_at` seeding where it only existed to satisfy the TTL) + +**Interfaces:** +- Consumes: `WithdrawalReader.withdrawals_allowed` (Task 1). +- Produces: + - `executor.manual_withdrawal_suspension(repo) -> bool` — `True` iff stored `withdrawals_enabled` is exactly `False`. No timestamp is read. + - `executor._withdrawal_capability(broker, repo, product_id: str) -> tuple[bool | None, str]` — `(verdict, detail)`. Order of decision: manual suspension → `(False, "manual suspension (keel withdrawals attest --suspended)")`; broker without `withdrawals_allowed` → `(None, "venue adapter cannot report withdrawals")`; raised read → `(None, "venue unreadable: ")`; quote wallet `None` → `(None, "no wallet reported")`; any requested wallet `False` → `(False, " withdrawals disabled at venue")`; else `(True, "")`. Requested currencies are the product's quote leg (`quote_currency_of`) and base leg; base `None` is allowed. + - `OrderIntent.withdrawals_detail: str = ""`. + +- [ ] **Step 1: Write the failing tests** in `test_withdrawal_capability.py`, one per branch above, through `executor.execute` on a live BUY with `FakeBroker`, asserting `result.placed is False`, `vetoed_by[0].startswith("withdrawal_capability:")`, the detail substring, and `broker.place_calls == []`. Plus: + +```python +def test_stale_attestation_no_longer_vetoes(): + repo.set_state("withdrawals_enabled", True) + repo.set_state("withdrawals_attested_at", NOW_TS - 30 * 86400) + assert execute(...).placed is True + +def test_never_attested_account_buys_when_venue_allows(): + # no withdrawals_* keys at all + assert execute(...).placed is True + +def test_base_wallet_missing_decides_on_quote(): + broker = FakeBroker(withdrawals={"USD": True}) # no FET wallet + assert execute(signal_for("FET-USD"), ...).placed is True + +def test_sell_never_reads_withdrawals(): + broker = FakeBroker(withdrawals_raises=RuntimeError()) + assert execute(exit_signal, ...).placed is True + assert broker.withdrawals_calls == 0 + +def test_paper_never_reads_withdrawals(): ... # offline path, rail recorded as skipped +``` + +In `test_guards.py`: the rail-17 messages for `None` and `False` include `intent.withdrawals_detail`, and neither says "DCA-exempt" or "attest". + +- [ ] **Step 2: Run, expect FAIL** (stale attestation still vetoes; missing method not handled). +- [ ] **Step 3: Implement.** The read happens in the ENTER branch where `_withdrawals_enabled` is called today, live only (`broker is not None`). Log `executor.withdrawal_capability` with product, verdict, detail. New rail-17 messages: + - `None`: `withdrawal_capability: UNKNOWN ({detail}). Under §65.4 possession requires that nothing prevents withdrawal on demand; an unverified state is not evidence that it holds. Buy skipped; it is retried next cycle within its window.` + - `False`: `withdrawal_capability: SUSPENDED ({detail}). Under §65.4 an asset that cannot be withdrawn may not have been validly possessed, so new entries are halted. Existing holdings and exits are deliberately unaffected.` +- [ ] **Step 4: Run** `pytest tests/execution tests/test_agent.py tests/test_cli.py tests/test_hostile_rule_rails.py -q` → PASS; `mypy` clean. +- [ ] **Step 5: Commit** — `feat(rail-17): decide withdrawal capability from the venue, drop the 7-day TTL (#1010)`. + +--- + +### Task 3: Remove the expiry surfaces; `withdrawals` CLI becomes a manual switch + +**Files:** +- Modify: `keel/commands/withdrawals.py` +- Modify: `keel/attestations.py` (drop the rail-17 `Attestation` from `survey_definitions`, delete `withdrawal_expiry` and `_withdrawal_dates`) +- Modify: `keel/commands/doctor.py` (rail-17 freshness check → a finding that FAILs only when a manual suspension is set; it never reads `withdrawals_attested_at`) +- Modify: `keel/commands/status.py` (`WithdrawalAttestationStatus` → shows `manual suspension: on/off`; no days-to-expiry; no venue call) +- Modify: `keel/notifications.py` (no rail-17 expiry event) +- Modify: `keel/commands/setup.py` (wizard step explains the venue read and the manual switch; no weekly cadence) +- Modify: `keel/web/payload.py`, `keel/web/api.py`, `keel/web/events.py`, `keel/web/static/js/*.js` and `index.html` wherever they render rail-17 expiry +- Test: `tests/test_cli.py`, `tests/test_attestation_expiry.py`, `tests/test_attestation_alert_once.py`, `tests/test_notifications.py`, `tests/commands/test_doctor.py`, `tests/commands/test_status.py`, `tests/web/test_attestation_banner.py`, `tests/web/test_server.py`, `tests/test_first_run_wizard.py`, `tests/commands/test_console_thinness.py` + +**Interfaces:** +- Consumes: `executor.manual_withdrawal_suspension` (Task 2), `WithdrawalReader` (Task 1). +- Produces: CLI output contracts below. + +- [ ] **Step 1: Write the failing tests.** + +```python +def test_attest_enabled_says_it_clears_suspension_and_never_expires(cli_tty_yes): + out = run("withdrawals attest --enabled") + assert "manual suspension cleared" in out and "expires" not in out + +def test_attest_suspended_has_no_expiry(): + out = run("withdrawals attest --suspended") + assert "expires" not in out + assert manual_withdrawal_suspension(repo) is True + +def test_show_prints_venue_answer_per_wallet(fake_broker_withdrawals={"USD": True, "BTC": False}): + out = run("withdrawals show") + assert "manual suspension: off" in out + assert re.search(r"BTC\s+withdrawals disabled", out) and re.search(r"USD\s+allowed", out) + +def test_show_reports_unreadable_venue_without_failing(): ... # exit 0, says "venue unreadable" + +def test_survey_has_no_withdrawal_attestation(): + assert "withdrawals" not in {a.key for a in attestations.survey_definitions()} + +def test_doctor_rail17_passes_without_any_attestation(): ... +def test_doctor_rail17_fails_on_manual_suspension(): ... +def test_notifications_never_emit_withdrawal_expiry_even_when_attested_long_ago(): ... +``` + +`withdrawals show` lists the quote currency plus the base of every live rule's product; it needs +the broker, so it builds one the way `keel status --live` does and degrades to "venue +unreadable" on failure. Existing tests that assert expiry wording are deleted, not loosened; +list each deleted test in the commit body. + +- [ ] **Step 2: Run, expect FAIL.** +- [ ] **Step 3: Implement.** Web console: remove the rail-17 expiry banner; keep a rail-17 line showing manual suspension on/off. +- [ ] **Step 4: Verify.** Full `pytest -q` PASS; `mypy`; `ruff check`; `node --check` on each changed JS file; open `keel serve` in a real browser with the service worker unregistered and confirm the console renders and the banner is gone. +- [ ] **Step 5: Commit** — `feat(rail-17): withdrawals is a manual switch; remove the expiry machinery (#1010)`. + +--- + +### Task 4: DCA catch-up until the next window + +**Files:** +- Modify: `keel/strategy/rules/dca.py` +- Modify: `keel/agent.py` (set catch-up state on each loaded `Dca` before `engine.evaluate` and before `_dca_fires_today`) +- Modify: `keel/data/repository.py` only if a rule-scoped order query is needed (`get_orders` filters by mode/product/status today; filtering `rule_id` in Python is acceptable) +- Test: `tests/strategy/test_dca_cadence.py`, `tests/test_agent.py` + +**Interfaces:** +- Produces (in `dca.py`): + - `def window_start_day(day: int, cadence_days: int) -> int` — `day - day % cadence_days`. + - `@dataclass(frozen=True) class DcaCatchUp: since_day: int; filled_window_day: int | None; in_flight: bool` + `since_day` = `promoted_at // 86400` (fallback `created_at`); `filled_window_day` = the + `window_start_day` of the most recent filled BUY for this rule in this mode, computed from + the order's `created_at // 86400 - 1` (the cycle on day d+1 buys bar d); `in_flight` = any BUY + for this rule in `RESTING_STATUSES`. + - `Dca.catch_up: DcaCatchUp | None = None` (class attribute default; backtest/sim never set it). + - `Setup.context` gains `"catch_up": bool` and `"window_start_day": int` on every DCA setup. +- Produces (in `agent.py`): `def _dca_catch_up_state(repo, rule: Dca, rule_row: dict, *, mode: str) -> DcaCatchUp`. + +`Dca.detect` decision with bar day `d`, `w = window_start_day(d, N)`: +1. `catch_up is None` → today's behaviour exactly (fire iff `d == w`). +2. `catch_up.in_flight` → no setup. +3. `catch_up.filled_window_day == w` → no setup (already bought this window). +4. `d == w` → setup, `catch_up=False`. +5. `w + 1 < catch_up.since_day` → no setup (the window's buying cycle, day `w + 1`, ran before the rule went live). +6. else → setup, `catch_up=True`, sized from bar `d`. + +- [ ] **Step 1: Write the failing tests** (`test_dca_cadence.py`): + +```python +def test_no_catch_up_state_is_unchanged_behaviour(): # backtests + assert Dca("BTC-USD", cadence_days=7).detect(bars_ending(day=7*k + 3)) is None + +def test_catch_up_fires_on_a_missed_window_day(): + rule = Dca("BTC-USD", cadence_days=14); rule.catch_up = DcaCatchUp(since_day=0, filled_window_day=None, in_flight=False) + s = rule.detect(bars_ending(day=14*k + 5)) + assert s is not None and s.context["catch_up"] is True and s.context["window_start_day"] == 14*k + +def test_catch_up_stops_once_the_window_filled(): + rule.catch_up = DcaCatchUp(0, filled_window_day=14*k, in_flight=False) + assert rule.detect(bars_ending(day=14*k + 5)) is None + +def test_window_open_day_after_fill_in_same_window_does_not_rebuy(): # Review Focus 4 + rule.catch_up = DcaCatchUp(0, filled_window_day=14*k, in_flight=False) + assert rule.detect(bars_ending(day=14*k)) is None + +def test_in_flight_order_blocks_catch_up(): ... +def test_rule_promoted_mid_window_waits_for_next_window(): # Review Focus 3 + rule.catch_up = DcaCatchUp(since_day=14*k + 4, filled_window_day=None, in_flight=False) + assert rule.detect(bars_ending(day=14*k + 6)) is None + assert rule.detect(bars_ending(day=14*(k+1))) is not None +def test_next_window_open_day_is_a_normal_buy(): ... +``` + +`tests/test_agent.py`: + +```python +def test_vetoed_dca_window_buys_on_next_cycle_then_not_again(): # Review Focus 2 + # cycle 1 (window-open): broker.withdrawals_raises -> vetoed, no order + # cycle 2 (next day): venue readable -> exactly one filled BUY for the rule + # cycle 3 (day after): no new order + assert [o["rule_id"] for o in repo.get_orders(mode="live", status="filled")] == [rule_id] + +def test_catch_up_day_counts_as_dca_day_for_sleeve_exclusion(): ... # _dca_fires_today True +``` + +Run red, implement, run green (`pytest tests/strategy tests/test_agent.py tests/sim -q`; the sim +tests prove backtests are unchanged), `mypy`. + +- [ ] **Step 2: Run, expect FAIL.** +- [ ] **Step 3: Implement** the decision above and the agent wiring (both live and paper agent paths; the ledger is filtered by the cycle's own `mode`). +- [ ] **Step 4: Run, expect PASS.** +- [ ] **Step 5: Commit** — `feat(dca): retry a skipped buy daily until the rule's next window (#1010)`. + +--- + +### Task 5: macOS notification for every skipped live DCA buy + +**Files:** +- Create: `packages/keel-core/keel_core/macos_notify.py` +- Modify: `keel/agent.py` (collect `skipped_buys` on the cycle result; post after `notify_after_cycle`) +- Test: `tests/core/test_macos_notify.py`, `tests/test_agent.py` + +**Interfaces:** +- Produces: + - `keel_core.macos_notify.post(title: str, message: str, *, runner: Callable[[list[str]], Any] | None = None) -> bool` — runs `["/usr/bin/osascript", "-e", script]` with a 10 s timeout; `False` (never raises) on non-darwin, `KEEL_DISABLE_MACOS_NOTIFY` set, `PYTEST_CURRENT_TEST` set without an injected `runner`, or any error. Escapes `"` and `\` in both strings. + - `@dataclass(frozen=True) class SkippedBuy: rule_id: int; product: str; reason: str` and `CycleResult.skipped_buys: list[SkippedBuy]`. + - `agent._post_skipped_buys(repo, skipped: list[SkippedBuy], now_ts: int, *, post=macos_notify.post) -> int` — posts at most once per `rule_id` per UTC day, remembered in `agent_state` key `dca_skip_notified:` = the UTC date string; returns the count posted. + +A skipped buy is a DCA setup this cycle (window-open or catch-up) on a non-offline cycle whose +`ExecutionResult.placed` is `False`. `reason` = `"; ".join(vetoed_by)` or `result.reason`. A DCA rule that +was due but produced no signal because entries were withheld (`entries_allowed` false / stale +feed / not ready) is also a skip with that reason. Notification text: +title `keel-live: DCA buy skipped`, message ` (rule ): . Retrying daily until .` + +- [ ] **Step 1: Write the failing tests.** + +```python +def test_post_is_noop_off_darwin(monkeypatch): monkeypatch.setattr(sys, "platform", "linux"); assert post("t", "m", runner=boom) is False +def test_post_escapes_quotes(): calls=[]; post('a"b', 'c\\d', runner=calls.append); assert '\\"' in calls[0][2] +def test_post_swallows_runner_errors(): assert post("t", "m", runner=raises) is False + +def test_skipped_live_dca_buy_posts_once_per_rule_per_day(): # Review Focus 5 + posted = [] + _post_skipped_buys(repo, [SkippedBuy(9, "PAXG-USD", "withdrawal_capability: UNKNOWN (...)")] * 2, NOW, post=lambda t, m: posted.append(m) or True) + _post_skipped_buys(repo, [SkippedBuy(9, "PAXG-USD", "...")], NOW + 3600, post=...) + assert len(posted) == 1 + _post_skipped_buys(repo, [SkippedBuy(9, "PAXG-USD", "...")], NOW + 86400, post=...) + assert len(posted) == 2 + +def test_cycle_with_vetoed_dca_records_skipped_buy(): ... # run_once, live, venue unreadable +def test_paper_cycle_records_no_skipped_buys(): ... +def test_notifier_failure_does_not_fail_cycle(): ... +``` + +- [ ] **Step 2: Run, expect FAIL.** +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run** `pytest -q` PASS; `mypy`; `ruff check`. +- [ ] **Step 5: Commit** — `feat(alerts): macOS notification for every skipped live DCA buy (#1010)`. + +--- + +### Task 6: Docs, planner note, release notes + +**Files:** +- Rewrite: `docs/rails/rail-17-withdrawal-attestation.md` → `docs/rails/rail-17-withdrawal-capability.md` (fix every inbound link: `grep -rn rail-17-withdrawal-attestation`) +- Modify: `docs/operator-runbook.md` (delete the weekly-attest item; add "a macOS notification says a DCA buy was skipped: read the reason; it retries daily"), `docs/go-live-runbook.md`, `docs/glossary.md`, `docs/fiqh-basis.md` (mechanism only; the §65.4 basis text stays) +- Modify: `keel/commands/dca_plan.py` output: one line stating catch-up can add up to one buy per rule to a calendar month and rail 14 still bounds it +- Modify: `CHANGELOG.md` (Unreleased: Changed/Removed/Added entries) +- Test: `tests/commands/test_dca_plan.py` (the new line), `tests/test_fiqh_basis.py` and any doc tests that pin rail-17 wording + +- [ ] **Step 1: Failing test** for the `dca-plan` line; update doc-pinning tests to the new wording. +- [ ] **Step 2: Run, expect FAIL.** +- [ ] **Step 3: Write the docs and the line.** +- [ ] **Step 4: Run** full `pytest -q`, `mypy`, `ruff check`, `ruff format --check`. +- [ ] **Step 5: Commit** — `docs(rail-17): venue-read withdrawal capability, DCA catch-up, skip alerts (#1010)`; open the PR with `Closes #1010`. + +## Deploy note (not a task) + +After release, deploy to `~/keel` as usual and confirm `keel versions` prints `[release]`. The +first live cycle after deploy will catch up every rule whose current window is unfilled. If it +lands before the 2026-11-12 bar (weekly) / 2026-11-19 bar (biweekly), that is all seven rules (≈ $40 + $25 + $25 + 4×$15 = $150), subject to +rails 13 and 14. From 2d2b21e9614995cf00175360a12352465d820b71 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:17:09 -0500 Subject: [PATCH 03/18] feat(coinbase): read per-wallet allow_withdrawals for rail 17 (#1010) Co-Authored-By: Claude Opus 5.5 --- .../keel-broker-api/keel_broker_api/port.py | 23 +++++- .../keel_broker_coinbase/adapter.py | 55 +++++++++++++ .../keel_broker_coinbase/transport.py | 5 ++ .../keel_broker_fake/adapter.py | 17 +++- tests/broker_coinbase/test_adapter.py | 82 ++++++++++++++++++- tests/broker_fake/test_adapter.py | 12 ++- 6 files changed, 190 insertions(+), 4 deletions(-) diff --git a/packages/keel-broker-api/keel_broker_api/port.py b/packages/keel-broker-api/keel_broker_api/port.py index 5cfa650b..655253dd 100644 --- a/packages/keel-broker-api/keel_broker_api/port.py +++ b/packages/keel-broker-api/keel_broker_api/port.py @@ -3,7 +3,8 @@ from __future__ import annotations import uuid -from typing import Protocol +from collections.abc import Sequence +from typing import Protocol, runtime_checkable from keel_core.types import Candle, Granularity @@ -230,12 +231,32 @@ def cancel_order(self, order_id: str) -> CancelOutcome: ... +@runtime_checkable +class WithdrawalReader(Protocol): + """A venue that can say whether withdrawals are enabled on the account's wallets. + + Separate from `Broker` because it is optional: only a venue that exposes a per-wallet + withdrawal flag can implement it, and rail 17 reads it per buy instead of trusting a human + attestation that lapses. + """ + + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + """Map each requested currency to the venue's `allow_withdrawals` flag. + + `True`/`False` is the venue's answer; `None` means the account has no wallet for that + currency. **Raises on any failure to read** (network, auth, unparseable) -- a caller + gating a buy on this must treat silence as "not proven", never as "allowed". + """ + ... + + __all__ = [ "CLIENT_ORDER_ID_NAMESPACE", "Broker", "CancelOutcome", "TradeScopeDenied", "UnsupportedOrder", + "WithdrawalReader", "default_market_schedule", "resolve_client_order_id", ] diff --git a/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py b/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py index b7f437de..74916c5c 100644 --- a/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py +++ b/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py @@ -14,6 +14,7 @@ import logging import time +from collections.abc import Sequence from decimal import Decimal from typing import Any @@ -152,6 +153,9 @@ class CashAccountRequired(RuntimeError): """ +_MAX_V2_ACCOUNT_PAGES = 20 + + class CoinbaseAdapter: """Implements the `Broker` port against Coinbase Advanced Trade.""" @@ -392,6 +396,57 @@ def place_order(self, spec: OrderSpec, *, idempotency_key: str | None = None) -> success=False, broker_order_id=None, reason=reason, client_order_id=client_order_id ) + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + """Read each wallet's `allow_withdrawals` from `GET /v2/accounts` (rail 17). + + Implements `WithdrawalReader`. Returns `True`/`False` per requested currency as the + venue states it, or `None` when the account holds no wallet for it. Currency codes are + compared upper-cased; `currency` may be a `{"code": ...}` object or a bare string. + + A 2026-11-06 read-only probe of the live account saw the flag `true` on every wallet + and never saw it flip, so what `false` looks like in practice is inferred from the + field's name, not observed. `can_transfer: false` on a wallet is a different flag and + does not affect this one. + + **Fails closed, unlike `verify_cash_account`, and for the opposite reason.** That check + can only refute, so silence is harmless; this one is asked to AFFIRM that withdrawals + are off-limits to the key, so a read that did not happen proves nothing. Every failure + raises: transport errors propagate, a flag that is not a real bool on a wallet we were + asked about raises `ValueError` (a string `"yes"` is not an answer), and so does still + paginating after 20 pages -- the caller decides what an unreadable venue means. + + Pages of 100 wallets, following `pagination.next_starting_after` via `starting_after`. + """ + wanted = {c.upper() for c in currencies} + found: dict[str, bool] = {} + transport = self._require_transport() + starting_after: str | None = None + for _ in range(_MAX_V2_ACCOUNT_PAGES): + params: dict[str, Any] = {"limit": 100} + if starting_after is not None: + params["starting_after"] = starting_after + response = transport.get("/v2/accounts", params=params) + for row in _field(response, "data", []) or []: + currency = _field(row, "currency") + code = _field(currency, "code") if not isinstance(currency, str) else currency + code = str(code or "").upper() + if code not in wanted: + continue + flag = _field(row, "allow_withdrawals") + if not isinstance(flag, bool): + raise ValueError( + f"Coinbase wallet {code} has an unreadable allow_withdrawals: {flag!r}" + ) + # Several wallets for one code is not expected; if it happens, any wallet + # that can withdraw makes the code withdrawable. + found[code] = found.get(code, False) or flag + starting_after = _field(_field(response, "pagination"), "next_starting_after") + if not starting_after: + return {c: found.get(c.upper()) for c in currencies} + raise RuntimeError( + f"Coinbase /v2/accounts still paginating after {_MAX_V2_ACCOUNT_PAGES} pages" + ) + def verify_cash_account(self) -> None: """Refuse an account with derivative capability. **Refutes only; never proves (#666).** diff --git a/packages/keel-broker-coinbase/keel_broker_coinbase/transport.py b/packages/keel-broker-coinbase/keel_broker_coinbase/transport.py index 62e01750..7619997a 100644 --- a/packages/keel-broker-coinbase/keel_broker_coinbase/transport.py +++ b/packages/keel-broker-coinbase/keel_broker_coinbase/transport.py @@ -27,6 +27,11 @@ def get_products(self, product_type: str = "SPOT", **kwargs: Any) -> Any: ... def get_accounts(self, **kwargs: Any) -> Any: ... + #: The raw authenticated GET, for the v2 `/v2/accounts` read (`allow_withdrawals`, rail 17). + #: The Advanced Trade `get_accounts` above does not carry that flag; only the v2 wallet + #: listing does, and the SDK has no named method for it. + def get(self, url_path: str, params: dict[str, Any] | None = None, **kwargs: Any) -> Any: ... + #: The portfolio list, for the cash-account posture check (#666). Declared here rather than #: called off an untyped client, because this Protocol IS the adapter's statement of what it #: needs from a transport -- a method reached without declaring it is a dependency the test diff --git a/packages/keel-broker-fake/keel_broker_fake/adapter.py b/packages/keel-broker-fake/keel_broker_fake/adapter.py index 72f17390..4a8926e1 100644 --- a/packages/keel-broker-fake/keel_broker_fake/adapter.py +++ b/packages/keel-broker-fake/keel_broker_fake/adapter.py @@ -24,6 +24,7 @@ from __future__ import annotations import itertools +from collections.abc import Sequence from dataclasses import dataclass from decimal import Decimal @@ -94,7 +95,12 @@ class _Trigger: class FakeAdapter: """An in-memory venue whose shape differs from Coinbase's in five deliberate ways.""" - def __init__(self, *, trade_scope_denied: str | None = None) -> None: + def __init__( + self, + *, + trade_scope_denied: str | None = None, + withdrawals_allowed: dict[str, bool | None] | None = None, + ) -> None: """`trade_scope_denied` makes every placement raise `TradeScopeDenied` carrying that message (#233). @@ -112,15 +118,24 @@ def __init__(self, *, trade_scope_denied: str | None = None) -> None: A STRING rather than a bool because the message is the payload: it is written verbatim into `venue_trade_scopes.refuted_reason`, so a test driving this knob is testing the real path only if it can supply the venue's own words the way a real adapter does. + + `withdrawals_allowed` is what the method of that name answers per currency + (`WithdrawalReader`); a currency it does not mention is `True`, so a bare `FakeAdapter()` + is a venue where withdrawals are enabled everywhere, as on the live account. """ self._ids = itertools.count(1) self.resting: list[_RestingOrder] = [] self.triggers: list[_Trigger] = [] self._trade_scope_denied = trade_scope_denied + self._withdrawals_allowed = dict(withdrawals_allowed or {}) def capabilities(self) -> BrokerCapabilities: return _CAPABILITIES + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + """Answer from the constructor's dict; unmentioned currencies are `True`.""" + return {c: self._withdrawals_allowed.get(c, True) for c in currencies} + def market_clock(self) -> SessionState: """This venue never closes: `SessionState.OPEN` as a constant, no state to consult. diff --git a/tests/broker_coinbase/test_adapter.py b/tests/broker_coinbase/test_adapter.py index d2764547..955aa649 100644 --- a/tests/broker_coinbase/test_adapter.py +++ b/tests/broker_coinbase/test_adapter.py @@ -15,7 +15,7 @@ import pytest from keel_broker_api.orders import LimitGTC, MarketIOCByBase, MarketIOCByQuote -from keel_broker_api.port import TradeScopeDenied +from keel_broker_api.port import TradeScopeDenied, WithdrawalReader from keel_broker_api.results import ( Balance, CancelOutcome, @@ -51,7 +51,12 @@ def __init__( order: dict[str, Any] | None = None, product: dict[str, Any] | None = None, products: dict[str, Any] | None = None, + v2_pages: list[dict[str, Any]] | None = None, + v2_raises: Exception | None = None, ) -> None: + self._v2_pages = list(v2_pages or []) + self._v2_raises = v2_raises + self.get_calls: list[tuple[str, dict[str, Any]]] = [] self._candles = candles self._accounts = accounts self._preview = preview @@ -66,6 +71,12 @@ def __init__( # distinction the real venue draws, and the whole point of that assertion. self._issued_order_ids: set[str] = set() + def get(self, url_path: str, params: dict[str, Any] | None = None, **kwargs: Any) -> Any: + self.get_calls.append((url_path, dict(params or {}))) + if self._v2_raises is not None: + raise self._v2_raises + return self._v2_pages[len(self.get_calls) - 1] + def get_products(self, product_type: str = "SPOT", **kwargs: Any) -> Any: self.calls["get_products"] = {"product_type": product_type} return self._products @@ -918,3 +929,72 @@ def test_a_missing_or_unusable_quote_increment_still_yields_an_instrument(bad: o assert instrument is not None, "an unusable price tick must not lose the size tick" assert instrument.base_increment == Decimal("0.00000001") assert instrument.quote_increment is None + + +def _v2_page(rows: list[dict[str, Any]], next_after: str | None = None) -> dict[str, Any]: + return {"data": rows, "pagination": {"next_starting_after": next_after}} + + +def test_withdrawals_allowed_reads_v2_accounts_flags(): + transport = FakeTransport( + v2_pages=[ + _v2_page( + [ + {"id": "a1", "currency": {"code": "USDC"}, "allow_withdrawals": True}, + {"id": "a2", "currency": "btc", "allow_withdrawals": False}, + ] + ) + ] + ) + got = CoinbaseAdapter(transport).withdrawals_allowed(["USDC", "BTC", "FET"]) + assert got == {"USDC": True, "BTC": False, "FET": None} + assert transport.get_calls[0][0] == "/v2/accounts" + assert transport.get_calls[0][1]["limit"] == 100 + + +def test_withdrawals_allowed_follows_pagination(): + transport = FakeTransport( + v2_pages=[ + _v2_page([{"id": "a1", "currency": {"code": "BTC"}, "allow_withdrawals": True}], "a1"), + _v2_page([{"id": "a2", "currency": {"code": "USDC"}, "allow_withdrawals": True}]), + ] + ) + got = CoinbaseAdapter(transport).withdrawals_allowed(["USDC", "BTC"]) + assert got == {"USDC": True, "BTC": True} + assert len(transport.get_calls) == 2 + assert "starting_after" not in transport.get_calls[0][1] + assert transport.get_calls[1][1]["starting_after"] == "a1" + + +def test_withdrawals_allowed_raises_when_still_paginating_after_20_pages(): + pages = [ + _v2_page([{"id": f"a{i}", "currency": {"code": "X"}, "allow_withdrawals": True}], f"a{i}") + for i in range(25) + ] + transport = FakeTransport(v2_pages=pages) + with pytest.raises(RuntimeError): + CoinbaseAdapter(transport).withdrawals_allowed(["USDC"]) + assert len(transport.get_calls) == 20 + + +def test_withdrawals_allowed_raises_when_unreadable(): + transport = FakeTransport(v2_raises=ConnectionError("reset")) + with pytest.raises(ConnectionError): + CoinbaseAdapter(transport).withdrawals_allowed(["USDC"]) + + +@pytest.mark.parametrize( + "row", + [ + {"currency": {"code": "USDC"}, "allow_withdrawals": "yes"}, + {"currency": {"code": "USDC"}}, + ], +) +def test_withdrawals_allowed_treats_non_bool_flag_as_unreadable(row: dict[str, Any]): + transport = FakeTransport(v2_pages=[_v2_page([row])]) + with pytest.raises(ValueError): + CoinbaseAdapter(transport).withdrawals_allowed(["USDC"]) + + +def test_coinbase_adapter_satisfies_withdrawal_reader(): + assert isinstance(CoinbaseAdapter(FakeTransport()), WithdrawalReader) diff --git a/tests/broker_fake/test_adapter.py b/tests/broker_fake/test_adapter.py index 157de61b..7ba9094c 100644 --- a/tests/broker_fake/test_adapter.py +++ b/tests/broker_fake/test_adapter.py @@ -10,7 +10,7 @@ import pytest from keel_broker_api.orders import LimitGTC, MarketIOCByBase, MarketIOCByQuote, StopLimitGTC -from keel_broker_api.port import TradeScopeDenied, UnsupportedOrder +from keel_broker_api.port import TradeScopeDenied, UnsupportedOrder, WithdrawalReader from keel_broker_api.results import Balance, CancelOutcome, OrderStatus, PlaceResult, SessionState from keel_broker_fake import FakeAdapter from keel_broker_fake.adapter import MAX_CANDLES_PER_CALL @@ -255,3 +255,13 @@ def test_an_unsupported_kind_is_still_refused_as_unsupported_first() -> None: adapter.place_order( MarketIOCByQuote(product_id="BTC-USD", side=Side.BUY, quote_size=Decimal("100")) ) + + +def test_fake_withdrawals_allowed_defaults_true_and_honours_overrides(): + fake = FakeAdapter(withdrawals_allowed={"BTC": False, "FET": None}) + assert isinstance(fake, WithdrawalReader) + assert fake.withdrawals_allowed(["USDC", "BTC", "FET"]) == { + "USDC": True, + "BTC": False, + "FET": None, + } From 7fd680f058b7312dde5a6345ddb3666a1c835b47 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:19:13 -0500 Subject: [PATCH 04/18] fix(coinbase): duplicate wallets fail closed; correct withdrawals_allowed docstring (#1010) Co-Authored-By: Claude Opus 5.5 --- .../keel_broker_coinbase/adapter.py | 17 +++++++++-------- tests/broker_coinbase/test_adapter.py | 10 ++++++++++ 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py b/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py index 74916c5c..d8901612 100644 --- a/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py +++ b/packages/keel-broker-coinbase/keel_broker_coinbase/adapter.py @@ -409,11 +409,12 @@ def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | Non does not affect this one. **Fails closed, unlike `verify_cash_account`, and for the opposite reason.** That check - can only refute, so silence is harmless; this one is asked to AFFIRM that withdrawals - are off-limits to the key, so a read that did not happen proves nothing. Every failure - raises: transport errors propagate, a flag that is not a real bool on a wallet we were - asked about raises `ValueError` (a string `"yes"` is not an answer), and so does still - paginating after 20 pages -- the caller decides what an unreadable venue means. + can only refute, so silence is harmless; this one is asked to AFFIRM the wallet is not + frozen (`True` is the passing answer), so a read that did not happen proves + nothing. Every failure raises: transport errors propagate, a flag that is not a real + bool on a wallet we were asked about raises `ValueError` (a string `"yes"` is not an + answer), and so does still paginating after 20 pages -- + the caller decides what an unreadable venue means. Pages of 100 wallets, following `pagination.next_starting_after` via `starting_after`. """ @@ -437,9 +438,9 @@ def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | Non raise ValueError( f"Coinbase wallet {code} has an unreadable allow_withdrawals: {flag!r}" ) - # Several wallets for one code is not expected; if it happens, any wallet - # that can withdraw makes the code withdrawable. - found[code] = found.get(code, False) or flag + # Several wallets for one code is not expected; if it happens, a False wins: + # one frozen wallet vetoes the code, because this read fails closed. + found[code] = found.get(code, True) and flag starting_after = _field(_field(response, "pagination"), "next_starting_after") if not starting_after: return {c: found.get(c.upper()) for c in currencies} diff --git a/tests/broker_coinbase/test_adapter.py b/tests/broker_coinbase/test_adapter.py index 955aa649..b8fe4280 100644 --- a/tests/broker_coinbase/test_adapter.py +++ b/tests/broker_coinbase/test_adapter.py @@ -998,3 +998,13 @@ def test_withdrawals_allowed_treats_non_bool_flag_as_unreadable(row: dict[str, A def test_coinbase_adapter_satisfies_withdrawal_reader(): assert isinstance(CoinbaseAdapter(FakeTransport()), WithdrawalReader) + + +@pytest.mark.parametrize("flags", [(True, False), (False, True)]) +def test_withdrawals_allowed_duplicate_wallets_false_wins(flags: tuple[bool, bool]): + rows = [ + {"id": f"a{i}", "currency": {"code": "USDC"}, "allow_withdrawals": f} + for i, f in enumerate(flags) + ] + transport = FakeTransport(v2_pages=[_v2_page(rows)]) + assert CoinbaseAdapter(transport).withdrawals_allowed(["USDC"]) == {"USDC": False} From 3d63154eb45dceb7e79e21f1c1211db8c48a4aac Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:35:12 -0500 Subject: [PATCH 05/18] feat(rail-17): decide withdrawal capability from the venue, drop the 7-day TTL (#1010) Co-Authored-By: Claude Opus 5.5 --- keel/execution/executor.py | 81 ++++++++- keel/execution/guards.py | 24 +-- tests/execution/test_executor.py | 26 +++ tests/execution/test_guards.py | 14 ++ .../execution/test_withdrawal_attestation.py | 93 ----------- tests/execution/test_withdrawal_capability.py | 156 ++++++++++++++++++ tests/test_agent.py | 21 ++- tests/test_cli.py | 20 ++- 8 files changed, 326 insertions(+), 109 deletions(-) delete mode 100644 tests/execution/test_withdrawal_attestation.py create mode 100644 tests/execution/test_withdrawal_capability.py diff --git a/keel/execution/executor.py b/keel/execution/executor.py index 852947a2..6469df0c 100644 --- a/keel/execution/executor.py +++ b/keel/execution/executor.py @@ -509,14 +509,73 @@ def _withdrawals_enabled(repo: Any, now_ts: int) -> bool | None: return bool(enabled) +def manual_withdrawal_suspension(repo: Any) -> bool: + """Has the operator frozen entries with `keel withdrawals attest --suspended`? + + `True` iff the stored `withdrawals_enabled` is exactly `False`. No timestamp is read: the + attestation no longer carries a TTL (#1010) because the VENUE, not the operator, is now the + source of truth for whether withdrawals work -- this flag is only the one-way brake that + reduces capability. An unreadable repo is NOT a suspension (it is not evidence of one), so + this returns `False` and the venue read decides. + """ + try: + return repo.get_state("withdrawals_enabled", default=None) is False + except Exception: + log_exception(logger, "executor.withdrawal_attestation_read_failed") + return False + + +def _withdrawal_capability(broker: Any, repo: Any, product_id: str) -> tuple[bool | None, str]: + """Rail 17's input: `(verdict, detail)` for whether `product_id` is withdrawable (§65.4). + + Read LIVE from the venue on every BUY -- never cached -- so a freeze takes effect on the very + next order and its lifting on the next one after, the same posture rail 14 takes with the + subscription record. The operator's manual suspension is checked first and wins without + touching the venue. + + `True` means every requested wallet answered allowed; `False` means one answered disabled; + `None` (UNKNOWN, which fails rail 17 closed) means the venue could not be asked or could not + answer for the QUOTE leg. The requested wallets are the product's quote leg (what the order + spends) and its base leg (what it acquires). A missing BASE wallet is normal -- the first + buy of an asset the account has never held -- and does not decide; a missing QUOTE wallet is + not, because then the account cannot be funded from a wallet we can verify. + + `detail` says why, for the veto message; it is `""` on a pass. + """ + if manual_withdrawal_suspension(repo): + return False, "manual suspension (keel withdrawals attest --suspended)" + + reader = getattr(broker, "withdrawals_allowed", None) + if reader is None: + return None, "venue adapter cannot report withdrawals" + + quote = quote_currency_of(product_id) + if not quote: + return None, f"no quote currency in {product_id!r}" + base = product_id.rpartition("-")[0].upper() + currencies = [quote] if not base or base == quote else [quote, base] + + try: + answers = reader(currencies) + except Exception as exc: + log_venue_failure(logger, "executor.withdrawal_read_failed", product=product_id) + return None, f"venue unreadable: {type(exc).__name__}" + + if answers.get(quote) is None: + return None, f"no {quote} wallet reported" + for currency in currencies: + if answers.get(currency) is False: + return False, f"{currency} withdrawals disabled at venue" + return True, "" + + #: `agent_state` key prefix for a product's venue-declared `base_increment` (#516). BASE_INCREMENT_PREFIX = "base_increment:" #: How long a cached increment is trusted. Increments change rarely -- a venue re-scaling a #: product is an announced event -- so this is long, and the cost of being wrong is bounded: #: a stale increment is still a multiple the venue accepts unless it got FINER, and a finer one -#: only means we round a little more than needed. Matches rail 17's 7-day attestation TTL rather -#: than inventing a second cadence. +#: only means we round a little more than needed. 7 days. BASE_INCREMENT_TTL_SEC = 7 * 24 * 60 * 60 @@ -929,7 +988,22 @@ def _build_intent( # The PRODUCT's quote leg -- what this order actually spends. available_quote = _fetch_available_quote(broker, quote_currency_of(signal.product_id)) - withdrawals = _withdrawals_enabled(repo, now_ts) + # Live only: paper passes no broker and rail 17 is skipped offline, so there is nothing + # to ask. The read is per BUY and never cached (see `_withdrawal_capability`). + if broker is None: + withdrawals, withdrawals_detail = None, "no venue read (offline)" + else: + withdrawals, withdrawals_detail = _withdrawal_capability( + broker, repo, signal.product_id + ) + log_event( + logger, + logging.INFO if withdrawals else logging.WARNING, + "executor.withdrawal_capability", + product=signal.product_id, + verdict=withdrawals, + detail=withdrawals_detail, + ) return OrderIntent( product_id=signal.product_id, @@ -943,6 +1017,7 @@ def _build_intent( rule_id=signal.rule_id, available_quote=available_quote, withdrawals_enabled=withdrawals, + withdrawals_detail=withdrawals_detail, ) # EXIT: sell the currently held position, reconstructed from the orders audit log. diff --git a/keel/execution/guards.py b/keel/execution/guards.py index 6abc25af..8e1c29d5 100644 --- a/keel/execution/guards.py +++ b/keel/execution/guards.py @@ -215,10 +215,13 @@ class OrderIntent: available_quote: Decimal | None = None # Rail 17 (§65.4 qabd/possession). Whether the account is currently in a state where the - # asset could be withdrawn on demand. Supplied by the caller (the executor) from the - # operator's attestation and any broker-reported restriction -- guards has no broker or - # clock access of its own. `None` means "unknown" and fails the BUY closed. + # asset could be withdrawn on demand. Supplied by the caller (the executor) from the VENUE's + # per-wallet `allow_withdrawals` read (or the operator's manual suspension) -- guards has no + # broker access of its own. `None` means "unknown" and fails the BUY closed. withdrawals_enabled: bool | None = None + # WHY the verdict above is what it is ("venue unreadable: ReadTimeout", "USD withdrawals + # disabled at venue"), so the veto message can name the cause instead of a generic remedy. + withdrawals_detail: str = "" # Rail 9, for a PROTECTIVE order (`place_bracket`/`_roll_stop`): the price this order # triggers at. A bracket carries that price in `entry` and leaves `stop` unset, because it is @@ -913,17 +916,16 @@ def check( if is_buy and not offline: if intent.withdrawals_enabled is None: violations.append( - "withdrawal_capability: UNKNOWN (no fresh attestation, or the broker did not " - "report). Under §65.4 possession requires that nothing prevents withdrawal on " - "demand; an unverified state is not evidence that it holds. Attest with " - "`keel withdrawals attest` (exits and DCA-exempt paths are unaffected)." + f"withdrawal_capability: UNKNOWN ({intent.withdrawals_detail}). Under §65.4 " + "possession requires that nothing prevents withdrawal on demand; an unverified " + "state is not evidence that it holds. Buy skipped; it is retried next cycle " + "within its window." ) elif intent.withdrawals_enabled is False: violations.append( - "withdrawal_capability: withdrawals are suspended/restricted for this account. " - "Under §65.4 an asset that cannot be withdrawn may not have been validly " - "possessed, so new ENTRIES are halted. Existing holdings and exits are " - "deliberately unaffected." + f"withdrawal_capability: SUSPENDED ({intent.withdrawals_detail}). Under §65.4 " + "an asset that cannot be withdrawn may not have been validly possessed, so new " + "entries are halted. Existing holdings and exits are deliberately unaffected." ) # 18. Settlement currency — the order's settlement leg must be one the operator configured diff --git a/tests/execution/test_executor.py b/tests/execution/test_executor.py index 7fa6a088..6d04c913 100644 --- a/tests/execution/test_executor.py +++ b/tests/execution/test_executor.py @@ -16,6 +16,7 @@ import logging import re import sqlite3 +from collections.abc import Sequence from decimal import Decimal from typing import Any from unittest import mock @@ -106,11 +107,18 @@ def __init__( place_order_id: str = "broker-order-1", usdc_balance: Decimal | None = Decimal("1000000"), balances: dict[str, Decimal | None] | None = None, + withdrawals: dict[str, bool | None] | None = None, + withdrawals_raises: BaseException | None = None, ) -> None: # `balances` models a REAL account: one entry per currency. Without it the fake funds # both USD and USDC with `usdc_balance`, so tests that only mean "the account is funded" # keep meaning that -- the mismatch tests below set the two independently on purpose. self._balances = balances + # Rail 17 reads per-wallet `allow_withdrawals` from the venue (#1010). `None` means every + # currency answers True; a dict answers per currency (`None` = no wallet). + self._withdrawals = withdrawals + self._withdrawals_raises = withdrawals_raises + self.withdrawals_calls = 0 # The default preview carries BOTH sides of a TIGHT book, because that is what the # real venue returns (`cb_client.preview_order` maps `best_bid`/`best_ask` to # `Decimal`; see `tests/fixtures/cb_preview_order.json`) and #350's spread gate fails @@ -155,6 +163,14 @@ def get_balances(self) -> list[Balance]: Balance(currency="USDC", available=self._usdc_balance, total=self._usdc_balance), ] + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + self.withdrawals_calls += 1 + if self._withdrawals_raises is not None: + raise self._withdrawals_raises + if self._withdrawals is None: + return {c: True for c in currencies} + return {c: self._withdrawals.get(c) for c in currencies} + def preview_order(self, spec: OrderSpec) -> Preview: self.preview_calls.append({"spec": spec}) return _preview_from(spec, self._preview) @@ -3951,6 +3967,16 @@ def get_accounts(self, **kwargs: Any) -> Any: row["available_balance"]["value"] = "1000000.00" return accounts + def get(self, path: str, params: dict[str, Any] | None = None, **kwargs: Any) -> Any: + """Rail 17's per-wallet `allow_withdrawals` read (`/v2/accounts`), every wallet open.""" + assert path == "/v2/accounts", path + return { + "data": [ + {"currency": {"code": code}, "allow_withdrawals": True} for code in ("USD", "BTC") + ], + "pagination": {"next_starting_after": None}, + } + def get_product(self, product_id: str, **kwargs: Any) -> Any: return self._fixture("cb_product.json") diff --git a/tests/execution/test_guards.py b/tests/execution/test_guards.py index aee3742f..ecf65ed3 100644 --- a/tests/execution/test_guards.py +++ b/tests/execution/test_guards.py @@ -1523,6 +1523,20 @@ def test_rail17_fails_CLOSED_on_unknown(repo: Repository) -> None: assert "withdrawal_capability" in _keys(result) +def test_rail17_messages_carry_the_venue_detail_and_no_attestation_advice( + repo: Repository, +) -> None: + """The remedy is no longer "attest": the verdict comes from the venue, and says why.""" + for state, word in ((None, "UNKNOWN"), (False, "SUSPENDED")): + intent = _intent(withdrawals_enabled=state, withdrawals_detail="USD wallet says no") + result = check(intent, repo, _config(), NOW_TS) + (msg,) = [v for v in result.violations if v.startswith("withdrawal_capability")] + assert word in msg + assert "(USD wallet says no)" in msg + assert "DCA-exempt" not in msg + assert "attest" not in msg + + def test_rail17_passes_when_withdrawals_are_attested_enabled(repo: Repository) -> None: result = check(_intent(withdrawals_enabled=True), repo, _config(), NOW_TS) assert "withdrawal_capability" not in _keys(result) diff --git a/tests/execution/test_withdrawal_attestation.py b/tests/execution/test_withdrawal_attestation.py deleted file mode 100644 index 9eaba213..00000000 --- a/tests/execution/test_withdrawal_attestation.py +++ /dev/null @@ -1,93 +0,0 @@ -"""Rail 17's input: the withdrawal-capability attestation (KB §65.4).""" - -from __future__ import annotations - -from click.testing import CliRunner - -from keel.cli import cli -from keel.data.db import connect, migrate -from keel.data.repository import Repository -from keel.execution.executor import WITHDRAWAL_ATTESTATION_TTL_SEC, _withdrawals_enabled - -NOW = 1_784_505_600 - - -def _repo(tmp_path): - conn = connect(str(tmp_path / "t.db")) - migrate(conn) - return Repository(conn) - - -def test_never_attested_reads_as_UNKNOWN(tmp_path): - assert _withdrawals_enabled(_repo(tmp_path), NOW) is None - - -def test_a_fresh_attestation_is_honoured(tmp_path): - repo = _repo(tmp_path) - repo.set_state("withdrawals_enabled", True) - repo.set_state("withdrawals_attested_at", NOW) - assert _withdrawals_enabled(repo, NOW) is True - - repo.set_state("withdrawals_enabled", False) - assert _withdrawals_enabled(repo, NOW) is False - - -def test_a_STALE_attestation_reads_as_UNKNOWN_not_as_suspended(tmp_path): - """ "Nobody has checked recently" is a different claim from "the broker says suspended". - - Both block entries, but the operator sees a different message and should. - """ - repo = _repo(tmp_path) - repo.set_state("withdrawals_enabled", True) - repo.set_state("withdrawals_attested_at", NOW) - assert _withdrawals_enabled(repo, NOW + WITHDRAWAL_ATTESTATION_TTL_SEC + 1) is None - - -def test_the_attestation_is_read_LIVE_so_a_revocation_takes_effect_immediately(tmp_path): - """Never cached -- same posture rail 14 takes with the subscription record.""" - repo = _repo(tmp_path) - repo.set_state("withdrawals_enabled", True) - repo.set_state("withdrawals_attested_at", NOW) - assert _withdrawals_enabled(repo, NOW) is True - - repo.set_state("withdrawals_enabled", False) - assert _withdrawals_enabled(repo, NOW) is False - - -def test_a_repo_failure_reads_as_UNKNOWN_rather_than_propagating(tmp_path): - class _BrokenRepo: - def get_state(self, *a, **k): - raise RuntimeError("db gone") - - assert _withdrawals_enabled(_BrokenRepo(), NOW) is None - - -# -- CLI ----------------------------------------------------------------------- - - -def test_cli_roundtrip_and_suspension_message(tmp_path, monkeypatch): - db_path = tmp_path / "t.db" - runner = CliRunner() - # `--enabled` RELEASES rail 17's entry halt, so it now demands a typed `yes` at a terminal - # (`--suspended` stays ungated -- it only ever reduces capability). - - # The TTY predicate lives in keel.commands._common; _require_interactive_confirmation - # calls it there, so patch it at its definition (see that module's docstring). - monkeypatch.setattr("keel.commands._common._is_interactive", lambda: True) - - unknown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) - assert "UNKNOWN (never attested)" in unknown.output - - assert ( - runner.invoke( - cli, ["--db", str(db_path), "withdrawals", "attest", "--enabled"], input="yes\n" - ).exit_code - == 0 - ) - assert "ENABLED" in runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]).output - - suspended = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "attest", "--suspended"]) - assert "ENTRIES are now halted" in suspended.output - shown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) - assert "SUSPENDED" in shown.output - assert "exits are unaffected" in shown.output diff --git a/tests/execution/test_withdrawal_capability.py b/tests/execution/test_withdrawal_capability.py new file mode 100644 index 00000000..772c49ef --- /dev/null +++ b/tests/execution/test_withdrawal_capability.py @@ -0,0 +1,156 @@ +"""Rail 17's input: the venue's per-wallet `allow_withdrawals` flag (KB §65.4).""" + +# `repo` is the executor tests' fixture, imported by name and redefined as a parameter. +# ruff: noqa: F811, F401 + +from __future__ import annotations + +from typing import Any + +from click.testing import CliRunner + +from keel.cli import cli +from keel.execution import guards +from keel.execution.executor import _build_intent, execute +from tests.execution.test_executor import ( + NOW_TS, + FakeBroker, + _config, + _enter_signal, + _exit_signal, + _seed_open_position, + _setup, + repo, +) + + +def _buy(repo, broker: Any, signal=None): + return execute( + signal or _enter_signal(), broker, repo, _config(), mode="autonomous", now_ts=NOW_TS + ) + + +def _assert_vetoed(result, broker, detail: str) -> None: + assert result.placed is False + assert result.vetoed_by[0].startswith("withdrawal_capability:") + assert detail in result.vetoed_by[0] + assert broker.place_calls == [] + + +def test_manual_suspension_vetoes_without_reading_the_venue(repo): + repo.set_state("withdrawals_enabled", False) + broker = FakeBroker() + result = _buy(repo, broker) + _assert_vetoed(result, broker, "manual suspension (keel withdrawals attest --suspended)") + assert "SUSPENDED" in result.vetoed_by[0] + assert broker.withdrawals_calls == 0 + + +def test_adapter_without_the_read_is_UNKNOWN(repo): + class _NoReader(FakeBroker): + withdrawals_allowed = None # type: ignore[assignment] + + broker = _NoReader() + result = _buy(repo, broker) + _assert_vetoed(result, broker, "venue adapter cannot report withdrawals") + assert "UNKNOWN" in result.vetoed_by[0] + + +def test_unreadable_venue_is_UNKNOWN_and_names_the_exception(repo): + broker = FakeBroker(withdrawals_raises=RuntimeError("boom")) + result = _buy(repo, broker) + _assert_vetoed(result, broker, "venue unreadable: RuntimeError") + assert "UNKNOWN" in result.vetoed_by[0] + + +def test_missing_quote_wallet_is_UNKNOWN(repo): + broker = FakeBroker(withdrawals={"BTC": True}) + result = _buy(repo, broker) + _assert_vetoed(result, broker, "no USD wallet reported") + assert "UNKNOWN" in result.vetoed_by[0] + + +def test_quote_wallet_disabled_is_SUSPENDED(repo): + broker = FakeBroker(withdrawals={"USD": False, "BTC": True}) + result = _buy(repo, broker) + _assert_vetoed(result, broker, "USD withdrawals disabled at venue") + assert "SUSPENDED" in result.vetoed_by[0] + + +def test_base_wallet_disabled_is_SUSPENDED(repo): + broker = FakeBroker(withdrawals={"USD": True, "BTC": False}) + result = _buy(repo, broker) + _assert_vetoed(result, broker, "BTC withdrawals disabled at venue") + + +def test_all_wallets_allowed_buys(repo): + broker = FakeBroker(withdrawals={"USD": True, "BTC": True}) + assert _buy(repo, broker).placed is True + assert broker.withdrawals_calls == 1 + + +def test_stale_attestation_no_longer_vetoes(repo): + repo.set_state("withdrawals_enabled", True) + repo.set_state("withdrawals_attested_at", NOW_TS - 30 * 86400) + assert _buy(repo, FakeBroker()).placed is True + + +def test_never_attested_account_buys_when_venue_allows(repo): + for key in ("withdrawals_enabled", "withdrawals_attested_at"): + repo.set_state(key, None) + assert _buy(repo, FakeBroker()).placed is True + + +def test_base_wallet_missing_decides_on_quote(repo): + broker = FakeBroker(withdrawals={"USD": True}) # no FET wallet + signal = _enter_signal(_setup(product_id="FET-USD")) + result = _buy(repo, broker, signal) + assert not any(v.startswith("withdrawal_capability") for v in result.vetoed_by) + + +def test_sell_never_reads_withdrawals(repo): + _seed_open_position(repo, "BTC-USD", __import__("decimal").Decimal("0.1"), 50000) + broker = FakeBroker(withdrawals_raises=RuntimeError()) + result = execute(_exit_signal(), broker, repo, _config(), mode="autonomous", now_ts=NOW_TS) + assert result.placed is True + assert broker.withdrawals_calls == 0 + + +def test_paper_never_reads_withdrawals(repo): + """Paper builds its intent with NO broker; rail 17 is recorded as skipped, not vetoed.""" + intent = _build_intent(_enter_signal(), None, repo, _config(), NOW_TS) + assert intent is not None + verdict = guards.check(intent, repo, _config(), NOW_TS, offline=True) + assert not any(v.startswith("withdrawal_capability") for v in verdict.violations) + assert "withdrawal_capability" in verdict.skipped_rails + + +# -- CLI ----------------------------------------------------------------------- + + +def test_cli_roundtrip_and_suspension_message(tmp_path, monkeypatch): + db_path = tmp_path / "t.db" + runner = CliRunner() + # `--enabled` RELEASES rail 17's entry halt, so it now demands a typed `yes` at a terminal + # (`--suspended` stays ungated -- it only ever reduces capability). + + # The TTY predicate lives in keel.commands._common; _require_interactive_confirmation + # calls it there, so patch it at its definition (see that module's docstring). + monkeypatch.setattr("keel.commands._common._is_interactive", lambda: True) + + unknown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) + assert "UNKNOWN (never attested)" in unknown.output + + assert ( + runner.invoke( + cli, ["--db", str(db_path), "withdrawals", "attest", "--enabled"], input="yes\n" + ).exit_code + == 0 + ) + assert "ENABLED" in runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]).output + + suspended = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "attest", "--suspended"]) + assert "ENTRIES are now halted" in suspended.output + shown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) + assert "SUSPENDED" in shown.output + assert "exits are unaffected" in shown.output diff --git a/tests/test_agent.py b/tests/test_agent.py index 83939965..980321a9 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -14,6 +14,7 @@ from __future__ import annotations import logging +from collections.abc import Sequence from dataclasses import replace from decimal import Decimal, InvalidOperation from types import SimpleNamespace @@ -70,8 +71,18 @@ class FakeBroker: `CoinbaseClient`-shaped responses (like `test_executor.FakeBroker`). """ - def __init__(self, series: dict[tuple[str, Granularity], list[Candle]] | None = None) -> None: + def __init__( + self, + series: dict[tuple[str, Granularity], list[Candle]] | None = None, + withdrawals: dict[str, bool | None] | None = None, + withdrawals_raises: BaseException | None = None, + ) -> None: self._series = series or {} + # Rail 17 reads per-wallet `allow_withdrawals` from the venue (#1010). `None` means every + # currency answers True; a dict answers per currency (`None` = no wallet). + self._withdrawals = withdrawals + self._withdrawals_raises = withdrawals_raises + self.withdrawals_calls = 0 self.get_candles_calls: list[tuple[str, Granularity, int, int]] = [] self.preview_calls: list[dict[str, Any]] = [] self.place_calls: list[dict[str, Any]] = [] @@ -91,6 +102,14 @@ def get_balances(self) -> list[Balance]: Balance(currency="USDC", available=Decimal("1000000"), total=Decimal("1000000")), ] + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + self.withdrawals_calls += 1 + if self._withdrawals_raises is not None: + raise self._withdrawals_raises + if self._withdrawals is None: + return {c: True for c in currencies} + return {c: self._withdrawals.get(c) for c in currencies} + def get_candles( self, product_id: str, granularity: Granularity, start: int, end: int ) -> list[Candle]: diff --git a/tests/test_cli.py b/tests/test_cli.py index 57681ad9..c1a7b283 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -13,6 +13,7 @@ from __future__ import annotations import time +from collections.abc import Sequence from decimal import Decimal from pathlib import Path from types import SimpleNamespace @@ -55,13 +56,30 @@ def _repo_at(db_path: Path) -> Repository: class FakeBroker: """No-network fake: canned (empty) candles + always-successful order responses.""" - def __init__(self) -> None: + def __init__( + self, + withdrawals: dict[str, bool | None] | None = None, + withdrawals_raises: BaseException | None = None, + ) -> None: self.get_candles_calls: list[tuple] = [] + # Rail 17 reads per-wallet `allow_withdrawals` from the venue (#1010). `None` means every + # currency answers True; a dict answers per currency (`None` = no wallet). + self._withdrawals = withdrawals + self._withdrawals_raises = withdrawals_raises + self.withdrawals_calls = 0 def get_candles(self, product_id: str, granularity: Any, start: int, end: int) -> list: self.get_candles_calls.append((product_id, granularity, start, end)) return [] + def withdrawals_allowed(self, currencies: Sequence[str]) -> dict[str, bool | None]: + self.withdrawals_calls += 1 + if self._withdrawals_raises is not None: + raise self._withdrawals_raises + if self._withdrawals is None: + return {c: True for c in currencies} + return {c: self._withdrawals.get(c) for c in currencies} + def preview_order(self, spec: OrderSpec) -> Preview: return Preview( product_id=spec.product_id, From cb57e4445a9a210a1f777ba6d565d8f9f6714b55 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:37:45 -0500 Subject: [PATCH 06/18] fix(rail-17): an unreadable suspension store fails closed (#1010) Co-Authored-By: Claude Opus 5.5 --- keel/execution/executor.py | 26 +++++++++++++------ tests/execution/test_withdrawal_capability.py | 19 ++++++++++++++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/keel/execution/executor.py b/keel/execution/executor.py index 6469df0c..3f829d8d 100644 --- a/keel/execution/executor.py +++ b/keel/execution/executor.py @@ -509,20 +509,27 @@ def _withdrawals_enabled(repo: Any, now_ts: int) -> bool | None: return bool(enabled) -def manual_withdrawal_suspension(repo: Any) -> bool: - """Has the operator frozen entries with `keel withdrawals attest --suspended`? +def _stored_suspension(repo: Any) -> bool | None: + """The operator's stored `--suspended` brake: `True` (set), `False` (not set), `None` (unreadable). `True` iff the stored `withdrawals_enabled` is exactly `False`. No timestamp is read: the - attestation no longer carries a TTL (#1010) because the VENUE, not the operator, is now the - source of truth for whether withdrawals work -- this flag is only the one-way brake that - reduces capability. An unreadable repo is NOT a suspension (it is not evidence of one), so - this returns `False` and the venue read decides. + attestation no longer carries a TTL (#1010) because the VENUE is the source of truth for + whether withdrawals work -- this flag is only the one-way brake that reduces capability. """ try: return repo.get_state("withdrawals_enabled", default=None) is False except Exception: log_exception(logger, "executor.withdrawal_attestation_read_failed") - return False + return None + + +def manual_withdrawal_suspension(repo: Any) -> bool: + """Display helper: is a manual suspension stored? An unreadable store reads as `False` here. + + NOT for the rail path: there, not knowing whether a suspension is stored is not evidence + that none is, so `_withdrawal_capability` fails closed on an unreadable store itself. + """ + return _stored_suspension(repo) is True def _withdrawal_capability(broker: Any, repo: Any, product_id: str) -> tuple[bool | None, str]: @@ -542,7 +549,10 @@ def _withdrawal_capability(broker: Any, repo: Any, product_id: str) -> tuple[boo `detail` says why, for the veto message; it is `""` on a pass. """ - if manual_withdrawal_suspension(repo): + suspended = _stored_suspension(repo) + if suspended is None: + return None, "suspension state unreadable" + if suspended: return False, "manual suspension (keel withdrawals attest --suspended)" reader = getattr(broker, "withdrawals_allowed", None) diff --git a/tests/execution/test_withdrawal_capability.py b/tests/execution/test_withdrawal_capability.py index 772c49ef..d60dd6e5 100644 --- a/tests/execution/test_withdrawal_capability.py +++ b/tests/execution/test_withdrawal_capability.py @@ -46,6 +46,25 @@ def test_manual_suspension_vetoes_without_reading_the_venue(repo): assert broker.withdrawals_calls == 0 +def test_unreadable_suspension_state_fails_closed_without_asking_the_venue(repo): + """Not knowing whether a suspension is stored is not evidence that none is.""" + + class _BrokenRead: + def __getattr__(self, name): + return getattr(repo, name) + + def get_state(self, key, *a, **k): + if key == "withdrawals_enabled": + raise RuntimeError("db gone") + return repo.get_state(key, *a, **k) + + broker = FakeBroker() + result = _buy(_BrokenRead(), broker) + _assert_vetoed(result, broker, "suspension state unreadable") + assert "UNKNOWN" in result.vetoed_by[0] + assert broker.withdrawals_calls == 0 + + def test_adapter_without_the_read_is_UNKNOWN(repo): class _NoReader(FakeBroker): withdrawals_allowed = None # type: ignore[assignment] From cbf7b20efb51f4593ecf158163cd9375fa1bfc9d Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:37:49 -0500 Subject: [PATCH 07/18] style: wrap docstring line (#1010) Co-Authored-By: Claude Opus 5.5 --- keel/execution/executor.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keel/execution/executor.py b/keel/execution/executor.py index 3f829d8d..ce203b63 100644 --- a/keel/execution/executor.py +++ b/keel/execution/executor.py @@ -510,7 +510,7 @@ def _withdrawals_enabled(repo: Any, now_ts: int) -> bool | None: def _stored_suspension(repo: Any) -> bool | None: - """The operator's stored `--suspended` brake: `True` (set), `False` (not set), `None` (unreadable). + """The stored `--suspended` brake: `True` (set), `False` (not set), `None` (unreadable). `True` iff the stored `withdrawals_enabled` is exactly `False`. No timestamp is read: the attestation no longer carries a TTL (#1010) because the VENUE is the source of truth for From 682ccd5f400d8d8eef489d2e31741cfbabdffbc4 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 13:58:28 -0500 Subject: [PATCH 08/18] feat(rail-17): withdrawals is a manual switch; remove the expiry machinery (#1010) Rail 17 reads the venue per buy, so nothing about it expires any more. - executor: delete _withdrawals_enabled and WITHDRAWAL_ATTESTATION_TTL_SEC. - withdrawals CLI: attest --enabled/--suspended only set/clear the manual brake (no timestamp, no expiry wording; the typed-yes TTY gate is unchanged). show prints the tri-state suspension and the venue's per-wallet answer via a bounded _build_broker; any build/read failure prints "venue unreadable", exit 0. - attestations: rail 17 leaves the survey; withdrawal_expiry/_withdrawal_dates deleted. - doctor: attestation_findings(subscription, now_ts, withdrawals_suspended): FAIL on suspension, WARN on unreadable store, OK otherwise; never reads the old timestamp. - status/pulse/web: WithdrawalAttestationStatus is just `suspended` (tri-state, unknown never rendered as off); console card shows the manual suspension line. - notifications: no rail-17 attestation event. - setup wizard step explains the venue read and the manual switch. - docs/fiqh-basis.md mechanics sentence + its pin test updated (full docs rewrite is Task 6). Deleted tests (expiry wording, not loosened): test_attestation_expiry.py: test_rail_17_expiry_comes_from_the_executors_constant_and_is_not_restated, test_an_unattested_rail_reports_missing_rather_than_expired, test_rail_17_passes_through_ok_then_expiring_then_expired, test_seconds_left_never_goes_negative, test_doctor_reads_rail_17s_TTL_from_the_executor_rather_than_restating_it, test_doctor_starts_warning_on_rail_17_exactly_when_the_banner_does, test_a_suspended_attestation_is_not_a_healthy_one, test_reordering_the_definitions_does_not_cross_the_wires, test_the_warning_window_never_outlives_the_attestation_it_guards, test_forgetting_rail_17s_verdict_reports_too_loud_rather_than_too_quiet. test_doctor.py: test_rail17_fresh_reports_days_remaining, test_rail17_absent_fails_and_names_the_attest_command, test_rail17_expired_fails_with_days_over. test_status.py: test_rail17_attested_fresh_shows_time_remaining, test_rail17_expired_names_the_halt_and_the_fix, test_rail17_expired_in_paper_names_the_state_not_a_halt, test_rail17_never_attested_is_said_as_such, test_rail17_suspended_attestation_still_names_the_halt, test_rail17_freshness_uses_the_executors_own_ttl. test_notifications.py: test_an_attestation_due_within_two_days_fires_and_a_fresh_one_does_not, test_an_expired_attestation_fires_too, test_both_attestation_rails_are_notified_when_both_are_unhealthy. test_payload.py: test_an_expired_withdrawal_attestation_reads_as_bad. test_fiqh_basis.py: test_rail_17s_seven_day_ttl_is_pinned_to_the_executor (replaced). Rewritten onto rail 22 / new shape: test_attestation_alert_once.py, test_attestation_banner.py, test_pulse.py rail-17 tests. Co-Authored-By: Claude Opus 5.5 --- docs/fiqh-basis.md | 8 +- keel/attestations.py | 144 +++------- keel/commands/doctor.py | 121 +++----- keel/commands/pulse.py | 22 +- keel/commands/setup.py | 36 ++- keel/commands/status.py | 105 ++----- keel/commands/withdrawals.py | 110 +++++--- keel/execution/executor.py | 32 --- keel/notifications.py | 32 +-- keel/web/payload.py | 32 +-- keel/web/static/js/docs.js | 2 +- keel/web/static/js/render.js | 6 +- tests/commands/test_console_thinness.py | 2 +- tests/commands/test_doctor.py | 102 ++++--- tests/commands/test_insights.py | 8 +- tests/commands/test_pulse.py | 35 ++- tests/commands/test_setup.py | 24 +- tests/commands/test_status.py | 122 +++----- tests/execution/test_executor.py | 6 +- tests/execution/test_reduce.py | 1 - tests/execution/test_withdrawal_capability.py | 7 +- tests/test_agent.py | 37 +-- tests/test_attestation_alert_once.py | 176 +++++++----- tests/test_attestation_expiry.py | 265 ++---------------- tests/test_cli.py | 129 +++++++++ tests/test_fiqh_basis.py | 23 +- tests/test_hostile_rule_rails.py | 1 - tests/test_notifications.py | 136 ++++----- tests/web/test_attestation_banner.py | 47 ++-- tests/web/test_client_assets.py | 2 +- tests/web/test_payload.py | 48 ++-- tests/web/test_view_sections.py | 6 +- 32 files changed, 807 insertions(+), 1020 deletions(-) diff --git a/docs/fiqh-basis.md b/docs/fiqh-basis.md index 151f3aad..200ef280 100644 --- a/docs/fiqh-basis.md +++ b/docs/fiqh-basis.md @@ -105,9 +105,11 @@ test: possession is the ability to dispose, not physical custody (§65.4 Ayub · Abuzaid & Oweida's *Handbook of Islamic Finance* (2022) quoting the OIC Fiqh Academy resolution (Res. 53/4-6) that holds electronic constructive possession sufficient. -Mechanics: the operator attests with `keel withdrawals attest`; the attestation is live-read -on every intent and expires after 7 days (`WITHDRAWAL_ATTESTATION_TTL_SEC`, -`keel/execution/executor.py`) — "a stale attestation is no better than none". ENTRIES ONLY, +Mechanics: on every BUY the executor asks the venue whether the wallets the order touches +allow withdrawals (`_withdrawal_capability`, `keel/execution/executor.py`; Coinbase's +`allow_withdrawals`), so a freeze takes effect on the next order and its lifting on the one +after. There is no attestation and no expiry. The operator keeps one manual brake, +`keel withdrawals attest --suspended`, which halts entries whatever the venue says. ENTRIES ONLY, like rails 11/16: existing holdings are already ours, and forcing a sale to "fix" a withdrawal freeze would be strictly worse than holding through it. Unknown fails closed. diff --git a/keel/attestations.py b/keel/attestations.py index 91b4f576..d18e296a 100644 --- a/keel/attestations.py +++ b/keel/attestations.py @@ -1,37 +1,29 @@ """When each attestation expires, and how long is left -- the one place that knows (#793). Live stopped placing orders on 2026-08-25 and nobody noticed for seventeen days. The exposure -caps were the primary cause, but rail 17's attestation expired on 09-08 and stacked on top, and +caps were the primary cause, but an attestation expired on 09-08 and stacked on top, and **nothing anywhere said so**. keel fails closed on a stale attestation, which is correct; it -failed closed SILENTLY, which is the hazard. An operator had to remember a 7-day TTL, or read +failed closed SILENTLY, which is the hazard. An operator had to remember the TTL, or read `guards.check_failed` records, to learn the agent was paused. -── WHY A MODEL AND NOT THREE WARNINGS ──────────────────────────────────────────────────────── +── WHY A MODEL AND NOT A WARNING PER SURFACE ───────────────────────────────────────────────── -Before this module there were four TTL constants in four places, across two incompatible shapes, -and one of them was a copy that claimed not to be: +Before this module there were four TTL constants in four places, and one of them was a copy that +claimed not to be. A banner, a doctor state and a webhook built on that would have been a fifth. +So the model comes first: every attestation reduces to the same two facts -- **when it was made, +and when it stops counting** -- and every surface renders those without computing either. - executor.WITHDRAWAL_ATTESTATION_TTL_SEC 7d rail 17, freshness derived at read time - cash_posture.ATTESTATION_TTL_SEC 90d rail 22, due date STORED at attest time - executor.BASE_INCREMENT_TTL_SEC 7d sizing - doctor.TTL_SEC 7d a duplicate, under the comment - "doctor only READS it" -- it did not - -A banner, a doctor state and a webhook built on that would have been a fifth copy. So the model -comes first: every attestation reduces to the same two facts -- **when it was made, and when it -stops counting** -- and every surface renders those without computing either. - -The two shapes are preserved rather than converted. Rail 17 derives its expiry from the TTL the -executor declares; rail 22 reads `attest_due_ts`, which is written at attest time because "a -record with no due date is a claim that never expires, which this record does not permit". This -module reads each the way its own rail does, and returns one shape. +**Rail 17 left this model in #1010.** It used to be the first definition here, a 7-day +attestation whose TTL the executor declared. Rail 17 now reads the venue's own +`allow_withdrawals` on every buy, so there is nothing to renew and nothing to expire; the only +thing an operator still sets is a manual suspension, which has no date. What remains is rail 22, +whose due date is STORED at attest time because "a record with no due date is a claim that never +expires, which this record does not permit". ── FOUR STATES, AND THE TWO ABSENCES ARE DIFFERENT ─────────────────────────────────────────── -`MISSING` is not `EXPIRED`. `_withdrawals_enabled` already keeps them apart -- "nobody has -checked recently" is not the claim "the broker says withdrawals are suspended" -- and a banner -that called a never-attested rail "expired" would send an operator hunting a lapse that never -happened. +`MISSING` is not `EXPIRED`. A banner that called a never-attested rail "expired" would send an +operator hunting a lapse that never happened. `EXPIRING` is the state that did not exist. An attestation was healthy until the instant it was refused, so the only warning was the halt itself. @@ -45,15 +37,13 @@ from keel_core.cash_posture import ATTESTATION_TTL_SEC as _POSTURE_TTL_SEC -#: The default warning window, and the one rail 17 uses. 48 hours: long enough that a daily agent -#: run and a daily doctor run each get a chance to say so before the halt, short enough that the -#: warning is not background noise for most of a 7-day TTL. +#: The default warning window. 48 hours: long enough that a daily agent run and a daily doctor +#: run each get a chance to say so before the halt. #: -#: **Per-attestation and not global, which the first draft got wrong.** A single window looks -#: tidy until it meets rail 22, whose TTL is 90 days: 48 hours' notice on a quarterly attestation -#: is no notice at all, and `doctor` already knew that -- it warns rail 22 at -#: `ATTESTATION_TTL_SEC // 6` and says why, "proportional to" a fixed TTL. The window belongs to -#: the attestation, so each keeps the schedule its own rail reasoned itself into. +#: **Per-attestation and not global.** 48 hours' notice on rail 22's 90-day attestation is no +#: notice at all, and `doctor` already knew that -- it warns rail 22 at `ATTESTATION_TTL_SEC // 6` +#: and says why, "proportional to" a fixed TTL. The window belongs to the attestation, so each +#: keeps the schedule its own rail reasoned itself into. WARN_WITHIN_SEC = 48 * 3_600 #: Nobody has ever attested this. Distinct from `EXPIRED` -- see the module note. @@ -65,10 +55,9 @@ #: Past its expiry. The rail is vetoing now. EXPIRED = "expired" -#: Attested, unexpired, and the answer was NO. Not an expiry and not an absence: rail 17 tells -#: the two apart in the operator's own words -- "nobody has checked recently" is not the same -#: claim as "the broker says withdrawals are suspended" -- and a surface that reported this one -#: as `ok` would be green while every BUY was refused. `keel doctor` was, until #793. +#: Attested, unexpired, and the answer was NO (rail 22: refuted, margin-enabled, or no due date). +#: Not an expiry and not an absence, and a surface that reported it as `ok` would be green while +#: every BUY was refused. REFUSED = "refused" @@ -82,9 +71,9 @@ class Attestation: rail: str #: What to call it on screen. label: str - #: The command that refreshes it. A TTY command: rails 17 and 22 are Tier 2 and deliberately - #: not admitted to the browser, so a cockpit banner has to TELL the operator what to type -- - #: an alarm with no action is worse than the silence it replaces. + #: The command that refreshes it. A TTY command: rail 22 is Tier 2 and deliberately not + #: admitted to the browser, so a cockpit banner has to TELL the operator what to type -- an + #: alarm with no action is worse than the silence it replaces. remedy: str #: When the attestation was made, or `None` if it never was. attested_at: int | None = None @@ -94,24 +83,18 @@ class Attestation: #: for a short one -- see `WARN_WITHIN_SEC`. warn_within_sec: int = WARN_WITHIN_SEC - #: The rail's own verdict where the rail has one -- the question the CLOCK does not answer. - #: Rail 17's attestation records an ANSWER as well as a date (`withdrawals_enabled`), and - #: `False` halts entries exactly as an expiry does; rail 22's record can be REFUTED by venue - #: evidence or attest a MARGIN-ENABLED account, both of which halt entries with the due date - #: still months away. An earlier draft left rail 22 `True` on the grounds that doctor already - #: reported its verdict and a notification already carried it -- which was true until #793 - #: began suppressing that notification against a window this field is part of. + #: The rail's own verdict -- the question the CLOCK does not answer. Rail 22's record can be + #: REFUTED by venue evidence or attest a MARGIN-ENABLED account, both of which halt entries + #: with the due date still months away. satisfied: bool = True @property def effective_warn_sec(self) -> int: """The window, never longer than the life of the attestation it guards. - `withdrawal_expiry` reads the rail's TTL at call time so that changing it moves every - surface at once -- but the 48-hour window is flat, so a TTL shortened below 48 hours - (the expiry test sets it to three days; a future rail could be tighter still) would make - every fresh attestation `EXPIRING` on the instant it was made, and the banner would be - permanently lit. Capped at half the TTL, so there is always an `OK` half. + The 48-hour window is flat, so an attestation whose whole life is shorter than 48 hours + would be `EXPIRING` on the instant it was made and the banner permanently lit. Capped at + half the life, so there is always an `OK` half. """ if self.attested_at is None or self.expires_at is None: return self.warn_within_sec @@ -148,30 +131,11 @@ def seconds_left(self, now_ts: int) -> int | None: return max(0, int(self.expires_at - now_ts)) -def withdrawal_expiry(*, attested_at: int) -> int: - """When a rail-17 attestation made at `attested_at` stops counting. - - Reads `executor.WITHDRAWAL_ATTESTATION_TTL_SEC` AT CALL TIME rather than importing the value - into a module constant, so changing the rail's TTL changes every surface at once. That is the - property `doctor.TTL_SEC` claimed and did not have, and `tests/test_attestation_expiry.py` - mutates the constant to prove this one does. - """ - from keel.execution import executor - - return int(attested_at) + int(executor.WITHDRAWAL_ATTESTATION_TTL_SEC) - - #: Every attestation a rail can veto on, with no dates filled in. `survey` returns these with the #: deployment's own dates attached. Separate so a surface can describe the SET without a database -#: -- and so `tests/test_attestation_expiry.py` can assert the set is exactly these two, which is -#: what makes adding a third rail an edit to a test that says why it was not covered. +#: -- and so `tests/test_attestation_expiry.py` can assert the set is exactly this one, which is +#: what makes adding a rail an edit to a test that says why it was not covered. _DEFINITIONS: tuple[Attestation, ...] = ( - Attestation( - key="withdrawal", - rail="17", - label="Withdrawal capability", - remedy="keel withdrawals attest --enabled", - ), Attestation( key="cash_posture", rail="22", @@ -193,8 +157,7 @@ def survey_definitions() -> tuple[Attestation, ...]: def definition(key: str) -> Attestation: """One dateless definition, for a surface that already holds the dates. - `doctor` is that surface: it is handed `withdrawals_attested_at` by its caller and needs the - window and the TTL, not another read of the same row. + A surface that already holds the dates needs the window, not another read of the same row. """ for candidate in _DEFINITIONS: if candidate.key == key: @@ -210,13 +173,12 @@ def survey(repo: Any, now_ts: int, *, venue: str = "coinbase") -> Sequence[Attes briefly locked would be worse than one that says "nobody has attested". """ readers = { - "withdrawal": lambda: _withdrawal_dates(repo), "cash_posture": lambda: _posture_dates(repo, venue), } - # By KEY, never by index. `_DEFINITIONS[0]`/`[1]` was the first spelling and it fails two - # ways in silence: a third rail appended reaches `survey_definitions()` and never `survey()`, - # so it can never be banner, doctor or ledger; and a REORDER attaches rail 17's dates to rail - # 22's definition. A rail with no reader here surveys as `MISSING`, which is the safe answer + # By KEY, never by index. `_DEFINITIONS[0]` was the first spelling and it fails two ways in + # silence: a rail appended reaches `survey_definitions()` and never `survey()`, so it can + # never be banner, doctor or ledger; and a REORDER attaches one rail's dates to another's + # definition. A rail with no reader here surveys as `MISSING`, which is the safe answer # -- it says "nobody has attested" rather than "this is fine". return tuple( replace(definition, **readers[definition.key]()) @@ -226,32 +188,6 @@ def survey(repo: Any, now_ts: int, *, venue: str = "coinbase") -> Sequence[Attes ) -def _withdrawal_dates(repo: Any) -> dict[str, Any]: - """Rail 17's dates, read the way `_withdrawals_enabled` reads them. - - `withdrawals_enabled` is checked as well as the timestamp, because the rail treats an - attestation with no verdict as no attestation -- and a surface that showed a countdown for - one would be counting down to the expiry of nothing. - """ - try: - attested_at = int(repo.get_state("withdrawals_attested_at", default=0) or 0) - enabled = repo.get_state("withdrawals_enabled", default=None) - if not attested_at or enabled is None: - return {"attested_at": None, "expires_at": None} - # INSIDE the `try`, with the reads. It was outside, which made "never raises" not quite - # true -- and the caller that would have worn it is `notify_after_cycle`, whose single - # broad `except` would have swallowed the whole cycle's notifications, `rail.armed` and - # `setup.unplaced` included, for a fault in the attestation survey. - return { - "attested_at": attested_at, - "expires_at": withdrawal_expiry(attested_at=attested_at), - # `bool` of the stored value, matching `executor._withdrawals_enabled`'s last line. - "satisfied": bool(enabled), - } - except Exception: - return {"attested_at": None, "expires_at": None} - - def _posture_dates(repo: Any, venue: str) -> dict[str, Any]: """Rail 22's dates, from the record's own STORED due date. diff --git a/keel/commands/doctor.py b/keel/commands/doctor.py index 4c947b2c..e2ce3e99 100644 --- a/keel/commands/doctor.py +++ b/keel/commands/doctor.py @@ -30,7 +30,7 @@ import json import re from collections.abc import Callable, Iterable -from dataclasses import dataclass, replace +from dataclasses import dataclass from datetime import UTC, datetime from decimal import ROUND_CEILING, ROUND_HALF_UP, Decimal, InvalidOperation from pathlib import Path @@ -46,7 +46,6 @@ from keel_core.telemetry import current_venue from keel_core.trade_scope import READ_ONLY, TRADING, TradeScopeState, VenueTradeScope -from keel import attestations from keel.data.feed_scope import reports_consolidated_volume from keel.data.freshness import Freshness from keel.execution import guards, sizing @@ -90,26 +89,20 @@ def _days(seconds: float) -> int: def attestation_findings( subscription: Any | None, - withdrawals_attested_at: int, now_ts: int, - withdrawals_enabled: bool | None = None, + withdrawals_suspended: bool | None, ) -> list[Finding]: - """Rails 14 and 17 -- days remaining, not just valid/invalid. - - Rail 17's TTL and its warning window both come from `keel.attestations`, which reads the - executor's constant. There was a `ttl_sec: int = TTL_SEC` parameter here and a module-level - `TTL_SEC = 7 * 86_400` above it under a comment reading "doctor only READS it" -- a second - copy of a number the comment promised was an import. Both callers used the default. - - `withdrawals_enabled` is the rail's OTHER key, and its absence here was the same lie one - column over: rail 17 vetoes on `enabled is False` with its own sentence, and this function, - handed the timestamp alone, reported `ok · 6 day(s) remain` for an account whose broker had - frozen withdrawals. It defaults to **`None`**, not `True`: `None` is the rail's own word for - "nobody has checked", it routes to the never-attested FAIL below, and a caller who forgets - the argument therefore gets a finding that is too LOUD rather than one that is green for a - frozen account. `cash_posture.py:91-94` states the rule for exactly this shape -- "inventing - one at read time would let a writer forget to set one and have the reader quietly cover for - it". `bool | None` rather than `Any`, so mypy can object. + """Rails 14 and 17 -- rail 14 by days remaining, rail 17 by the manual suspension alone. + + Rail 17 reads the venue's own `allow_withdrawals` on every buy (#1010), so there is no + attestation to age and this function never reads `withdrawals_attested_at`. What it reports + is the one thing an operator still sets: the manual `--suspended` brake. + + `withdrawals_suspended` is `executor._stored_suspension`'s tri-state, passed through + unchanged and with no default: `True` FAILs (entries are halted), `False` passes, and + `None` -- the store could not be read -- WARNs, never passes. The rail fails closed on an + unreadable store, so a doctor that called it `ok` would be green while every BUY was refused. + This function cannot see the venue, so its pass says only what rail 17 will do. """ findings: list[Finding] = [] @@ -157,73 +150,38 @@ def attestation_findings( ) ) - # `enabled is None` joins a missing timestamp rather than becoming a REFUSED, and the rail's - # own words are why: it vetoes UNKNOWN with "no fresh attestation, or the broker did not - # report" and `False` with "withdrawals are suspended/restricted for this account". Those are - # different facts about the account and an operator acts on them differently. - if withdrawals_attested_at <= 0 or withdrawals_enabled is None: + if withdrawals_suspended is None: + findings.append( + Finding( + "attest.withdrawals", + WARN, + "manual withdrawal suspension unreadable", + "the stored switch could not be read, so rail 17 fails closed and halts every " + "BUY entry until it can be", + "keel withdrawals show", + ) + ) + elif withdrawals_suspended: findings.append( Finding( "attest.withdrawals", FAIL, - "withdrawal capability never attested", - "rail 17 (qabd) halts every BUY entry until it is", + "withdrawals manually suspended", + "rail 17 halts every BUY entry until the suspension is cleared; exits are " + "unaffected", "keel withdrawals attest --enabled", ) ) else: - rail17 = replace( - attestations.definition("withdrawal"), - attested_at=withdrawals_attested_at, - expires_at=attestations.withdrawal_expiry(attested_at=withdrawals_attested_at), - satisfied=bool(withdrawals_enabled), - ) - assert rail17.expires_at is not None - ttl_sec = rail17.expires_at - withdrawals_attested_at - age = now_ts - withdrawals_attested_at - remaining = _days(ttl_sec - age) - state = rail17.state(now_ts) - if state == attestations.REFUSED: - findings.append( - Finding( - "attest.withdrawals", - FAIL, - "withdrawals suspended for this account", - "the attestation is fresh and says NO; rail 17 halts entries until it says " - "otherwise", - "keel withdrawals attest --enabled", - ) - ) - elif state == attestations.EXPIRED: - findings.append( - Finding( - "attest.withdrawals", - FAIL, - "withdrawal attestation expired", - f"expired {abs(_days(age - ttl_sec))} day(s) ago; rail 17 halts entries", - "keel withdrawals attest --enabled", - ) - ) - elif state == attestations.EXPIRING: - findings.append( - Finding( - "attest.withdrawals", - WARN, - "withdrawal attestation due soon", - f"{remaining} day(s) remain on the {ttl_sec // 86_400}-day TTL", - "keel withdrawals attest --enabled", - ) - ) - else: - findings.append( - Finding( - "attest.withdrawals", - OK, - "withdrawal capability attested", - f"{remaining} day(s) remain on the {ttl_sec // 86_400}-day TTL", - "-", - ) + findings.append( + Finding( + "attest.withdrawals", + OK, + "no manual withdrawal suspension", + "rail 17 reads withdrawal capability from the venue on every buy", + "-", ) + ) return findings @@ -2390,11 +2348,10 @@ def gather_findings(repo: Any, config: Any, log_lines: Iterable[str], now_ts: in subscription = repo.get_broker_subscription(venue) findings += attestation_findings( subscription=subscription, - withdrawals_attested_at=int(repo.get_state("withdrawals_attested_at", default=0) or 0), - # Rail 17's other key. `None` -- never attested either way -- reaches the finding as the - # falsey it is, and `_withdrawals_attested_at` being 0 gets there first anyway. - withdrawals_enabled=repo.get_state("withdrawals_enabled", default=None), now_ts=now_ts, + # The tri-state, so an unreadable store reaches the finding as `None` (WARN), never as + # an `off` that reads green. + withdrawals_suspended=executor_mod._stored_suspension(repo), ) # #718: a RECORDED window (`--attest-due`) that nothing reads back is a column nobody # benefits from. Reporting only -- see `_attestation_window_findings`'s docstring for why diff --git a/keel/commands/pulse.py b/keel/commands/pulse.py index 20413251..bcf1926d 100644 --- a/keel/commands/pulse.py +++ b/keel/commands/pulse.py @@ -57,11 +57,6 @@ ) -#: The attestation states rail 17 halts entries on -- the ones `status._rail17_line` renders -#: with "entries halted". "attested" passes; "unknown" is unreadable, and status says so itself. -_RAIL17_HALTING = frozenset({"suspended", "expired", "unattested"}) - - @dataclass(frozen=True) class TodayCounts: """The scoped cycles' five counts (plus the cycle count), summed. `blocked` is the wide one @@ -189,17 +184,22 @@ def _last_cycle_ts(feed: ActivityFeed) -> float | None: def _why_not(status: StatusReport, feed: ActivityFeed) -> tuple[str, ...]: """Standing blockers, then every reason today's cycles recorded, newest cycle first, each - sentence once. Rail 17 is named in every mode but paper, for every state that - `status._rail17_line` says halts entries (suspended, expired, never attested): it is a - LIVE_STATE rail, not evaluated in paper, so a lapsed attestation there halts nothing.""" + sentence once. Rail 17 is named in every mode but paper, when the manual suspension is on or + its store is unreadable (the rail fails closed on that): it is a LIVE_STATE rail, not + evaluated in paper, so a suspension there halts nothing.""" reasons: list[str] = [] if status.kill_switch_engaged: reasons.append(KILL_ENGAGED_LINE) if status.rail11_status == "HALTED": reasons.append(f"rail 11: {status.rail11_status}") - attestation = status.withdrawal_attestation.state - if status.mode != "paper" and attestation in _RAIL17_HALTING: - reasons.append(f"rail 17: withdrawal attestation {attestation}") + suspended = status.withdrawal_attestation.suspended + if status.mode != "paper": + if suspended is None: + reasons.append( + "rail 17: manual suspension unknown (store unreadable); buys fail closed" + ) + elif suspended: + reasons.append("rail 17: manual withdrawal suspension is on") # `feed.cycles` is newest-first (`feed_from_lines`), so iteration order is the order asked for. for cycle in feed.cycles: for highlight in cycle.highlights: diff --git a/keel/commands/setup.py b/keel/commands/setup.py index bd883fda..731a9796 100644 --- a/keel/commands/setup.py +++ b/keel/commands/setup.py @@ -275,15 +275,18 @@ def blocking(self) -> bool: ), Step( key="withdrawals_attested", - title="Withdrawal capability attested (rail 17)", + title="Withdrawal capability (rail 17)", kind=StepKind.JUDGEMENT, stage=Stage.LIVE, why=( "Holding a balance at a venue you cannot withdraw from is a compliance precondition, " - "not a technical one, so the rail fails closed until a human says otherwise. The " - "attestation carries a 7-day TTL and needs a terminal." + "not a technical one. Rail 17 asks the venue itself: before every BUY it reads " + "Coinbase's per-wallet allow_withdrawals flag and fails closed if it cannot. There " + "is nothing to attest and nothing to renew. The only thing left to you is a manual " + "brake, `keel withdrawals attest --suspended`, which halts new entries whatever the " + "venue says until you clear it (that needs a terminal)." ), - how="keel withdrawals attest --enabled (needs an interactive terminal)", + how="nothing to do; keel withdrawals show (what the venue says, per wallet)", ), Step( key="scope_attested", @@ -629,20 +632,31 @@ def _asset_observation(conn: sqlite3.Connection, config: Any) -> tuple[bool | No def _withdrawals_observation(conn: sqlite3.Connection) -> tuple[bool | None, str]: - """Rail 17's attestation, read from the same `state` row `keel withdrawals show` reads. + """Rail 17's step: done unless the operator's manual suspension is set. - Presence is reported, NOT freshness. The attestation carries a 7-day TTL and the status - report computes it properly; duplicating that arithmetic here would be a second place to get - it wrong, and this checklist's question is "has this ever been done", not "is it fresh now". + Rail 17 reads the venue on every buy (#1010), so there is no attestation whose presence or + freshness could be reported -- and this checklist is a pure read of the database, so it makes + no venue call. It reports the one stored fact an operator controls, read from the same + `state` row `keel withdrawals show` reads. An unreadable value is `None`, not done. """ if not _table_exists(conn, STATE_TABLE): - return (False, "no withdrawal attestation") + return (True, "read from the venue on every buy -- nothing to attest") row = conn.execute( f"SELECT value FROM {STATE_TABLE} WHERE key = ?", (WITHDRAWALS_STATE_KEY,) ).fetchone() if row is None: - return (False, "no withdrawal attestation") - return (True, "attested at some point -- keel status reports whether it is still fresh") + return (True, "read from the venue on every buy -- nothing to attest") + try: + suspended = json.loads(row[0]) is False + except TypeError, ValueError: + return (None, "the manual-suspension row could not be decoded") + if suspended: + return ( + False, + "manually suspended -- rail 17 halts entries; clear with keel withdrawals attest " + "--enabled", + ) + return (True, "read from the venue on every buy -- nothing to attest") def _utc_date(ts: int) -> str: diff --git a/keel/commands/status.py b/keel/commands/status.py index 758ec203..20854d18 100644 --- a/keel/commands/status.py +++ b/keel/commands/status.py @@ -99,20 +99,19 @@ class SubscriptionStatusRow: @dataclass(frozen=True) class WithdrawalAttestationStatus: - """Rail 17's input, as a displayable state (§65.4) -- the same reading `keel withdrawals - show` prints, so the two commands can never disagree about whether entries are halted. + """Rail 17's operator-set input, as a displayable state (§65.4) -- the manual suspension. - `expires_in_sec` is set only when `state == "attested"` (a fresh `--enabled` attestation); - `expired_for_sec` only when `state == "expired"` -- how long ago the 7-day TTL lapsed, not - how old the attestation is, because "unknown for 12 days" is the fact that matters to an - operator deciding how urgently to re-attest. + Rail 17 decides from the VENUE's own `allow_withdrawals` on every buy (#1010), and this + dashboard is "local DB + config only", so it makes no venue call and shows only what an + operator can set: the `keel withdrawals attest --suspended` brake. There is no expiry to + show because there is no attestation to age. + + `suspended` is `executor._stored_suspension`'s tri-state: `True` (set), `False` (not set), + `None` (the store could not be read -- rendered `unknown`, never `off`, because the rail + fails closed on it). """ - state: str # "attested" | "suspended" | "expired" | "unattested" - enabled: bool | None - attested_at: int | None - expires_in_sec: int | None - expired_for_sec: int | None + suspended: bool | None @dataclass(frozen=True) @@ -301,45 +300,11 @@ def _market_session(repo: Repository, config: Config, now_ts: int) -> MarketSess ) -def _withdrawal_attestation(repo: Repository, now_ts: int) -> WithdrawalAttestationStatus: - """Rail 17's input, resolved and aged for display. - - Resolution goes through the executor's OWN `_withdrawals_enabled` and expiry math goes - through the executor's OWN `WITHDRAWAL_ATTESTATION_TTL_SEC` -- never a restated 7 days -- - so this display can never call an attestation fresh on the very cycle the rail vetoes it. - Staleness takes precedence over suspension, matching `withdrawals show`: a stale attested - suspension is UNKNOWN, not a fresh claim that withdrawals are down. - - The corrupt-state read is guarded the way the executor guards its own (`try/except` -> - unknown): a dashboard must not inherit a crash path the rail deliberately does not have. - """ - resolved = executor._withdrawals_enabled(repo, now_ts) - try: - attested_at = int(repo.get_state("withdrawals_attested_at", default=0) or 0) - except TypeError, ValueError: - attested_at = 0 - - if not attested_at: - return WithdrawalAttestationStatus("unattested", None, None, None, None) - enabled_flag = repo.get_state("withdrawals_enabled", default=None) - if resolved is None and enabled_flag is None: - # `withdrawals show`'s UNKNOWN corner, kept word-for-word: an attested_at exists but - # the enabled flag itself is unreadable/absent, which only DB surgery produces (the - # CLI writes both keys). Not "expired" -- the staleness question was never reached. - return WithdrawalAttestationStatus("unknown", None, attested_at, None, None) - if resolved is None: - expired_for = max((now_ts - attested_at) - executor.WITHDRAWAL_ATTESTATION_TTL_SEC, 0) - return WithdrawalAttestationStatus( - "expired", - enabled_flag, - attested_at, - None, - expired_for, - ) - if resolved is False: - return WithdrawalAttestationStatus("suspended", False, attested_at, None, None) - expires_in = max((attested_at + executor.WITHDRAWAL_ATTESTATION_TTL_SEC) - now_ts, 0) - return WithdrawalAttestationStatus("attested", True, attested_at, expires_in, None) +def _withdrawal_attestation(repo: Repository) -> WithdrawalAttestationStatus: + """Rail 17's manual suspension, resolved through the executor's OWN `_stored_suspension` so + this display can never disagree with the rail about whether the brake is set, and keeping the + tri-state: an unreadable store is `None`, not `False`.""" + return WithdrawalAttestationStatus(executor._stored_suspension(repo)) def gather_status(repo: Repository, config: Config, now_ts: int) -> StatusReport: @@ -372,7 +337,7 @@ def gather_status(repo: Repository, config: Config, now_ts: int) -> StatusReport max_total_dd_pct=max_total, max_weekly_dd_pct=max_weekly, rail11_status=_rail11_status(dd_total, dd_weekly, max_total, max_weekly), - withdrawal_attestation=_withdrawal_attestation(repo, now_ts), + withdrawal_attestation=_withdrawal_attestation(repo), paper_cash_usdc=( repo.get_state("paper_cash_usdc") if config.auto_trade.mode == "paper" else None ), @@ -417,31 +382,21 @@ def _human_remaining(remaining_sec: int) -> str: def _rail17_line(w: WithdrawalAttestationStatus, rail_evaluated: bool) -> str: - """The rail-17 line, naming the halt and the release in the same breath whenever entries - are halted -- the 2026-08-14 event was invisible until the veto fired precisely because - nothing surfaced the attestation's age BEFORE rail 17 acted on it (#340). - - `rail_evaluated` is False in paper mode: rail 17 is a LIVE_STATE rail, skipped offline, - so a stale attestation halts nothing there. Claiming "entries halted" on a paper - dashboard would be a permanently-red alert for a halt that cannot occur -- the exact - alert-fatigue failure #340 exists to fix -- so the paper rendering names the state and - says the rail is not evaluated, and the re-attest prompt is kept (the LIVE deployment's - attestation is refreshed by the same typed command, and paper status should still nudge). + """The rail-17 line: the manual suspension, on, off or unknown. + + `rail_evaluated` is False in paper mode: rail 17 is a LIVE_STATE rail, skipped offline, so a + suspension halts nothing there. Claiming "entries halted" on a paper dashboard would be a + permanently-red alert for a halt that cannot occur (#340), so the paper rendering names the + state and says the rail is not evaluated. """ - prefix = "rail 17 (withdrawal capability)" - if w.state == "attested": - return f"{prefix}: attested, expires in {_human_remaining(w.expires_in_sec or 0)}" - if w.state == "unknown": - return f"{prefix}: UNKNOWN (state unreadable); re-attest with keel withdrawals attest" - halt = " -- entries halted" if rail_evaluated else " (rail 17 not evaluated in paper)" - if w.state == "suspended": - return f"{prefix}: SUSPENDED{halt}; re-attest with keel withdrawals attest --enabled" - if w.state == "expired": - return ( - f"{prefix}: EXPIRED {_human_age(w.expired_for_sec or 0)}{halt}; " - "re-attest with keel withdrawals attest" - ) - return f"{prefix}: never attested{halt}; re-attest with keel withdrawals attest" + prefix = "rail 17 (withdrawal capability): manual suspension" + if w.suspended is None: + return f"{prefix}: unknown (store unreadable)" + if not w.suspended: + return f"{prefix}: off" + if not rail_evaluated: + return f"{prefix}: ON (rail 17 not evaluated in paper)" + return f"{prefix}: ON -- entries halted; clear with keel withdrawals attest --enabled" def _session_line(s: MarketSessionStatus) -> str | None: diff --git a/keel/commands/withdrawals.py b/keel/commands/withdrawals.py index ed8f1e38..c2d3ab40 100644 --- a/keel/commands/withdrawals.py +++ b/keel/commands/withdrawals.py @@ -1,23 +1,39 @@ -"""`keel withdrawals` -- withdrawal-capability attestation, rail 17's input (KB §65.4 qabd). +"""`keel withdrawals` -- rail 17's manual switch and a view of what the venue says (§65.4 qabd). + +Rail 17 no longer asks the operator to vouch for withdrawals on a clock (#1010): it reads +Coinbase's per-wallet `allow_withdrawals` on every BUY (`executor._withdrawal_capability`). What +is left for a human is one manual brake: `--suspended` halts entries regardless of what the venue +says, and `--enabled` clears it. Neither carries an expiry. `--enabled` RELEASES a rail-17 entry halt, so it demands a typed `yes` at a terminal via the -shared confirmation gate; `--suspended` only ever reduces capability and stays ungated. The group -reads and writes attestation state through the repository seam and never touches the broker. +shared confirmation gate; `--suspended` only ever reduces capability and stays ungated. `attest` +reads and writes through the repository seam; only `show` touches the broker, to print what the +venue answers. """ from __future__ import annotations -import time - import click - -from keel.commands._common import _open_repo, _require_interactive_confirmation, with_disclaimer +from keel_core.products import quote_currency_of + +from keel.commands._common import ( + _build_broker, + _load_cfg, + _open_repo, + _require_interactive_confirmation, + with_disclaimer, +) +from keel.data.repository import Repository from keel.execution import executor +#: How long `show` waits on the venue. Bounded because `show` is a viewer an operator runs by +#: hand: a hung network call must degrade to "venue unreadable", not hang the terminal. +SHOW_VENUE_TIMEOUT_SEC = 15 + @click.group("withdrawals") def withdrawals_group() -> None: - """Withdrawal-capability attestation -- rail 17's input (KB §65.4 qabd/possession).""" + """Rail 17's manual withdrawal switch and the venue's answer (KB §65.4 qabd/possession).""" #: The typed gate's exact action/detail wording (O3), module-level so the console's @@ -35,16 +51,17 @@ def withdrawals_group() -> None: "--enabled/--suspended", "enabled", required=True, - help="Are BTC/ETH/PAXG/USDC balances withdrawable on demand right now?", + help="--suspended halts new entries by hand; --enabled clears that suspension.", ) @click.pass_context @with_disclaimer def withdrawals_attest(ctx: click.Context, enabled: bool) -> None: - """Attest the account's current withdrawal state. + """Set or clear rail 17's manual suspension. Under §65.4 possession (`qabd`) holds only while "there is nothing to prevent the buyer from - taking physical possession whenever he desires". An asset we cannot withdraw is an asset we - may not validly possess -- so rail 17 halts new ENTRIES when this is suspended or unknown. + taking physical possession whenever he desires". Rail 17 now asks the VENUE that on every + buy; this switch is the operator's own brake on top of it. `--suspended` halts new ENTRIES + until cleared, whatever the venue says. Neither direction expires. **Asymmetric, like `autonomy`.** `--suspended` only ever REDUCES capability and is ungated, usable from anywhere. `--enabled` RELEASES a rail-17 entry halt, so it demands a typed `yes` @@ -59,32 +76,59 @@ def withdrawals_attest(ctx: click.Context, enabled: bool) -> None: repo = _open_repo(ctx) if enabled: _require_interactive_confirmation(WITHDRAWALS_ATTEST_ACTION, WITHDRAWALS_ATTEST_DETAIL) - now_ts = int(time.time()) repo.set_state("withdrawals_enabled", bool(enabled)) - repo.set_state("withdrawals_attested_at", now_ts) - ttl_days = executor.WITHDRAWAL_ATTESTATION_TTL_SEC // 86400 - state = "ENABLED" if enabled else "SUSPENDED" - click.echo(f"withdrawals attested {state}; expires in {ttl_days} days") - if not enabled: - click.echo("new ENTRIES are now halted (rail 17). Exits are deliberately unaffected.") + if enabled: + click.echo( + "manual suspension cleared; rail 17 reads withdrawal capability from the venue " + "on every buy." + ) + else: + click.echo("manual suspension set. new ENTRIES are now halted (rail 17).") + click.echo( + "Exits are deliberately unaffected. Clear it with: keel withdrawals attest --enabled" + ) + + +#: What the venue's answer for one wallet reads as, keyed by the adapter's tri-state. +_WALLET_WORDS = {True: "allowed", False: "withdrawals disabled", None: "no wallet"} + + +def _suspension_line(suspended: bool | None) -> str: + """The stored brake, tri-state: an unreadable store is `unknown`, never `off` -- the rail + itself fails closed on it, so a display saying `off` would be green for a refused buy.""" + if suspended is None: + return "manual suspension: unknown (store unreadable)" + return "manual suspension: ON -- entries halted" if suspended else "manual suspension: off" + + +def _wallet_currencies(repo: Repository) -> list[str]: + """The quote currency plus the base of every LIVE rule's product, in first-seen order.""" + currencies: list[str] = [] + for row in repo.get_rules(status="live"): + product_id = (row.get("params") or {}).get("product_id") + quote = quote_currency_of(product_id) + if not quote: + continue + for currency in (quote, str(product_id).rpartition("-")[0].upper()): + if currency and currency not in currencies: + currencies.append(currency) + return currencies or ["USD"] @withdrawals_group.command("show") @click.pass_context def withdrawals_show(ctx: click.Context) -> None: - """Show the current attestation and whether it is still fresh.""" + """Show the manual suspension and what the venue says about each wallet rail 17 reads.""" repo = _open_repo(ctx) - now_ts = int(time.time()) - resolved = executor._withdrawals_enabled(repo, now_ts) - attested_at = int(repo.get_state("withdrawals_attested_at", default=0) or 0) - - if resolved is None and not attested_at: - click.echo("withdrawals: UNKNOWN (never attested) -- rail 17 blocks new entries") + click.echo(_suspension_line(executor._stored_suspension(repo))) + try: + config = _load_cfg(ctx) + broker = _build_broker(config, timeout=SHOW_VENUE_TIMEOUT_SEC, repo=repo) + answers = broker.withdrawals_allowed(_wallet_currencies(repo)) + except Exception: + # A viewer, not a gate: a venue that cannot be asked is reported, never raised, so the + # operator can still read the manual-suspension line above it. + click.echo("venue unreadable -- rail 17 refuses entries while it cannot read the venue") return - age_days = (now_ts - attested_at) / 86400 if attested_at else 0 - stale = resolved is None and attested_at - label = {True: "ENABLED", False: "SUSPENDED", None: "UNKNOWN (attestation STALE)"}[resolved] - click.echo(f"withdrawals: {label}") - click.echo(f"attested {age_days:.1f} days ago") - if stale or resolved is False: - click.echo("rail 17 is blocking new entries; exits are unaffected") + for currency, allowed in answers.items(): + click.echo(f" {currency:<8}{_WALLET_WORDS[allowed]}") diff --git a/keel/execution/executor.py b/keel/execution/executor.py index ce203b63..9b18f179 100644 --- a/keel/execution/executor.py +++ b/keel/execution/executor.py @@ -477,38 +477,6 @@ def _dca_budget(context: dict[str, Any], fallback: Decimal) -> tuple[Decimal, st return amount, "rule" -#: How long a withdrawal-capability attestation stays fresh (§65.4). Deliberately short: the -#: attestation is about the account's CURRENT state, and a freeze can appear at any time, so a -#: stale attestation is no better than none. 7 days. -WITHDRAWAL_ATTESTATION_TTL_SEC = 7 * 24 * 3600 - - -def _withdrawals_enabled(repo: Any, now_ts: int) -> bool | None: - """Rail 17's input: is the account currently in a withdrawable state (§65.4)? - - Read LIVE from the operator's attestation on every intent -- never cached -- so - `keel withdrawals attest` (or a revocation) takes effect on the very next order, the same - posture rail 14 takes with the subscription record. - - Returns `None` when there is no attestation or it has gone stale, which fails rail 17 closed. - An expired attestation is treated as UNKNOWN rather than as `False`: the difference matters - in the message the operator sees, and "nobody has checked recently" is not the same claim as - "the broker says withdrawals are suspended". - """ - try: - attested_at = int(repo.get_state("withdrawals_attested_at", default=0) or 0) - enabled = repo.get_state("withdrawals_enabled", default=None) - except Exception: - log_exception(logger, "executor.withdrawal_attestation_read_failed") - return None - - if not attested_at or enabled is None: - return None - if now_ts - attested_at > WITHDRAWAL_ATTESTATION_TTL_SEC: - return None - return bool(enabled) - - def _stored_suspension(repo: Any) -> bool | None: """The stored `--suspended` brake: `True` (set), `False` (not set), `None` (unreadable). diff --git a/keel/notifications.py b/keel/notifications.py index 7eb76075..c3168639 100644 --- a/keel/notifications.py +++ b/keel/notifications.py @@ -7,7 +7,7 @@ * `events_from_state` is PURE. It takes doctor's own finding lists (`keel.commands.doctor.attestation_findings`/`rail_state_findings` outputs), the allowance numbers doctor's `allowance_findings` receives, and the cycle facts the agent loop already - records. It re-implements NO threshold math -- rail 17's WARN-at-<=2-days and the rails' + records. It re-implements NO threshold math -- rail 22's WARN window and the rails' armed/healthy verdicts are read off doctor's findings, so the notification layer cannot drift from the surface an operator diagnoses with. @@ -69,24 +69,25 @@ #: learn about the rail the hard way, when it vetoes the next setup. ALLOWANCE_NEARING_USED_PCT = Decimal("80") -#: The doctor findings rail-17's event reads. Rail 14's `attest.subscription` is deliberately -#: absent: a lapsed (or never-attested) subscription surfaces through the ALLOWANCE event -#: instead -- the unsubscribed allowance (0 by default) with month-to-date spend IS that +#: The doctor findings the attestation event reads. Rail 14's `attest.subscription` is +#: deliberately absent: a lapsed (or never-attested) subscription surfaces through the ALLOWANCE +#: event instead -- the unsubscribed allowance (0 by default) with month-to-date spend IS that #: event's zero-runway case, so a rail-14 finding here would double-notify the same fact. -#: `attest.cash_posture` (#691) joins rail 17 here because it has the same failure shape and -#: a longer fuse: nothing re-confirms it, so it lapses on a clock, and the live profile runs -#: unattended. A doctor finding nobody is told about is only marginally better than the veto. -_ATTESTATION_FINDINGS = frozenset({"attest.withdrawals", "attest.cash_posture"}) +#: **Rail 17's `attest.withdrawals` is absent too (#1010).** It reads the venue per buy and has no +#: expiry to warn about; what the finding still says is the operator's OWN manual suspension (or +#: an unreadable store), which the person who set it knows -- the reasoning that keeps the kill +#: switch out of `rail.armed`. `attest.cash_posture` (#691) stays: nothing re-confirms it, so it +#: lapses on a clock, and the live profile runs unattended. +_ATTESTATION_FINDINGS = frozenset({"attest.cash_posture"}) #: Which rail each attestation finding belongs to, for the event message. -_RAIL_LABEL = {"attest.withdrawals": "rail 17", "attest.cash_posture": "rail 22"} +_RAIL_LABEL = {"attest.cash_posture": "rail 22"} #: Each attestation finding's key in `keel.attestations`, which is where its expiry lives. #: Two tables rather than one because they answer to different modules and a finding can be in #: `_ATTESTATION_FINDINGS` without the model knowing it -- `test_every_finding_that_can_fire_the #: _event_has_a_window` is the pin that says they must agree. _ATTESTATION_MODEL_KEY = { - "attest.withdrawals": "withdrawal", "attest.cash_posture": "cash_posture", } @@ -403,14 +404,9 @@ def notify_after_cycle( venue = current_venue() or guards.DEFAULT_VENUE subscription = repo.get_broker_subscription(venue) attestation = [ - *doctor.attestation_findings( - subscription=subscription, - withdrawals_attested_at=int( - repo.get_state("withdrawals_attested_at", default=0) or 0 - ), - withdrawals_enabled=repo.get_state("withdrawals_enabled", default=None), - now_ts=now_ts, - ), + # No `doctor.attestation_findings` here since #1010: its only deliverable finding + # was rail 17's, which no longer has an expiry (see `_ATTESTATION_FINDINGS`), and + # rail 14's is the allowance event's. # #732. `attest.cash_posture` was in `_ATTESTATION_FINDINGS` and this call was not # here, so the registration was real and the delivery path was not: an operator who # wired a webhook for it would never have been told. diff --git a/keel/web/payload.py b/keel/web/payload.py index a53fbc32..46b21323 100644 --- a/keel/web/payload.py +++ b/keel/web/payload.py @@ -662,16 +662,6 @@ def _rail_state(status: str | None) -> str: return GOOD -#: Rail 17's four states, judged. `expired` and `suspended` are the two that halt live entries; -#: `unattested` has never been answered at all, which is a gap rather than a breach. -_ATTESTATION_STATES: Mapping[str, str] = { - "attested": GOOD, - "suspended": BAD, - "expired": BAD, - "unattested": WARN, - "unknown": UNKNOWN, -} - #: The venue clock, judged. `clock_unavailable` is a WARNING and not an error: the agent #: fails closed on it (cycles skip), so the deployment is safe but not trading, and an operator #: needs to know which of those two it is looking at. @@ -723,16 +713,18 @@ def _autonomy_payload(autonomy: AutonomyStatus) -> dict[str, Any]: def _attestation_payload(attestation: WithdrawalAttestationStatus) -> dict[str, Any]: - return { - "state": label( - attestation.state, - state=_ATTESTATION_STATES.get(attestation.state or "", UNKNOWN), - ), - "enabled": flag(attestation.enabled, on="enabled", off="disabled"), - "attested_at": moment(attestation.attested_at), - "expires_in": duration(attestation.expires_in_sec, elapsed=False), - "expired_for": duration(attestation.expired_for_sec), - } + """Rail 17's manual suspension, tri-state and judged (#1010). + + Rail 17 decides from the venue and has no expiry, so the card carries the one thing an operator + sets. `unknown` is its own word, never `off`: the rail fails closed on an unreadable store. + """ + if attestation.suspended is None: + suspended = label("unknown", display="unknown (store unreadable)", state=WARN) + elif attestation.suspended: + suspended = label("on", display="ON — entries halted", state=BAD) + else: + suspended = label("off", display="off", state=NEUTRAL) + return {"suspended": suspended} def attestation_alerts(surveyed: Sequence[Any], now_ts: int) -> list[dict[str, Any]]: diff --git a/keel/web/static/js/docs.js b/keel/web/static/js/docs.js index 5a4c71ae..9587f2f9 100644 --- a/keel/web/static/js/docs.js +++ b/keel/web/static/js/docs.js @@ -108,7 +108,7 @@ export const TERMS = { "autonomy lapses": { slug: "glossary", anchor: "autonomy" }, "kill switch": { slug: "glossary", anchor: "kill-switch" }, "rail 11": { slug: "glossary", anchor: "rail" }, - "withdrawal attestation (rail 17)": { slug: "glossary", anchor: "attestation" }, + "manual withdrawal suspension (rail 17)": { slug: "glossary", anchor: "attestation" }, "market session": { slug: "glossary", anchor: "market-clock" }, "paper stage": { slug: "glossary", anchor: "paper-mode" }, }; diff --git a/keel/web/static/js/render.js b/keel/web/static/js/render.js index dece7124..1dbce885 100644 --- a/keel/web/static/js/render.js +++ b/keel/web/static/js/render.js @@ -774,11 +774,7 @@ function pulseState(data) { fragment.append( gridCard([ - kv("withdrawal attestation (rail 17)", data.withdrawal_attestation.state), - kv("withdrawals", data.withdrawal_attestation.enabled), - kv("attested at", data.withdrawal_attestation.attested_at), - kv("expires in", data.withdrawal_attestation.expires_in), - kv("expired for", data.withdrawal_attestation.expired_for), + kv("manual withdrawal suspension (rail 17)", data.withdrawal_attestation.suspended), ]), ); diff --git a/tests/commands/test_console_thinness.py b/tests/commands/test_console_thinness.py index 8debc40f..0c05cae5 100644 --- a/tests/commands/test_console_thinness.py +++ b/tests/commands/test_console_thinness.py @@ -21,7 +21,7 @@ report), `keel.analysis.*` (FIFO/P&L math) -- EXCEPT (a) CONSTANTS and TYPES, read from the source module's own AST (a class, or an ALL_CAPS module-level assignment): the console modules legitimately import gate WORDING constants and screen vocabulary - (e.g. `KNOWN_BACKINGS`, `WITHDRAWAL_ATTESTATION_TTL_SEC`) and constructing a frozen + (e.g. `KNOWN_BACKINGS`, `WITHDRAWALS_ATTEST_ACTION`) and constructing a frozen parameter dataclass is keyword passthrough, not math; (b) submodules (their CALLS are Rule 2's business); (c) the audited `IMPORT_ALLOWLIST` below, whose every entry also carries a Rule-2 call allowance. diff --git a/tests/commands/test_doctor.py b/tests/commands/test_doctor.py index 2406ed79..5d08b0a8 100644 --- a/tests/commands/test_doctor.py +++ b/tests/commands/test_doctor.py @@ -70,44 +70,45 @@ def status(self) -> str: return self._status -def test_rail17_fresh_reports_days_remaining() -> None: - findings = attestation_findings( - subscription=None, - withdrawals_attested_at=NOW - 4 * DAY, - withdrawals_enabled=True, - now_ts=NOW, - ) +def _rail17(suspended: bool | None) -> Any: + findings = attestation_findings(subscription=None, now_ts=NOW, withdrawals_suspended=suspended) (rail17,) = [f for f in findings if f.name == "attest.withdrawals"] + return rail17 + + +def test_doctor_rail17_passes_without_any_attestation() -> None: + """#1010: rail 17 reads the venue per buy; nothing was ever attested and that is fine.""" + rail17 = _rail17(False) assert rail17.status == "ok" - assert "3 day" in rail17.detail - assert rail17.fix + assert "venue" in rail17.detail + assert "attest" not in rail17.headline.lower() + assert "expire" not in (rail17.headline + rail17.detail).lower() -def test_rail17_absent_fails_and_names_the_attest_command() -> None: - findings = attestation_findings( - subscription=None, withdrawals_attested_at=0, withdrawals_enabled=True, now_ts=NOW - ) - (rail17,) = [f for f in findings if f.name == "attest.withdrawals"] +def test_doctor_rail17_fails_on_manual_suspension() -> None: + rail17 = _rail17(True) assert rail17.status == "fail" + assert "suspen" in rail17.headline.lower() assert "keel withdrawals attest --enabled" in rail17.fix -def test_rail17_expired_fails_with_days_over() -> None: - findings = attestation_findings( - subscription=None, - withdrawals_attested_at=NOW - 9 * DAY, - withdrawals_enabled=True, - now_ts=NOW, - ) - (rail17,) = [f for f in findings if f.name == "attest.withdrawals"] - assert rail17.status == "fail" - assert "expired" in rail17.detail.lower() +def test_doctor_rail17_warns_when_the_store_is_unreadable() -> None: + """`None` is not `False`: the rail fails closed on it, so doctor cannot call it a pass.""" + rail17 = _rail17(None) + assert rail17.status == "warn" + assert "unreadable" in (rail17.headline + rail17.detail).lower() + + +def test_doctor_never_reads_the_old_attestation_timestamp() -> None: + import inspect + + from keel.commands import doctor + + assert '"withdrawals_attested_at"' not in inspect.getsource(doctor) def test_rail14_absent_fails_naming_the_venue() -> None: - findings = attestation_findings( - subscription=None, withdrawals_attested_at=0, withdrawals_enabled=True, now_ts=NOW - ) + findings = attestation_findings(subscription=None, now_ts=NOW, withdrawals_suspended=False) (rail14,) = [f for f in findings if f.name == "attest.subscription"] assert rail14.status == "fail" assert "keel subscription attest" in rail14.fix @@ -115,9 +116,7 @@ def test_rail14_absent_fails_naming_the_venue() -> None: def test_rail14_attestation_due_soon_warns_with_days() -> None: record = FakeSubscription(Decimal("500"), NOW + 2 * DAY, "attested") - findings = attestation_findings( - subscription=record, withdrawals_attested_at=NOW, withdrawals_enabled=True, now_ts=NOW - ) + findings = attestation_findings(subscription=record, now_ts=NOW, withdrawals_suspended=False) (rail14,) = [f for f in findings if f.name == "attest.subscription"] assert rail14.status == "warn" assert "2 day" in rail14.detail @@ -321,9 +320,11 @@ def test_exit_code_fails_only_on_real_faults() -> None: def test_every_finding_names_a_fix_or_says_none_needed() -> None: - samples = attestation_findings( - subscription=None, withdrawals_attested_at=NOW, withdrawals_enabled=True, now_ts=NOW - ) + rail_state_findings( + samples = [ + *attestation_findings(subscription=None, now_ts=NOW, withdrawals_suspended=False), + *attestation_findings(subscription=None, now_ts=NOW, withdrawals_suspended=True), + *attestation_findings(subscription=None, now_ts=NOW, withdrawals_suspended=None), + ] + rail_state_findings( kill_switch=False, streak_halt_until=0, drawdown_total=Decimal("0"), now_ts=NOW ) for finding in samples: @@ -1692,15 +1693,14 @@ def test_an_unmigrated_posture_table_is_not_reported_as_an_unattested_posture( assert "never attested" not in posture.headline -def test_gather_findings_hands_rail_17_both_of_its_keys(tmp_path, valid_config_path) -> None: - """`attestation_findings` grew a `withdrawals_enabled` parameter and `gather_findings` grew - the read that feeds it. Delete that one line and every other test stays green while - `keel doctor` reports `ok` for an account whose broker has frozen withdrawals -- the exact - defect the parameter was added to fix. +def test_gather_findings_hands_rail_17_its_stored_suspension(tmp_path, valid_config_path) -> None: + """`gather_findings` reads the stored suspension and hands it to `attestation_findings`. + Delete that read and every other test stays green while `keel doctor` reports `ok` for an + account the operator has manually suspended. Only the one key is seeded: the timestamp the + attestation used to carry is gone (#1010). """ repo = _seeded_repo(tmp_path / "keel.db") now = NOW - repo.set_state("withdrawals_attested_at", str(now - DAY)) repo.set_state("withdrawals_enabled", False) findings = gather_findings(repo, load_config(valid_config_path), [], now) @@ -1709,6 +1709,30 @@ def test_gather_findings_hands_rail_17_both_of_its_keys(tmp_path, valid_config_p assert "suspend" in rail17.headline.lower() +def test_gather_findings_rail_17_passes_on_a_database_that_never_attested( + tmp_path, valid_config_path +) -> None: + repo = _seeded_repo(tmp_path / "keel.db") + + findings = gather_findings(repo, load_config(valid_config_path), [], NOW) + (rail17,) = [f for f in findings if f.name == "attest.withdrawals"] + assert rail17.status == "ok" + + +def test_gather_findings_rail_17_warns_on_an_unreadable_store( + tmp_path, valid_config_path, monkeypatch +) -> None: + """The display reads the tri-state, so an unreadable store is `warn`, never a green `ok`.""" + from keel.execution import executor + + repo = _seeded_repo(tmp_path / "keel.db") + monkeypatch.setattr(executor, "_stored_suspension", lambda _repo: None) + + findings = gather_findings(repo, load_config(valid_config_path), [], NOW) + (rail17,) = [f for f in findings if f.name == "attest.withdrawals"] + assert rail17.status == "warn" + + # -- ledger venue drift: the ledger against the venue's recorded holding (#798, plan R3) ---------- diff --git a/tests/commands/test_insights.py b/tests/commands/test_insights.py index 4d168ada..3bb7115b 100644 --- a/tests/commands/test_insights.py +++ b/tests/commands/test_insights.py @@ -359,13 +359,7 @@ def test_account_summary_projects_verbatim_from_status_report() -> None: max_total_dd_pct=Decimal("0.20"), max_weekly_dd_pct=Decimal("0.08"), rail11_status="ok", - withdrawal_attestation=WithdrawalAttestationStatus( - state="attested", - enabled=True, - attested_at=NOW_TS - 86400, - expires_in_sec=6 * 86400, - expired_for_sec=None, - ), + withdrawal_attestation=WithdrawalAttestationStatus(suspended=False), paper_cash_usdc=Decimal("955.25"), open_positions=[], rule_counts={}, diff --git a/tests/commands/test_pulse.py b/tests/commands/test_pulse.py index a5b65ebc..28fbe93e 100644 --- a/tests/commands/test_pulse.py +++ b/tests/commands/test_pulse.py @@ -179,31 +179,28 @@ def test_why_not_takes_the_newest_cycle_first_and_keeps_only_the_named_prefixes( def test_why_not_omits_rail_17_in_paper_and_names_it_in_live() -> None: - lapsed = WithdrawalAttestationStatus("expired", True, NOW - 9 * _DAY, None, 2 * _DAY) - assert _pulse(mode="paper", withdrawal_attestation=lapsed).why_not == () - assert _pulse(mode="live", withdrawal_attestation=lapsed).why_not == ( - "rail 17: withdrawal attestation expired", + suspended = WithdrawalAttestationStatus(True) + assert _pulse(mode="paper", withdrawal_attestation=suspended).why_not == () + assert _pulse(mode="live", withdrawal_attestation=suspended).why_not == ( + "rail 17: manual withdrawal suspension is on", ) - unattested = WithdrawalAttestationStatus("unattested", None, None, None, None) - assert _pulse(mode="live", withdrawal_attestation=unattested).why_not == ( - "rail 17: withdrawal attestation unattested", + assert ( + _pulse(mode="live", withdrawal_attestation=WithdrawalAttestationStatus(False)).why_not == () ) - attested = WithdrawalAttestationStatus("attested", True, NOW - 3600, 6 * _DAY, None) - assert _pulse(mode="live", withdrawal_attestation=attested).why_not == () def test_why_not_names_rail_17_wherever_status_says_entries_are_halted() -> None: - """`status._rail17_line` evaluates rail 17 in every non-paper mode and says "entries halted" - for suspended, expired and never-attested alike; the why-not list must not be narrower.""" - lapsed = WithdrawalAttestationStatus("expired", True, NOW - 9 * _DAY, None, 2 * _DAY) - assert _pulse(mode="confirm", withdrawal_attestation=lapsed).why_not == ( - "rail 17: withdrawal attestation expired", + """`status._rail17_line` evaluates rail 17 in every non-paper mode; the why-not list must not + be narrower. An unreadable store fails the rail closed, so it is a blocker too.""" + assert _pulse( + mode="confirm", withdrawal_attestation=WithdrawalAttestationStatus(True) + ).why_not == ("rail 17: manual withdrawal suspension is on",) + assert _pulse( + mode="live", withdrawal_attestation=WithdrawalAttestationStatus(None) + ).why_not == ("rail 17: manual suspension unknown (store unreadable); buys fail closed",) + assert ( + _pulse(mode="paper", withdrawal_attestation=WithdrawalAttestationStatus(None)).why_not == () ) - suspended = WithdrawalAttestationStatus("suspended", False, NOW - 3600, None, None) - assert _pulse(mode="live", withdrawal_attestation=suspended).why_not == ( - "rail 17: withdrawal attestation suspended", - ) - assert _pulse(mode="paper", withdrawal_attestation=suspended).why_not == () # -- next DCA buy ------------------------------------------------------------------------------- diff --git a/tests/commands/test_setup.py b/tests/commands/test_setup.py index 2422dc64..e2543b44 100644 --- a/tests/commands/test_setup.py +++ b/tests/commands/test_setup.py @@ -75,20 +75,38 @@ def test_every_table_this_module_reads_exists(fresh: tuple[Path, Path]) -> None: def test_the_withdrawals_state_key_is_the_one_the_command_writes(fresh: tuple[Path, Path]) -> None: - """Same failure mode one level down: a right table with a wrong key reads as unattested.""" + """Same failure mode one level down: a right table with a wrong key reads as no suspension + forever. Rail 17 needs no attestation (#1010), so the step is done on a fresh database and + NOT done once the manual suspension -- the only thing the command still writes -- is set.""" config_path, db_path = fresh + + state = inspect(config_path, db_path) + step = next(s for s in state.states if s.step.key == "withdrawals_attested") + assert step.done is True, step.detail + assert "venue" in step.detail + conn = connect(str(db_path)) try: from keel.data.repository import Repository - Repository(conn).set_state(WITHDRAWALS_STATE_KEY, True) + Repository(conn).set_state(WITHDRAWALS_STATE_KEY, False) conn.commit() finally: conn.close() state = inspect(config_path, db_path) step = next(s for s in state.states if s.step.key == "withdrawals_attested") - assert step.done is True, step.detail + assert step.done is False + assert "suspended" in step.detail + + +def test_the_withdrawals_step_explains_the_venue_read_and_the_manual_switch() -> None: + from keel.commands.setup import STEPS + + step = next(s for s in STEPS if s.key == "withdrawals_attested") + text = f"{step.title} {step.why} {step.how}".lower() + assert "venue" in text and "--suspended" in text + assert "7-day" not in text and "ttl" not in text and "week" not in text and "expire" not in text # -- what a brand-new machine looks like ------------------------------------------------------ diff --git a/tests/commands/test_status.py b/tests/commands/test_status.py index 86242f2e..7eb11c24 100644 --- a/tests/commands/test_status.py +++ b/tests/commands/test_status.py @@ -35,7 +35,6 @@ ) from keel.data.db import connect, migrate from keel.data.repository import Repository -from keel.execution.executor import WITHDRAWAL_ATTESTATION_TTL_SEC from keel.types import Candle, Granularity NOW_TS = 1_800_000_000 @@ -522,108 +521,68 @@ def test_market_session_state_is_json_serialisable_for_the_tui(repo: Repository) # -- rail 17 (withdrawal attestation, §65.4) ------------------------------------------------- -def _attest(repo: Repository, enabled: bool, age_sec: int) -> None: - repo.set_state("withdrawals_enabled", enabled) - repo.set_state("withdrawals_attested_at", NOW_TS - age_sec) +def _suspend(repo: Repository, suspended: bool) -> None: + repo.set_state("withdrawals_enabled", not suspended) -def test_rail17_attested_fresh_shows_time_remaining(repo: Repository) -> None: - """A fresh attestation is visible BEFORE it vetoes: 4 days of the 7 spent -> 3 days left.""" - _attest(repo, enabled=True, age_sec=4 * 86400) - - report = gather_status(repo, _config(), now_ts=NOW_TS) - - w = report.withdrawal_attestation - assert w.state == "attested" - assert w.expires_in_sec == 3 * 86400 - assert "rail 17 (withdrawal capability): attested, expires in 3d" in render_human(report) - - -def test_rail17_expired_names_the_halt_and_the_fix(repo: Repository) -> None: - """19 days since attestation against the 7-day TTL -> expired 12 days ago, with the command - that releases it -- the 2026-08-14 event, visible in status instead of in a veto log. - - The halt wording is the LIVE-mode rendering: rail 17 is a LIVE_STATE rail, so the line is - asserted under a non-paper mode, and the paper variant (which cannot claim a halt the rail - will not run) is pinned by its own test below.""" - _attest(repo, enabled=True, age_sec=19 * 86400) - live = _config(auto_trade=AutoTradeConfig(mode="confirm")) - - report = gather_status(repo, live, now_ts=NOW_TS) +def test_rail17_default_is_off_and_says_the_venue_decides(repo: Repository) -> None: + """#1010: no attestation exists to be missing. A fresh database reads `manual suspension: + off`, and status makes no venue call (it takes no broker at all).""" + report = gather_status(repo, _config(auto_trade=AutoTradeConfig(mode="confirm")), now_ts=NOW_TS) - w = report.withdrawal_attestation - assert w.state == "expired" - assert w.expired_for_sec == 12 * 86400 - assert ( - "rail 17 (withdrawal capability): EXPIRED 12d ago -- entries halted; " - "re-attest with keel withdrawals attest" in render_human(report) + assert report.withdrawal_attestation.suspended is False + assert "rail 17 (withdrawal capability): manual suspension: off" in "\n".join( + render_human(report) ) -def test_rail17_expired_in_paper_names_the_state_not_a_halt(repo: Repository) -> None: - """Paper mode never evaluates rail 17 (`LIVE_STATE_RAILS`), so claiming "entries halted" - there would be a permanently-red alert for a halt that cannot occur -- alert fatigue, - the exact failure #340 exists to fix. The paper rendering names the state and says the - rail is not evaluated, keeping only the re-attest nudge.""" - _attest(repo, enabled=True, age_sec=19 * 86400) - - report = gather_status(repo, _config(), now_ts=NOW_TS) # _config() defaults to paper +def test_rail17_manual_suspension_names_the_halt_and_the_release(repo: Repository) -> None: + _suspend(repo, True) + report = gather_status(repo, _config(auto_trade=AutoTradeConfig(mode="confirm")), now_ts=NOW_TS) + assert report.withdrawal_attestation.suspended is True + text = "\n".join(render_human(report)) assert ( - "rail 17 (withdrawal capability): EXPIRED 12d ago " - "(rail 17 not evaluated in paper); " - "re-attest with keel withdrawals attest" in render_human(report) + "rail 17 (withdrawal capability): manual suspension: ON -- entries halted; " + "clear with keel withdrawals attest --enabled" in text ) -def test_rail17_never_attested_is_said_as_such(repo: Repository) -> None: - """A fresh DB has no attestation at all -- rail 17 fails closed on that too, so status - must not render the silence as health.""" - report = gather_status(repo, _config(auto_trade=AutoTradeConfig(mode="confirm")), now_ts=NOW_TS) +def test_rail17_suspension_in_paper_names_the_state_not_a_halt(repo: Repository) -> None: + """Paper mode never evaluates rail 17 (`LIVE_STATE_RAILS`), so it must not claim a halt.""" + _suspend(repo, True) + text = "\n".join(render_human(gather_status(repo, _config(), now_ts=NOW_TS))) - w = report.withdrawal_attestation - assert w.state == "unattested" - assert w.attested_at is None - assert ( - "rail 17 (withdrawal capability): never attested -- entries halted; " - "re-attest with keel withdrawals attest" in render_human(report) - ) + assert "manual suspension: ON (rail 17 not evaluated in paper)" in text + assert "entries halted" not in text -def test_rail17_suspended_attestation_still_names_the_halt(repo: Repository) -> None: - """A FRESH `--suspended` attestation is a deliberate rail-17 halt, not staleness -- the - line must say which, because the release is the same command with `--enabled`.""" - _attest(repo, enabled=False, age_sec=3600) +def test_rail17_unreadable_store_is_unknown_never_off( + repo: Repository, monkeypatch: pytest.MonkeyPatch +) -> None: + from keel.execution import executor + monkeypatch.setattr(executor, "_stored_suspension", lambda _repo: None) report = gather_status(repo, _config(auto_trade=AutoTradeConfig(mode="confirm")), now_ts=NOW_TS) - w = report.withdrawal_attestation - assert w.state == "suspended" - assert ( - "rail 17 (withdrawal capability): SUSPENDED -- entries halted; " - "re-attest with keel withdrawals attest --enabled" in render_human(report) - ) + assert report.withdrawal_attestation.suspended is None + text = "\n".join(render_human(report)) + assert "manual suspension: unknown (store unreadable)" in text + assert "manual suspension: off" not in text -def test_rail17_freshness_uses_the_executors_own_ttl(repo: Repository) -> None: - """Pin the boundary to `WITHDRAWAL_ATTESTATION_TTL_SEC` itself, at ±1s: status must age the - attestation with the SAME constant the executor vetoes on, never a restated 7 days -- a - drift between the two would report "attested" on the very cycle rail 17 vetoes.""" - _attest(repo, enabled=True, age_sec=WITHDRAWAL_ATTESTATION_TTL_SEC - 1) - fresh = gather_status(repo, _config(), now_ts=NOW_TS).withdrawal_attestation - assert fresh.state == "attested" - assert fresh.expires_in_sec == 1 +def test_rail17_status_carries_no_expiry_fields(repo: Repository) -> None: + """No days-to-expiry, no attested-at: the shape itself cannot render an expiry.""" + report = gather_status(repo, _config(), now_ts=NOW_TS) - _attest(repo, enabled=True, age_sec=WITHDRAWAL_ATTESTATION_TTL_SEC + 1) - stale = gather_status(repo, _config(), now_ts=NOW_TS).withdrawal_attestation - assert stale.state == "expired" - assert stale.expired_for_sec == 1 + assert set(asdict(report.withdrawal_attestation)) == {"suspended"} + (line,) = [ln for ln in render_human(report) if ln.startswith("rail 17")] + assert "expire" not in line.lower() and "attested" not in line.lower() def test_rail17_line_sits_beside_rail11(repo: Repository) -> None: """The two halt-rails render adjacently, so one glance answers "can the agent enter today?" for both the drawdown breaker and the possession rail.""" - _attest(repo, enabled=True, age_sec=3600) lines = render_human(gather_status(repo, _config(), now_ts=NOW_TS)) rail11_at = next(i for i, line in enumerate(lines) if line.startswith("rail11")) @@ -652,9 +611,8 @@ def test_status_command_runs_read_only_and_prints_key_facts(tmp_path, valid_conf assert "mode: paper" in result.output assert "kill_switch: clear" in result.output.lower() or "clear" in result.output.lower() assert "no open positions" in result.output.lower() - # A fresh DB has never been attested -- rail 17 is halting entries, and status says so - # rather than rendering the silence as health. - assert "rail 17 (withdrawal capability): never attested" in result.output + # Rail 17 reads the venue per buy (#1010); a fresh DB has no manual suspension. + assert "rail 17 (withdrawal capability): manual suspension: off" in result.output def test_status_command_json_flag_emits_parseable_json(tmp_path, valid_config_path) -> None: @@ -672,7 +630,7 @@ def test_status_command_json_flag_emits_parseable_json(tmp_path, valid_config_pa assert payload["mode"] == "paper" assert payload["kill_switch_engaged"] is False assert payload["rail11_status"] in {"ok", "HALTED", "unknown"} - assert payload["withdrawal_attestation"]["state"] == "unattested" + assert payload["withdrawal_attestation"] == {"suspended": False} assert "open_positions" in payload assert "rule_counts" in payload assert "data_freshness" in payload diff --git a/tests/execution/test_executor.py b/tests/execution/test_executor.py index 6d04c913..93dea126 100644 --- a/tests/execution/test_executor.py +++ b/tests/execution/test_executor.py @@ -232,11 +232,9 @@ def repo() -> Repository: r = Repository(conn) r.set_state("kill_switch", False) r.set_state("last_feed_ts", NOW_TS) - # Rail 17 (§65.4) fails closed without a fresh withdrawal attestation, so every test that - # is not ABOUT rail 17 supplies one -- same reason this fixture seeds the kill-switch and - # feed timestamp for rails 12. + # Rail 17 (§65.4) reads the venue per buy (#1010); the manual brake is stated as off, same + # reason this fixture seeds the kill-switch and feed timestamp for rails 12. r.set_state("withdrawals_enabled", True) - r.set_state("withdrawals_attested_at", NOW_TS) # A very large, attested monthly allowance so pre-existing (non-rail-14) tests aren't # incidentally tripped by it; rail-14-specific tests below override with `_attest(...)`. _attest(r, free_volume_usd=Decimal("10000000")) diff --git a/tests/execution/test_reduce.py b/tests/execution/test_reduce.py index 1c46ca5d..47a4f287 100644 --- a/tests/execution/test_reduce.py +++ b/tests/execution/test_reduce.py @@ -46,7 +46,6 @@ def repo() -> Repository: r.set_state("kill_switch", False) r.set_state("last_feed_ts", NOW_TS) r.set_state("withdrawals_enabled", True) - r.set_state("withdrawals_attested_at", NOW_TS) _attest(r, free_volume_usd=D("10000000")) attest_trade_scope(r, now_ts=NOW_TS) attest_cash_posture(r, now_ts=NOW_TS) diff --git a/tests/execution/test_withdrawal_capability.py b/tests/execution/test_withdrawal_capability.py index d60dd6e5..bc89f652 100644 --- a/tests/execution/test_withdrawal_capability.py +++ b/tests/execution/test_withdrawal_capability.py @@ -157,8 +157,7 @@ def test_cli_roundtrip_and_suspension_message(tmp_path, monkeypatch): # calls it there, so patch it at its definition (see that module's docstring). monkeypatch.setattr("keel.commands._common._is_interactive", lambda: True) - unknown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) - assert "UNKNOWN (never attested)" in unknown.output + assert runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]).exit_code == 0 assert ( runner.invoke( @@ -166,10 +165,8 @@ def test_cli_roundtrip_and_suspension_message(tmp_path, monkeypatch): ).exit_code == 0 ) - assert "ENABLED" in runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]).output suspended = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "attest", "--suspended"]) assert "ENTRIES are now halted" in suspended.output shown = runner.invoke(cli, ["--db", str(db_path), "withdrawals", "show"]) - assert "SUSPENDED" in shown.output - assert "exits are unaffected" in shown.output + assert "manual suspension: ON" in shown.output diff --git a/tests/test_agent.py b/tests/test_agent.py index 980321a9..4c0afe7a 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -201,12 +201,9 @@ def repo() -> Repository: migrate(conn) r = Repository(conn) r.set_state("kill_switch", False) - # Rail 17 (§65.4) fails closed without a fresh withdrawal attestation. Seeded here for the - # same reason the kill-switch is: these tests are not ABOUT rail 17, and a fail-closed rail - # would otherwise veto every BUY in the module. A huge attested_at keeps it fresh regardless - # of each test's `now_ts`. + # Rail 17 (§65.4) reads the venue per buy (#1010); the module's FakeBroker answers "allowed". + # The manual brake is stated as off for the same reason the kill-switch is. r.set_state("withdrawals_enabled", True) - r.set_state("withdrawals_attested_at", 10**12) # Autonomy is a PROFILE choice now. Most tests in this module were written against "it just # places"; opting the profile in here preserves that, and the tests that are specifically # about the confirm/autonomous decision set the profile themselves. @@ -3709,7 +3706,6 @@ def _live_ready_repo(repo): """Clear every live-BUY rail that isn't the confirmation itself.""" repo.set_state("kill_switch", False) repo.set_state("withdrawals_enabled", True) - repo.set_state("withdrawals_attested_at", 10**12) attest_subscription(repo, now_ts=0, free_volume_usd=Decimal("10000000")) return repo @@ -4829,12 +4825,11 @@ def test_retry_after_a_blocked_cycle_does_not_duplicate_an_already_placed_exit(r # done. These two tests pin the two contract halves -- disabled means ZERO transport calls, # and a warn-state cycle with the event opted in delivers exactly that event. -# A realistic epoch (2026-09-13 08:00 UTC), so the rail-17 fixture below is genuinely the -# WARN branch: attested 5 days before NOW, i.e. 2 of the 7 TTL days remain. A negative -# `withdrawals_attested_at` would take doctor's never-attested FAIL branch instead -- a -# different (sharper) finding than the comments and assertions here describe. +# A realistic epoch (2026-09-13 08:00 UTC), so the rail-22 fixture below is genuinely the WARN +# branch: a posture attestation with 10 days left, inside its 15-day warning window. (Rail 17 has +# no clock any more -- #1010 -- so the attestation event can only come from rail 22.) _NOTIFY_NOW = 1_800_000_000 -_NOTIFY_ATTESTED_AT = _NOTIFY_NOW - 5 * 86_400 +_NOTIFY_POSTURE_DUE = _NOTIFY_NOW + 10 * 86_400 # The latest CLOSED daily candle as of NOW (day 20_833 opens at 1_799_971_200), on a DCA # cadence boundary (20_832 % 7 == 0) and inside rail 12's staleness window (< 150_000s old). _NOTIFY_CANDLE_TS = 1_799_884_800 @@ -4860,7 +4855,7 @@ def _notify_config(events: frozenset[str]) -> Config: def _notify_re_attest_subscription(repo) -> None: """The fixture's subscription (attested at ts 0, due 31.5M) is long EXPIRED by the realistic epoch these tests run at -- rail 14 would veto the BUY and the test would stop - being about rail 17's notification. Re-attest a valid one for the same reason the fixture + being about the notification. Re-attest a valid one for the same reason the fixture attests at all: rail 14 is not what these tests are about.""" attest_subscription(repo, now_ts=_NOTIFY_NOW - 60 * 86_400, free_volume_usd=Decimal("10000000")) @@ -4872,12 +4867,11 @@ def test_a_cycle_with_notifications_disabled_makes_zero_transport_calls(repo, mo sink = _NotifySink() monkeypatch.setattr("keel_core.notifications.post_json", sink) monkeypatch.setenv("KEEL_ALERT_WEBHOOK", "https://alerts.example/hook") - # rail 17 at 5 of its 7 days (2 remain): the WARN state an opted-in deployment would be - # told about -- doctor's `attest.withdrawals` WARN, not the never-attested FAIL. - repo.set_state("withdrawals_attested_at", _NOTIFY_ATTESTED_AT) # The module fixture attests the cash posture at ts 0, which is already expired by - # `_NOTIFY_NOW`. Rail 22 is not what this test is about, so re-attest on its clock (#691). - attest_cash_posture(repo, now_ts=_NOTIFY_NOW) + # `_NOTIFY_NOW`. Re-attest on this clock with 10 days left: the WARN state an opted-in + # deployment would be told about -- doctor's `attest.cash_posture` WARN, still admitted by + # rail 22 (#691). + attest_cash_posture(repo, now_ts=_NOTIFY_NOW, attest_due_ts=_NOTIFY_POSTURE_DUE) _notify_re_attest_subscription(repo) repo.insert_rule("dca", {"product_id": PRODUCT}, status="live") broker = FakeBroker( @@ -4898,12 +4892,11 @@ def test_a_warn_state_cycle_emits_exactly_the_opted_in_events(repo, monkeypatch) sink = _NotifySink() monkeypatch.setattr("keel_core.notifications.post_json", sink) monkeypatch.setenv("KEEL_ALERT_WEBHOOK", "https://alerts.example/hook") - # attested 5 days before NOW: 2 of the 7 TTL days remain -> doctor WARNs (not the - # never-attested FAIL a negative epoch would take). - repo.set_state("withdrawals_attested_at", _NOTIFY_ATTESTED_AT) # The module fixture attests the cash posture at ts 0, which is already expired by - # `_NOTIFY_NOW`. Rail 22 is not what this test is about, so re-attest on its clock (#691). - attest_cash_posture(repo, now_ts=_NOTIFY_NOW) + # `_NOTIFY_NOW`. Re-attest on this clock with 10 days left: the WARN state an opted-in + # deployment would be told about -- doctor's `attest.cash_posture` WARN, still admitted by + # rail 22 (#691). + attest_cash_posture(repo, now_ts=_NOTIFY_NOW, attest_due_ts=_NOTIFY_POSTURE_DUE) _notify_re_attest_subscription(repo) repo.insert_rule("dca", {"product_id": PRODUCT}, status="live") broker = FakeBroker( diff --git a/tests/test_attestation_alert_once.py b/tests/test_attestation_alert_once.py index 27dfd871..6e529b8a 100644 --- a/tests/test_attestation_alert_once.py +++ b/tests/test_attestation_alert_once.py @@ -1,10 +1,11 @@ """One alert per attestation per window (#793). `attestation.expiring` has fired since #444 and it fires on EVERY cycle in which doctor's -finding is WARN or FAIL. On the live deployment on 2026-09-11 that is both rails at once, every -cycle, for as long as they stay lapsed -- rail 17 expired on 09-08 and rail 22 has never been -attested at all. An alert that repeats until it is fixed is an alert nobody reads by the third -day, which is the failure mode this issue is named after. +finding is WARN or FAIL. On the live deployment on 2026-09-11 that was two rails at once, every +cycle, for as long as they stayed lapsed. An alert that repeats until it is fixed is an alert +nobody reads by the third day, which is the failure mode this issue is named after. (Rail 17 has +since left this path -- #1010 -- so the cases below are rail 22's, the rail with a stored due +date and a never-attested state.) So the event is suppressed once it has been reported for the window it is in. A **window** is the attestation's state paired with its expiry, so: @@ -13,8 +14,8 @@ * actually expiring reports again -- it is a different state and a different sentence ("entries are halted" is not "entries will be"), and an operator who missed the warning gets the halt; * renewing and lapsing again reports again, because the new attestation has a new expiry; -* a never-attested rail reports once and then holds its tongue, which is the only shape that - works for rail 22 -- it has no expiry to cross and would otherwise alert forever. +* a never-attested rail reports once and then holds its tongue -- it has no expiry to cross and + would otherwise alert forever. The ledger is a repo key, and writing it is a departure from `notify_after_cycle`'s documented "never writes". `notifications.py` carries the reasoning; the tests here pin the behaviour. @@ -53,6 +54,24 @@ def _events(*names: str) -> list[Any]: ] +def _posture(repo: Repository, *, due_ts: int, attested_ts: int = NOW - DAY) -> None: + """Attest rail 22's cash posture with a STORED due date -- the only dated attestation left.""" + from keel_core.cash_posture import CashPostureState, VenueCashPosture + + repo.upsert_venue_cash_posture( + VenueCashPosture( + venue="coinbase", + state=CashPostureState.ATTESTED, + attested_posture="spot_cash", + attested_ts=attested_ts, + attest_due_ts=due_ts, + refuted_ts=None, + refuted_reason=None, + credential_fingerprint="fp-1", + ) + ) + + def _windows(repo: Repository, now_ts: int) -> dict[str, str]: return notifications.attestation_windows(attestations.survey(repo, now_ts), now_ts) @@ -64,27 +83,33 @@ def test_a_window_is_the_state_and_the_expiry_together(repo: Repository) -> None """Either half alone is wrong. The state alone never changes while a rail stays expired, so a renewal that lapses again would be silent; the expiry alone does not move when EXPIRING becomes EXPIRED, so the halt itself would be.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", "true") + _posture(repo, due_ts=NOW + 10 * DAY) - expiring = _windows(repo, NOW)["attest.withdrawals"] - expired = _windows(repo, NOW + 2 * DAY)["attest.withdrawals"] + expiring = _windows(repo, NOW)["attest.cash_posture"] + expired = _windows(repo, NOW + 11 * DAY)["attest.cash_posture"] assert expiring != expired - repo.set_state("withdrawals_attested_at", str(NOW + 2 * DAY)) - renewed = _windows(repo, NOW + 8 * DAY)["attest.withdrawals"] + _posture(repo, due_ts=NOW + 200 * DAY) + renewed = _windows(repo, NOW + 11 * DAY)["attest.cash_posture"] assert renewed not in (expiring, expired) def test_the_window_holds_still_across_the_cycles_inside_it(repo: Repository) -> None: - """The property the whole feature rests on: an hourly wrapper inside the final 48 hours must + """The property the whole feature rests on: an hourly wrapper inside the warning window must compute the SAME id every time, or suppression suppresses nothing.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", "true") - ids = {_windows(repo, NOW + n * HOUR)["attest.withdrawals"] for n in range(24)} + _posture(repo, due_ts=NOW + 10 * DAY) + ids = {_windows(repo, NOW + n * HOUR)["attest.cash_posture"] for n in range(24)} assert len(ids) == 1 +def test_rail_17_has_no_window_whatever_its_old_keys_say(repo: Repository) -> None: + """#1010: a database still carrying the TTL-era timestamp must not grow a rail-17 window.""" + repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) + repo.set_state("withdrawals_enabled", True) + + assert "attest.withdrawals" not in _windows(repo, NOW) + + def test_every_finding_that_can_fire_the_event_has_a_window(repo: Repository) -> None: """Rejects a third rail joining `_ATTESTATION_FINDINGS` without joining the model: its events would have no window, and the suppression below would have to choose between dropping them @@ -96,40 +121,41 @@ def test_every_finding_that_can_fire_the_event_has_a_window(repo: Repository) -> def test_the_first_report_of_a_window_goes_out() -> None: - windows = {"attest.withdrawals": "expiring:100"} - assert notifications.unreported(_events("attest.withdrawals"), windows=windows, already={}) + windows = {"attest.cash_posture": "expiring:100"} + assert notifications.unreported(_events("attest.cash_posture"), windows=windows, already={}) def test_the_second_report_of_the_same_window_does_not() -> None: - windows = {"attest.withdrawals": "expiring:100"} - already = {"attest.withdrawals": "expiring:100"} + windows = {"attest.cash_posture": "expiring:100"} + already = {"attest.cash_posture": "expiring:100"} assert ( - notifications.unreported(_events("attest.withdrawals"), windows=windows, already=already) + notifications.unreported(_events("attest.cash_posture"), windows=windows, already=already) == [] ) def test_a_new_window_reports_again_over_a_stale_ledger_entry() -> None: - windows = {"attest.withdrawals": "expired:100"} - already = {"attest.withdrawals": "expiring:100"} + windows = {"attest.cash_posture": "expired:100"} + already = {"attest.cash_posture": "expiring:100"} assert ( len( notifications.unreported( - _events("attest.withdrawals"), windows=windows, already=already + _events("attest.cash_posture"), windows=windows, already=already ) ) == 1 ) -def test_suppressing_one_rail_does_not_suppress_the_other() -> None: - """The #732 bug one layer down: rail 17 reported and rail 22 silenced by it.""" - windows = {"attest.withdrawals": "expired:100", "attest.cash_posture": "missing:None"} - already = {"attest.withdrawals": "expired:100"} +def test_suppressing_one_finding_does_not_suppress_another() -> None: + """The #732 bug one layer down: one rail reported and another silenced by it. Two findings, + though only one rail is registered today -- the property is about the keying.""" + windows = {"attest.cash_posture": "expired:100", "attest.next_rail": "missing:None"} + already = {"attest.cash_posture": "expired:100"} sent = notifications.unreported( - _events("attest.withdrawals", "attest.cash_posture"), windows=windows, already=already + _events("attest.cash_posture", "attest.next_rail"), windows=windows, already=already ) - assert [event.fields["finding"] for event in sent] == ["attest.cash_posture"] + assert [event.fields["finding"] for event in sent] == ["attest.next_rail"] @pytest.mark.parametrize("already", [{}, {"rail.streak_halt": "x"}]) @@ -162,11 +188,25 @@ def test_an_event_carrying_no_finding_at_all_is_never_suppressed() -> None: def test_a_delivered_report_is_recorded_and_an_undelivered_one_is_not(repo: Repository) -> None: """A webhook that was down must not cost the only alert for this window.""" - notifications.record_reported(repo, {"attest.withdrawals": "expiring:100"}) - assert json.loads(str(repo.get_state(LEDGER))) == {"attest.withdrawals": "expiring:100"} + notifications.record_reported(repo, {"attest.cash_posture": "expiring:100"}) + assert json.loads(str(repo.get_state(LEDGER))) == {"attest.cash_posture": "expiring:100"} notifications.record_reported(repo, {}) - assert json.loads(str(repo.get_state(LEDGER))) == {"attest.withdrawals": "expiring:100"} + assert json.loads(str(repo.get_state(LEDGER))) == {"attest.cash_posture": "expiring:100"} + + +def test_recording_a_window_merges_into_the_ledger_rather_than_replacing_it( + repo: Repository, +) -> None: + """Rejects a `record_reported` that REPLACES the ledger: the second rail's alert would evict + the first's entry, invisible until the next cycle reads a ledger that has forgotten it.""" + notifications.record_reported(repo, {"attest.cash_posture": "expiring:100"}) + notifications.record_reported(repo, {"attest.next_rail": "missing:None"}) + + assert json.loads(str(repo.get_state(LEDGER))) == { + "attest.cash_posture": "expiring:100", + "attest.next_rail": "missing:None", + } def test_an_unreadable_ledger_reports_rather_than_staying_silent(repo: Repository) -> None: @@ -187,14 +227,9 @@ def _config() -> Any: return _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) -def _rail17(calls: list[Any]) -> int: - """Deliveries mentioning rail 17. - - Counted per RAIL rather than per cycle because a fresh deployment has attested no cash - posture either, so rail 22 alerts alongside on the first cycle -- which is right, and is the - live deployment's own state on 2026-09-11, but it is not what these tests are about. - """ - return len([call for call in calls if "rail 17" in json.dumps(call)]) +def _rail22(calls: list[Any]) -> int: + """Deliveries mentioning rail 22 -- the one rail whose attestation still has a window.""" + return len([call for call in calls if "rail 22" in json.dumps(call)]) def _cycle(repo: Repository, calls: list[Any], now_ts: int) -> int: @@ -216,55 +251,68 @@ def _transport(url: str, body: bytes) -> None: def test_a_lapsed_rail_alerts_on_the_first_cycle_and_not_on_the_next(repo: Repository) -> None: """The whole point. The wrapper fires hourly; without this the operator gets the same two sentences 24 times a day for as long as the rail stays lapsed.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", True) + _posture(repo, due_ts=NOW + 10 * DAY) calls: list[Any] = [] for hour in range(3): _cycle(repo, calls, NOW + hour * HOUR) - assert _rail17(calls) == 1 + assert _rail22(calls) == 1 + + +def test_a_never_attested_rail_alerts_once_and_holds_its_tongue(repo: Repository) -> None: + calls: list[Any] = [] + + for hour in range(3): + _cycle(repo, calls, NOW + hour * HOUR) + + assert _rail22(calls) == 1 + + +def test_rail_17_never_alerts_from_a_database_that_still_carries_its_old_keys( + repo: Repository, +) -> None: + repo.set_state("withdrawals_attested_at", str(NOW - 30 * DAY)) + repo.set_state("withdrawals_enabled", True) + _posture(repo, due_ts=NOW + 200 * DAY) + calls: list[Any] = [] + + _cycle(repo, calls, NOW) + + assert calls == [] def test_the_halt_itself_alerts_even_though_the_warning_already_did(repo: Repository) -> None: """ "Entries are halted" is not "entries will be", and an operator who missed the warning has exactly one more chance to hear about it.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", True) + _posture(repo, due_ts=NOW + DAY) calls: list[Any] = [] _cycle(repo, calls, NOW) - assert _rail17(calls) == 1 + assert _rail22(calls) == 1 _cycle(repo, calls, NOW + 2 * DAY) - assert _rail17(calls) == 2 - # And rail 22, whose own window never moved, stays quiet through all of it -- INCLUDING the - # cycle after rail 17's second alert, which is the one that matters. Rejects a - # `record_reported` that REPLACES the ledger instead of merging into it: rail 17's second - # alert evicts rail 22's entry, and the eviction is invisible until the next cycle reads a - # ledger that has forgotten rail 22. Measured: without this third cycle the mutant survives. + assert _rail22(calls) == 2 _cycle(repo, calls, NOW + 2 * DAY + HOUR) - assert len([call for call in calls if "rail 22" in json.dumps(call)]) == 1 + assert _rail22(calls) == 2, "the halt is reported once, not every cycle" def test_renewing_and_lapsing_again_alerts_again(repo: Repository) -> None: """Rejects a ledger keyed on the finding name alone: one alert, ever, per deployment.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", True) + _posture(repo, due_ts=NOW + 10 * DAY) calls: list[Any] = [] _cycle(repo, calls, NOW) - assert _rail17(calls) == 1 + assert _rail22(calls) == 1 - repo.set_state("withdrawals_attested_at", str(NOW + DAY)) + _posture(repo, due_ts=NOW + 200 * DAY) _cycle(repo, calls, NOW + DAY) - assert _rail17(calls) == 1, "renewed: nothing to say" - _cycle(repo, calls, NOW + 6 * DAY) - assert _rail17(calls) == 2, "lapsing again is news again" + assert _rail22(calls) == 1, "renewed: nothing to say" + _cycle(repo, calls, NOW + 190 * DAY) + assert _rail22(calls) == 2, "lapsing again is news again" def test_an_undelivered_alert_is_retried_on_the_next_cycle(repo: Repository) -> None: """A webhook that was down must not cost the only alert for this window.""" - repo.set_state("withdrawals_attested_at", str(NOW - 6 * DAY)) - repo.set_state("withdrawals_enabled", True) + _posture(repo, due_ts=NOW + 10 * DAY) def _dead(url: str, body: bytes) -> None: raise OSError("connection refused") @@ -279,7 +327,7 @@ def _dead(url: str, body: bytes) -> None: ) calls: list[Any] = [] _cycle(repo, calls, NOW + HOUR) - assert _rail17(calls) == 1 + assert _rail22(calls) == 1 def test_the_alert_reads_the_venue_the_cycle_is_bound_to(repo: Repository) -> None: diff --git a/tests/test_attestation_expiry.py b/tests/test_attestation_expiry.py index 9c1afcf2..cd18fbe6 100644 --- a/tests/test_attestation_expiry.py +++ b/tests/test_attestation_expiry.py @@ -11,23 +11,19 @@ left doctor reporting the old one. A banner, a doctor state and a webhook built on that would have been a fifth copy across two incompatible shapes. -So every test below is about the model answering for BOTH shapes -- rail 17 derives freshness -from a TTL, rail 22 stores a due date -- and about no surface ever computing one itself. +Since #1010 rail 17 is no longer in this model: it reads the venue per buy and has nothing to +expire, so the model covers rail 22 alone and the tests below pin that rail 17 stays out. """ from __future__ import annotations -from dataclasses import replace -from pathlib import Path from typing import Any import pytest from keel import attestations -from keel.commands import doctor from keel.data.db import connect, migrate from keel.data.repository import Repository -from keel.execution import executor NOW = 1_789_000_000 @@ -45,60 +41,18 @@ def test_a_survey_covers_every_attestation_a_rail_can_veto_on() -> None: it here, and deleting a line that says it was covered.""" keys = {a.key for a in attestations.survey_definitions()} - assert keys == {"withdrawal", "cash_posture"} + assert keys == {"cash_posture"} -def test_rail_17_expiry_comes_from_the_executors_constant_and_is_not_restated() -> None: - """**The acceptance criterion for the whole issue.** - - `doctor.TTL_SEC` was a duplicate of this number under a comment claiming it was an import. - If the survey restates it, the same drift returns one module over -- so this MUTATES the - rail's constant and requires the computed expiry to move with it.""" - original = executor.WITHDRAWAL_ATTESTATION_TTL_SEC - try: - executor.WITHDRAWAL_ATTESTATION_TTL_SEC = 3 * 86_400 - shifted = attestations.withdrawal_expiry(attested_at=NOW) - assert shifted == NOW + 3 * 86_400, "the survey restates the TTL instead of reading it" - finally: - executor.WITHDRAWAL_ATTESTATION_TTL_SEC = original - - assert attestations.withdrawal_expiry(attested_at=NOW) == NOW + original - - -def test_an_unattested_rail_reports_missing_rather_than_expired(repo) -> None: - """ "Nobody has ever attested" and "the attestation went stale" are different facts, and - `_withdrawals_enabled` already keeps them apart in the veto message. A banner that called the - first one "expired" would send an operator looking for a lapse that never happened.""" - found = {a.key: a for a in attestations.survey(repo, now_ts=NOW)} - - assert found["withdrawal"].state(NOW) == attestations.MISSING - assert found["withdrawal"].attested_at is None - assert found["withdrawal"].expires_at is None - assert found["withdrawal"].seconds_left(NOW) is None - - -@pytest.mark.parametrize( - ("age_days", "expected"), - [ - (0, attestations.OK), - (4, attestations.OK), - (5.1, attestations.EXPIRING), # inside the final 48h of a 7-day TTL - (6.9, attestations.EXPIRING), - (7.1, attestations.EXPIRED), - (30, attestations.EXPIRED), - ], -) -def test_rail_17_passes_through_ok_then_expiring_then_expired(repo, age_days, expected) -> None: - """**The state that did not exist before: `EXPIRING`.** - - An attestation was `ok` until the instant it was refused, so the only warning an operator - ever got was the halt itself. The window is 48h, which for a 7-day TTL means the last two - days of every week.""" - repo.set_state("withdrawals_attested_at", int(NOW - age_days * 86_400)) +def test_survey_has_no_withdrawal_attestation(repo) -> None: + """#1010: rail 17 reads the venue per buy, so there is no attestation to expire -- not in + the definitions, and not in a survey of a database that still carries the old keys.""" + repo.set_state("withdrawals_attested_at", NOW - 30 * 86_400) repo.set_state("withdrawals_enabled", True) - found = {a.key: a for a in attestations.survey(repo, now_ts=NOW)} - assert found["withdrawal"].state(NOW) == expected + assert "withdrawal" not in {a.key for a in attestations.survey_definitions()} + assert "withdrawal" not in {a.key for a in attestations.survey(repo, NOW)} + assert not hasattr(attestations, "withdrawal_expiry") def test_rail_22_reports_from_its_STORED_due_date(repo) -> None: @@ -133,147 +87,27 @@ def test_every_attestation_names_the_command_that_refreshes_it() -> None: banner that said "expired" without saying what to do would be an alarm with no action.""" for definition in attestations.survey_definitions(): assert definition.remedy.startswith("keel "), definition.key - assert definition.rail in {"17", "22"}, definition.key + assert definition.rail == "22", definition.key assert definition.label, definition.key def test_each_attestation_warns_on_a_window_proportional_to_its_own_life() -> None: - """**Not one global window, which the first draft had.** - - 48 hours is right for rail 17's 7-day TTL and is no notice at all on rail 22's 90-day one. - `doctor` already knew that -- it warns rail 22 at `ATTESTATION_TTL_SEC // 6` and says the - window is "proportional to" a fixed TTL -- so the window belongs to the attestation, and - rail 22's is READ from the same constant rather than becoming a second opinion about it.""" + """Rail 22's window is READ from the rail's own 90-day constant rather than becoming a second + opinion about it: 15 days of a 90-day claim.""" from keel_core.cash_posture import ATTESTATION_TTL_SEC found = {a.key: a for a in attestations.survey_definitions()} - assert found["withdrawal"].warn_within_sec == 48 * 3_600 assert found["cash_posture"].warn_within_sec == ATTESTATION_TTL_SEC // 6 - assert found["cash_posture"].warn_within_sec > found["withdrawal"].warn_within_sec - - -def test_seconds_left_never_goes_negative(repo) -> None: - """An expired attestation reports zero, not a negative countdown a surface might render as - "expires in -3 days".""" - repo.set_state("withdrawals_attested_at", NOW - 30 * 86_400) - repo.set_state("withdrawals_enabled", True) - - found = {a.key: a for a in attestations.survey(repo, now_ts=NOW)} - assert found["withdrawal"].seconds_left(NOW) == 0 - - -# -- the same answer on every surface ------------------------------------------------------- - - -def test_doctor_reads_rail_17s_TTL_from_the_executor_rather_than_restating_it( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """The point of the whole module, applied to the surface that had the duplicate. - - `doctor.TTL_SEC` was `7 * 86_400` under a comment reading "doctor only READS it". Shorten - the executor's TTL to 3 days and an attestation made 4 days ago is expired; if doctor is - still holding its own copy of the number it reports 3 days remaining and says `ok`.""" - from keel.execution import executor - - source = Path(executor.__file__).read_text(encoding="utf-8") - monkeypatch.setattr(executor, "WITHDRAWAL_ATTESTATION_TTL_SEC", 3 * 86_400) - assert "WITHDRAWAL_ATTESTATION_TTL_SEC = 7 * 24 * 3600" in source, "the constant moved" - - (rail17,) = [ - f - for f in doctor.attestation_findings( - subscription=None, - withdrawals_attested_at=NOW - 4 * 86_400, - withdrawals_enabled=True, - now_ts=NOW, - ) - if f.name == "attest.withdrawals" - ] - assert rail17.status == doctor.FAIL - # The DAYS too, not just the verdict. `ttl_sec` feeds every number in doctor's sentence, and - # a local `7 * 86_400` there leaves the state right (it comes from the model) while the - # sentence reads "expired 0 day(s) ago" on a 3-day TTL. Measured: the mutant survived an - # assertion that only checked the status and the word "expired". - assert "expired 1 day(s) ago" in rail17.detail - - -@pytest.mark.parametrize( - ("hours_left", "expected"), - [(72, attestations.OK), (60, attestations.OK), (47, attestations.EXPIRING)], -) -def test_doctor_starts_warning_on_rail_17_exactly_when_the_banner_does( - hours_left: int, expected: str -) -> None: - """The divergence this feature exists to prevent, in the one place it already existed. - - Doctor warned at `_days(remaining) <= 2`, which truncation makes anything under 72 hours; - the banner's window is 48. Between those two an operator saw a doctor WARNING and a cockpit - that called the same attestation healthy. One window, read from the attestation.""" - attested_at = NOW + hours_left * 3_600 - attestations.withdrawal_expiry(attested_at=0) - (rail17,) = [ - f - for f in doctor.attestation_findings( - subscription=None, - withdrawals_attested_at=attested_at, - withdrawals_enabled=True, - now_ts=NOW, - ) - if f.name == "attest.withdrawals" - ] - model = attestations.Attestation( - key="withdrawal", - rail="17", - label="", - remedy="", - attested_at=attested_at, - expires_at=attestations.withdrawal_expiry(attested_at=attested_at), - ).state(NOW) - - assert model == expected - assert rail17.status == (doctor.WARN if expected == attestations.EXPIRING else doctor.OK) # -- the verdict, not just the clock ----------------------------------------------------------- -def test_a_suspended_attestation_is_not_a_healthy_one(monkeypatch: pytest.MonkeyPatch) -> None: - """Found while wiring doctor, and it is the same lie one column over. - - Rail 17 reads TWO keys -- `withdrawals_attested_at` and `withdrawals_enabled` -- and vetoes - on `enabled is False` with its own sentence ("withdrawals are suspended/restricted for this - account"). `attestation_findings` was handed the timestamp alone, so an operator who ran - `keel withdrawals attest --disabled` because their broker had frozen withdrawals got - `ok · withdrawal capability attested · 6 day(s) remain` from `keel doctor` while every BUY - was refused. `keel status` never had this bug: it resolves through the executor's own - `_withdrawals_enabled`.""" - conn = connect(":memory:") - migrate(conn) - repo = Repository(conn) - repo.set_state("withdrawals_attested_at", str(NOW - 86_400)) - repo.set_state("withdrawals_enabled", False) - - (withdrawal, _posture) = attestations.survey(repo, NOW) - assert withdrawal.state(NOW) == attestations.REFUSED - - (rail17,) = [ - f - for f in doctor.attestation_findings( - subscription=None, - withdrawals_attested_at=NOW - 86_400, - withdrawals_enabled=False, - now_ts=NOW, - ) - if f.name == "attest.withdrawals" - ] - assert rail17.status == doctor.FAIL - assert "suspend" in rail17.headline.lower() - - def test_the_verdict_does_not_outrank_the_clock() -> None: """An expired "no" reports as EXPIRED, not REFUSED: what an operator must do about it is re-attest either way, and `expired` is the word that says the reading is stale.""" - definition = attestations.definition("withdrawal") + definition = attestations.definition("cash_posture") stale = attestations.Attestation( key=definition.key, rail=definition.rail, @@ -337,7 +171,7 @@ def test_a_refuted_posture_is_not_a_healthy_one() -> None: from keel_core.cash_posture import CashPostureState repo = _PostureRepo(_posture(state=CashPostureState.REFUTED, refuted_ts=NOW - 3_600)) - (_withdrawal, posture) = attestations.survey(repo, NOW) + (posture,) = attestations.survey(repo, NOW) assert posture.state(NOW) == attestations.REFUSED @@ -350,7 +184,7 @@ def test_a_margin_enabled_posture_is_not_a_healthy_one() -> None: from keel_core.cash_posture import MARGIN_ENABLED repo = _PostureRepo(_posture(attested_posture=MARGIN_ENABLED)) - (_withdrawal, posture) = attestations.survey(repo, NOW) + (posture,) = attestations.survey(repo, NOW) assert posture.state(NOW) == attestations.REFUSED @@ -359,7 +193,7 @@ def test_a_posture_attested_with_no_due_date_is_not_reported_as_never_attested() expiry, which `VenueCashPosture` refuses to invent and rail 22 refuses to treat as "never expires". Doctor words it correctly at `doctor.py:516` and the model did not.""" repo = _PostureRepo(_posture(attest_due_ts=None)) - (_withdrawal, posture) = attestations.survey(repo, NOW) + (posture,) = attestations.survey(repo, NOW) assert posture.state(NOW) == attestations.REFUSED assert posture.attested_at == NOW - 86_400, "the date it WAS attested must survive" @@ -367,11 +201,8 @@ def test_a_posture_attested_with_no_due_date_is_not_reported_as_never_attested() def test_a_rail_nobody_has_attested_still_reports_missing() -> None: """The negative control for the three above: `MISSING` must not become unreachable.""" repo = _PostureRepo(None) - (withdrawal, posture) = attestations.survey(repo, NOW) - assert (withdrawal.state(NOW), posture.state(NOW)) == ( - attestations.MISSING, - attestations.MISSING, - ) + (posture,) = attestations.survey(repo, NOW) + assert posture.state(NOW) == attestations.MISSING def test_survey_covers_every_definition_rather_than_the_first_two( @@ -389,42 +220,8 @@ def test_survey_covers_every_definition_rather_than_the_first_two( monkeypatch.setattr(attestations, "_DEFINITIONS", (*attestations.survey_definitions(), third)) surveyed = attestations.survey(_PostureRepo(None), NOW) - assert [a.key for a in surveyed] == ["withdrawal", "cash_posture", "third_rail"] - assert surveyed[2].state(NOW) == attestations.MISSING, "a rail with no reader is not ok" - - -def test_reordering_the_definitions_does_not_cross_the_wires( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """The other half of the index coupling, and the one that is actively wrong rather than - merely absent: `replace(_DEFINITIONS[0], **_withdrawal_dates(repo))` on a reordered tuple - attaches rail 17's dates to rail 22's definition.""" - monkeypatch.setattr( - attestations, "_DEFINITIONS", tuple(reversed(attestations.survey_definitions())) - ) - repo = _PostureRepo(_posture()) - surveyed = {a.key: a for a in attestations.survey(repo, NOW)} - - assert surveyed["cash_posture"].attested_at == NOW - 86_400 - assert surveyed["withdrawal"].attested_at is None - - -def test_the_warning_window_never_outlives_the_attestation_it_guards( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """`withdrawal_expiry` reads the TTL at call time and the 48-hour window is flat, so a TTL - shortened below 48 hours makes every attestation `EXPIRING` the instant it is made -- a - banner lit permanently, which is the same thing as a banner nobody reads.""" - from keel.execution import executor - - monkeypatch.setattr(executor, "WITHDRAWAL_ATTESTATION_TTL_SEC", 6 * 3_600) - fresh = replace( - attestations.definition("withdrawal"), - attested_at=NOW, - expires_at=attestations.withdrawal_expiry(attested_at=NOW), - ) - assert fresh.state(NOW) == attestations.OK - assert fresh.state(NOW + 4 * 3_600) == attestations.EXPIRING + assert [a.key for a in surveyed] == ["cash_posture", "third_rail"] + assert surveyed[1].state(NOW) == attestations.MISSING, "a rail with no reader is not ok" def test_the_survey_never_raises_even_when_every_read_fails() -> None: @@ -440,28 +237,10 @@ def get_venue_cash_posture(self, *_args: Any, **_kwargs: Any) -> Any: raise OSError("database is locked") surveyed = attestations.survey(_Broken(), NOW) - assert [a.state(NOW) for a in surveyed] == [attestations.MISSING, attestations.MISSING] + assert [a.state(NOW) for a in surveyed] == [attestations.MISSING] def test_asking_for_an_attestation_that_does_not_exist_is_an_error_not_a_blank() -> None: """`definition` returning a placeholder would put a rail on the banner that no rail owns.""" with pytest.raises(KeyError): attestations.definition("no_such_rail") - - -def test_forgetting_rail_17s_verdict_reports_too_loud_rather_than_too_quiet() -> None: - """The DEFAULT, which every caller now passes explicitly -- so nothing else notices it. - - A defaulted `True` is what shipped first, and it means a future caller who forgets the - argument gets `ok · N day(s) remain` for an account whose broker has frozen withdrawals. - `None` is the rail's own word for "nobody has checked" and routes to the never-attested FAIL, - so the same forgetfulness costs a false alarm instead of false silence. - `cash_posture.py:91-94` states the rule for exactly this shape.""" - (rail17,) = [ - f - for f in doctor.attestation_findings( - subscription=None, withdrawals_attested_at=NOW - 3_600, now_ts=NOW - ) - if f.name == "attest.withdrawals" - ] - assert rail17.status == doctor.FAIL diff --git a/tests/test_cli.py b/tests/test_cli.py index c1a7b283..964ae241 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -2329,6 +2329,135 @@ def test_withdrawals_attest_ENABLED_proceeds_on_a_typed_yes(tmp_path, monkeypatc assert _repo_at(db).get_state("withdrawals_enabled") is True +def test_attest_enabled_says_it_clears_suspension_and_never_expires(tmp_path, monkeypatch): + """#1010: rail 17 reads the venue per buy, so `--enabled` only clears the manual brake. It + must not promise a TTL, and must not write the timestamp that used to carry one.""" + _at_a_terminal(monkeypatch) + db = tmp_path / "w4.db" + _repo_at(db).set_state("withdrawals_enabled", False) + + result = CliRunner().invoke( + cli, ["--db", str(db), "withdrawals", "attest", "--enabled"], input="yes\n" + ) + + assert result.exit_code == 0, result.output + assert "manual suspension cleared" in result.output + assert "expires" not in result.output + assert _repo_at(db).get_state("withdrawals_attested_at") is None + + +def test_attest_suspended_has_no_expiry(tmp_path, monkeypatch): + _at_a_terminal(monkeypatch, yes=False) + db = tmp_path / "w5.db" + _repo_at(db) + + result = CliRunner().invoke(cli, ["--db", str(db), "withdrawals", "attest", "--suspended"]) + + assert result.exit_code == 0, result.output + assert "expires" not in result.output + assert "entries" in result.output.lower() and "halted" in result.output.lower() + from keel.execution import executor + + assert executor.manual_withdrawal_suspension(_repo_at(db)) is True + assert _repo_at(db).get_state("withdrawals_attested_at") is None + + +def _withdrawals_show(tmp_path, valid_config_path, monkeypatch, broker, *, rules=("BTC-USD",)): + """Run `withdrawals show` against `broker`, recording the kwargs the broker was built with.""" + built: dict[str, Any] = {} + + def fake_build(config, **kw): + built.update(kw) + if isinstance(broker, BaseException): + raise broker + return broker + + monkeypatch.setattr("keel.commands.withdrawals._build_broker", fake_build) + db = tmp_path / "show.db" + repo = _repo_at(db) + for product in rules: + repo.insert_rule("dca", {"product_id": product}, status="live") + result = CliRunner().invoke( + cli, ["--db", str(db), "--config", str(valid_config_path), "withdrawals", "show"] + ) + return result, built, db + + +def test_show_prints_venue_answer_per_wallet(tmp_path, valid_config_path, monkeypatch): + import re + + broker = FakeBroker(withdrawals={"USD": True, "BTC": False}) + result, built, _db = _withdrawals_show(tmp_path, valid_config_path, monkeypatch, broker) + + assert result.exit_code == 0, result.output + assert "manual suspension: off" in result.output + assert re.search(r"BTC\s+withdrawals disabled", result.output), result.output + assert re.search(r"USD\s+allowed", result.output), result.output + assert broker.withdrawals_calls == 1 + assert built.get("timeout"), "the venue read must be bounded" + assert built.get("repo") is not None + + +def test_show_says_no_wallet_for_an_unheld_asset(tmp_path, valid_config_path, monkeypatch): + import re + + broker = FakeBroker(withdrawals={"USD": True}) # BTC -> None: no wallet + result, _built, _db = _withdrawals_show(tmp_path, valid_config_path, monkeypatch, broker) + + assert result.exit_code == 0, result.output + assert re.search(r"BTC\s+no wallet", result.output), result.output + + +def test_show_reports_unreadable_venue_without_failing(tmp_path, valid_config_path, monkeypatch): + broker = FakeBroker(withdrawals_raises=RuntimeError("boom")) + result, _built, _db = _withdrawals_show(tmp_path, valid_config_path, monkeypatch, broker) + + assert result.exit_code == 0, result.output + assert "venue unreadable" in result.output + assert "manual suspension: off" in result.output + + +def test_show_reports_unbuildable_broker_without_failing(tmp_path, valid_config_path, monkeypatch): + result, _built, _db = _withdrawals_show( + tmp_path, valid_config_path, monkeypatch, RuntimeError("no credentials") + ) + + assert result.exit_code == 0, result.output + assert "venue unreadable" in result.output + + +def test_show_names_a_manual_suspension(tmp_path, valid_config_path, monkeypatch): + broker = FakeBroker() + monkeypatch.setattr("keel.commands.withdrawals._build_broker", lambda config, **kw: broker) + db = tmp_path / "show2.db" + _repo_at(db).set_state("withdrawals_enabled", False) + + result = CliRunner().invoke( + cli, ["--db", str(db), "--config", str(valid_config_path), "withdrawals", "show"] + ) + + assert result.exit_code == 0, result.output + assert "manual suspension: ON" in result.output + + +def test_show_renders_an_unreadable_store_as_unknown_never_off( + tmp_path, valid_config_path, monkeypatch +): + broker = FakeBroker() + monkeypatch.setattr("keel.commands.withdrawals._build_broker", lambda config, **kw: broker) + monkeypatch.setattr("keel.execution.executor._stored_suspension", lambda repo: None) + db = tmp_path / "show3.db" + _repo_at(db) + + result = CliRunner().invoke( + cli, ["--db", str(db), "--config", str(valid_config_path), "withdrawals", "show"] + ) + + assert result.exit_code == 0, result.output + assert "manual suspension: unknown (store unreadable)" in result.output + assert "manual suspension: off" not in result.output + + def test_autonomy_on_for_hours_sets_an_expiry_that_lapses(tmp_path, monkeypatch, valid_config_path): """A forgotten `autonomy on` should not be able to grant unattended trading forever.""" _at_a_terminal(monkeypatch) diff --git a/tests/test_fiqh_basis.py b/tests/test_fiqh_basis.py index 9a4220c7..265ead19 100644 --- a/tests/test_fiqh_basis.py +++ b/tests/test_fiqh_basis.py @@ -199,19 +199,22 @@ def test_the_bare_holder_semantics_are_pinned_to_the_experiment_record(): ) -def test_rail_17s_seven_day_ttl_is_pinned_to_the_executor(): - """The doc names the TTL by its constant, two-sided with the executor that defines it. +def test_rail_17s_venue_read_is_pinned_to_the_executor(): + """The doc names the mechanism by its symbol, two-sided with the executor that defines it. - 'Fresh attestation' is a rubbery phrase; `WITHDRAWAL_ATTESTATION_TTL_SEC` is 7 days, and a - reader auditing the rail needs the number and the symbol that owns it. + Rail 17 reads the venue per buy and has no TTL (#1010); a reader auditing the rail needs the + function that does the reading, and a rename must fail here rather than orphan the citation. """ - assert "WITHDRAWAL_ATTESTATION_TTL_SEC" in _doc(), ( - f"{_DOC} must name rail 17's TTL as WITHDRAWAL_ATTESTATION_TTL_SEC (7 days), so the " - "number is traceable to the constant that enforces it" + assert "_withdrawal_capability" in _doc(), ( + f"{_DOC} must name rail 17's mechanism as `_withdrawal_capability`, so the claim is " + "traceable to the function that enforces it" ) - assert "WITHDRAWAL_ATTESTATION_TTL_SEC" in _rel("keel/execution/executor.py"), ( - "keel/execution/executor.py must still define WITHDRAWAL_ATTESTATION_TTL_SEC -- " - "the doc cites it, so a rename must fail here rather than orphan the citation" + assert "def _withdrawal_capability" in _rel("keel/execution/executor.py"), ( + "keel/execution/executor.py must still define _withdrawal_capability -- the doc cites " + "it, so a rename must fail here rather than orphan the citation" + ) + assert "WITHDRAWAL_ATTESTATION_TTL_SEC" not in _doc(), ( + f"{_DOC} still cites the 7-day TTL constant that #1010 deleted" ) diff --git a/tests/test_hostile_rule_rails.py b/tests/test_hostile_rule_rails.py index 0256ec4e..d28651f4 100644 --- a/tests/test_hostile_rule_rails.py +++ b/tests/test_hostile_rule_rails.py @@ -63,7 +63,6 @@ def _fresh_repo() -> Repository: r = Repository(conn) r.set_state("kill_switch", False) r.set_state("withdrawals_enabled", True) - r.set_state("withdrawals_attested_at", 10**12) r.set_autonomous(True, now_ts=0) attest_subscription(r, now_ts=0, free_volume_usd=Decimal("10000000")) attest_trade_scope(r, now_ts=0) diff --git a/tests/test_notifications.py b/tests/test_notifications.py index 2eb545e3..16a05afb 100644 --- a/tests/test_notifications.py +++ b/tests/test_notifications.py @@ -29,7 +29,11 @@ ) from keel import notifications -from keel.commands.doctor import attestation_findings, rail_state_findings +from keel.commands.doctor import ( + attestation_findings, + cash_posture_findings, + rail_state_findings, +) from keel.config import AutoTradeConfig, Caps, Config, MarketDataConfig from keel.execution.executor import ReduceResult from keel.execution.sleeve_exit import ExitWatch @@ -48,21 +52,11 @@ # -- builders --------------------------------------------------------------------------------- -def _attestation(withdrawals_attested_at: int, now: int = NOW, *, enabled: bool | None = True): +def _attestation(*, suspended: bool | None = False, now: int = NOW): """doctor's own computation, fed a `None` subscription (rail 14 is not what these events - are about) and the withdrawal-attestation freshness under test. - - `enabled=True` explicitly. `attestation_findings` used to default it to `True` and #794 - changed that to `None` -- "nobody has checked" -- because a defaulted `True` on a - safety-reporting function reports `ok` for an account whose broker has frozen withdrawals. - These tests are about the CLOCK, so they state the verdict rather than inherit it. - """ - return attestation_findings( - subscription=None, - withdrawals_attested_at=withdrawals_attested_at, - withdrawals_enabled=enabled, - now_ts=now, - ) + are about) and the manual-suspension state under test. Rail 17 carries no clock any more + (#1010), so the only inputs are the three states of the stored brake.""" + return attestation_findings(subscription=None, now_ts=now, withdrawals_suspended=suspended) def _rails( @@ -94,7 +88,7 @@ def _state( watch: tuple[ExitWatch, ...] = (), ): return events_from_state( - attestation_findings=attestation if attestation is not None else _attestation(NOW), + attestation_findings=attestation if attestation is not None else _attestation(), rail_findings=rails if rails is not None else _rails(), month_to_date_spend=month_to_date_spend, allowance=allowance, @@ -107,28 +101,42 @@ def _state( ) -# -- rail 17: attestation nearing expiry ------------------------------------------------------ +# -- rail 17 has no expiry event (#1010) ------------------------------------------------------ -def test_an_attestation_due_within_two_days_fires_and_a_fresh_one_does_not(): - """The issue's sharpest silent failure: rail 17 has a 7-day TTL, fails closed, and the - veto is a WARNING -- not a CRITICAL. doctor WARNs at <=2 days remaining; the notification - fires on exactly that threshold, from exactly that computation.""" - due_soon = _state(attestation=_attestation(NOW - 5 * DAY)) # 2 of 7 days remain - assert [e.key for e in due_soon] == ["attestation.expiring"] - assert "2 day(s)" in due_soon[0].message # doctor's own detail, numbers included - assert due_soon[0].fields["status"] == "warn" +@pytest.mark.parametrize("suspended", [False, True, None]) +def test_rail_17_never_raises_an_attestation_event(suspended): + """Rail 17 reads the venue per buy: there is no attestation to expire, so there is no + `attestation.expiring` for it -- not even when the operator's own suspension is set (the + person who set it knows, the way they know about the kill switch) or unreadable.""" + assert _state(attestation=_attestation(suspended=suspended)) == [] - fresh = _state(attestation=_attestation(NOW)) # 7 days remain - assert fresh == [] +def test_the_notification_registry_does_not_list_rail_17(): + assert "attest.withdrawals" not in _ATTESTATION_FINDINGS + assert "attest.withdrawals" not in notifications._RAIL_LABEL + assert "attest.withdrawals" not in notifications._ATTESTATION_MODEL_KEY -def test_an_expired_attestation_fires_too(): - """Expired is the sharper case the WARN threshold exists to precede: rail 17 is already - vetoing every entry at that point.""" - expired = _state(attestation=_attestation(NOW - 8 * DAY)) - assert [e.key for e in expired] == ["attestation.expiring"] +def test_notifications_never_emit_withdrawal_expiry_even_when_attested_long_ago(): + """A database that still carries a 30-day-old `withdrawals_attested_at` from the TTL era must + say nothing about it: with a healthy posture the cycle is silent.""" + calls: list[tuple[str, str]] = [] + repo = _Repo(legacy_attested_at=NOW - 30 * DAY) + repo.cash_posture = _healthy_posture() + config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) + + sent = notify_after_cycle( + repo, + config, + _LoopResult(), + NOW, + url="https://alerts.example/hook", + transport=_recording_transport(calls), + ) + + assert sent == 0 + assert calls == [] # -- rails arming ----------------------------------------------------------------------------- @@ -283,11 +291,12 @@ class _Repo: def __init__( self, *, - withdrawals_attested_at: int, held: tuple[str, ...] = (), withdrawals_enabled: bool | None = True, + legacy_attested_at: int | None = None, ) -> None: - self._withdrawals_attested_at = withdrawals_attested_at + #: The TTL-era timestamp, kept only so a test can prove it is no longer read. + self._legacy_attested_at = legacy_attested_at self._withdrawals_enabled = withdrawals_enabled self._held = held self.state_writes: list[tuple[str, object]] = [] @@ -298,10 +307,7 @@ def __init__( def get_state(self, key: str, default: object = None) -> object: if key == "withdrawals_attested_at": - return self._withdrawals_attested_at - # Rail 17's second key (#793). The double is a deployment that HAS attested, so the - # answer is the attestation's verdict; without it every fixture here reads as an account - # nobody ever attested for, and these tests are about the expiry clock. + return self._legacy_attested_at if self._legacy_attested_at is not None else default if key == "withdrawals_enabled": return self._withdrawals_enabled if key == "kill_switch": @@ -359,13 +365,10 @@ def _transport(url: str, body: bytes) -> None: def test_notify_after_cycle_reads_doctor_seams_and_sends_only_opted_in_events(): calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW - 5 * DAY) # rail 17: 2 days remain - # A HEALTHY posture, so this test stays about rail 17 alone. #732 wired - # `cash_posture_findings` into the same path, and the default double carries no posture at - # all -- which `doctor` reports as "cash posture never attested", a FAIL, because rail 22 - # vetoes on it. That is a real second event, not a fixture artefact, and it belongs to the - # tests below rather than to this one. - repo.cash_posture = _healthy_posture() + # A LAPSED posture: the one attestation finding this path can still deliver (#1010 took rail + # 17's out), so the test has exactly one event to count. The default double carries no posture + # at all -- "cash posture never attested", also a FAIL -- which belongs to its own test below. + repo = _lapsed_posture_repo() settings = NotificationSettings(events=frozenset({"attestation.expiring"})) config = _config_with(settings) @@ -390,7 +393,7 @@ def test_notify_after_cycle_reads_doctor_seams_and_sends_only_opted_in_events(): def test_notify_after_cycle_without_a_url_makes_zero_network_calls(monkeypatch): calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW - 5 * DAY) + repo = _Repo() config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) sent = notify_after_cycle( @@ -413,7 +416,7 @@ def get_state(self, key: str, default: object = None) -> object: config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) sent = notify_after_cycle( - _ExplodingRepo(withdrawals_attested_at=NOW), + _ExplodingRepo(), config, _LoopResult(), NOW, @@ -434,7 +437,7 @@ def test_notify_after_cycle_lets_base_exceptions_through_to_the_operator(): def _interrupt(url: str, body: bytes) -> None: raise KeyboardInterrupt - repo = _Repo(withdrawals_attested_at=NOW - 5 * DAY) + repo = _Repo() config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) with pytest.raises(KeyboardInterrupt): @@ -475,7 +478,8 @@ class TestEveryTaxonomyEventSurvivesDelivery: @staticmethod def _all_five_events(): return _state( - attestation=_attestation(NOW - 5 * DAY), # rail 17: 2 of 7 days remain -> WARN + # rail 22: never attested -> FAIL. Rail 17 has no event of its own any more (#1010). + attestation=cash_posture_findings(None, venue="coinbase", now_ts=NOW), rails=_rails(streak_halt_until=NOW + DAY), # rail 16's halt is armed month_to_date_spend=Decimal("850"), allowance=Decimal("1000"), # 85% used -- the event that used to carry a Decimal @@ -548,11 +552,11 @@ def _healthy_posture(): # -- the registry and the call site must agree (#732) ---------------------------------------------- -def _lapsed_posture_repo(*, withdrawals_attested_at: int) -> _Repo: - """A book where BOTH registered attestation findings are unhealthy at once.""" +def _lapsed_posture_repo() -> _Repo: + """A book whose rail-22 posture attestation has lapsed.""" from keel_core.cash_posture import CashPostureState, VenueCashPosture - repo = _Repo(withdrawals_attested_at=withdrawals_attested_at) + repo = _Repo() repo.cash_posture = VenueCashPosture( venue="coinbase", state=CashPostureState.ATTESTED, @@ -584,7 +588,7 @@ def test_every_registered_attestation_finding_is_actually_deliverable(): gatherer fails here rather than in production quiet. """ calls: list[tuple[str, str]] = [] - repo = _lapsed_posture_repo(withdrawals_attested_at=NOW - 5 * DAY) + repo = _lapsed_posture_repo() config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) notify_after_cycle( @@ -601,13 +605,13 @@ def test_every_registered_attestation_finding_is_actually_deliverable(): assert name in delivered, f"{name} is registered and nothing delivers it" -def test_both_attestation_rails_are_notified_when_both_are_unhealthy(): - """The `break` said "one event per cycle: the finding list carries one rail-17 verdict" -- - true when the registry held rail 17 alone. With rail 22 in it, a break makes a cash-posture - problem invisible whenever a withdrawals problem also exists: the same silence, one layer - down.""" +def test_only_rail_22_is_notified_when_the_posture_has_lapsed(): + """Rail 17 once shared this path (one event per unhealthy rail, so a withdrawals problem could + not hide a cash-posture one). It left in #1010; what must hold now is that a lapsed posture + still arrives alone, and that nothing about rail 17 does -- whatever its suspension says.""" calls: list[tuple[str, str]] = [] - repo = _lapsed_posture_repo(withdrawals_attested_at=NOW - 5 * DAY) + repo = _lapsed_posture_repo() + repo._withdrawals_enabled = False # a manual suspension is set: still no event for it config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) sent = notify_after_cycle( @@ -619,16 +623,16 @@ def test_both_attestation_rails_are_notified_when_both_are_unhealthy(): transport=_recording_transport(calls), ) - assert sent == 2, f"expected one event per unhealthy rail, got {sent}" + assert sent == 1, f"expected one event, for rail 22, got {sent}" delivered = " ".join(body for _url, body in calls) - assert "rail 17" in delivered assert "rail 22" in delivered + assert "rail 17" not in delivered def test_a_healthy_cash_posture_notifies_nothing(): """The other direction: an alert that fired on a healthy posture is the alert nobody reads.""" calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW - DAY) # rail 17 comfortably in date + repo = _Repo() repo.cash_posture = _healthy_posture() config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) @@ -650,7 +654,7 @@ def test_a_deployment_that_never_attested_a_posture_is_told(monkeypatch): vetoes live entries on it. This is the standing case #732 is really about: nothing lapsed, nothing broke, the agent simply cannot enter and had no way to say so.""" calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW - DAY) # rail 17 fine; no posture at all + repo = _Repo() # no posture at all config = _config_with(NotificationSettings(events=frozenset({"attestation.expiring"}))) sent = notify_after_cycle( @@ -735,7 +739,7 @@ def test_the_wiring_derives_sleeve_proposals_from_the_cycle_result(): """`notify_after_cycle` reads `result.reduce_results` (R19) and writes nothing new: the write list is still exactly the #793 ledger key, and here not even that.""" calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW) + repo = _Repo() repo.cash_posture = _healthy_posture() config = _config_with(NotificationSettings(events=frozenset({"sleeve.proposal"}))) @@ -759,7 +763,7 @@ def test_the_wiring_reads_which_stale_products_only_a_sleeve_rule_watches(): """`notify_after_cycle` hands `result.sleeve_only_products` to the pure derivation, so a stale, held product that only a sleeve rule watches is worded as paused proposals.""" calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW, held=("PAXG-USD", "BTC-USD")) + repo = _Repo(held=("PAXG-USD", "BTC-USD")) repo.cash_posture = _healthy_posture() config = _config_with(NotificationSettings(events=frozenset({"feed.stale_open_position"}))) @@ -832,7 +836,7 @@ def test_the_wiring_derives_exit_watch_events_from_the_cycle_result(): """`notify_after_cycle` reads `result.exit_watch_transitions` and writes nothing new: the write list is still exactly the #793 ledger key, and here not even that.""" calls: list[tuple[str, str]] = [] - repo = _Repo(withdrawals_attested_at=NOW) + repo = _Repo() repo.cash_posture = _healthy_posture() config = _config_with(NotificationSettings(events=frozenset({"sleeve.exit_watch"}))) diff --git a/tests/web/test_attestation_banner.py b/tests/web/test_attestation_banner.py index e0a5170e..a573a784 100644 --- a/tests/web/test_attestation_banner.py +++ b/tests/web/test_attestation_banner.py @@ -1,8 +1,10 @@ """The cockpit's attestation banner (#793). -The evidence this exists for: rail 17 expired on 2026-09-08 and the Status page read +The evidence this exists for: an attestation expired on 2026-09-08 and the Status page read `expired · EXPIRED FOR 2d ago` for three days while live placed no orders and nobody noticed. The state was on screen. It was a field in a card, below the fold, on one page of eleven. +Since #1010 the banner serves rail 22 alone: rail 17 reads the venue per buy and has no expiry, +so it is never a banner row -- its manual suspension is a card line, not an alert. So the banner is carried by the ENVELOPE rather than by the status payload -- every page gets it, including the ones an operator is actually looking at when the attestation lapses. The key @@ -29,8 +31,10 @@ def _at(state: str) -> attestations.Attestation: - """One attestation of `key` "withdrawal", positioned to be in `state` at `NOW`.""" - definition = attestations.definition("withdrawal") + """One attestation of `key` "cash_posture" (rail 22), positioned to be in `state` at `NOW`.""" + from keel_core.cash_posture import ATTESTATION_TTL_SEC + + definition = attestations.definition("cash_posture") if state == attestations.MISSING: return definition offsets = { @@ -39,7 +43,6 @@ def _at(state: str) -> attestations.Attestation: attestations.EXPIRED: -3 * 86_400, } expires_at = NOW + offsets[state] - made = expires_at - attestations.withdrawal_expiry(attested_at=0) assert definition.__class__ is attestations.Attestation return attestations.Attestation( key=definition.key, @@ -47,7 +50,7 @@ def _at(state: str) -> attestations.Attestation: label=definition.label, remedy=definition.remedy, warn_within_sec=definition.warn_within_sec, - attested_at=made, + attested_at=expires_at - ATTESTATION_TTL_SEC, expires_at=expires_at, ) @@ -68,10 +71,10 @@ def test_anything_but_ok_reaches_the_banner(state: str) -> None: def test_the_banner_names_the_remedy_because_the_browser_cannot_run_it() -> None: - """Rails 17 and 22 are attested from a TTY and #781 deliberately did not put them behind the - browser gate. A banner that says "expired" without the command is a dead end.""" + """Rail 22 is attested from a TTY and #781 deliberately did not put it behind the browser + gate. A banner that says "expired" without the command is a dead end.""" (alert,) = payload.attestation_alerts([_at(attestations.EXPIRED)], NOW) - assert alert["remedy"] == "keel withdrawals attest --enabled" + assert alert["remedy"] == "keel posture attest --spot-cash" def test_an_expiry_is_bad_and_an_approaching_one_is_a_warning() -> None: @@ -134,30 +137,28 @@ def _cfg(deployment: tuple[str, str]) -> Any: def test_every_endpoint_answers_with_the_banner_not_just_the_status_one( deployment: tuple[str, str], ) -> None: - """The point of putting it on the envelope. A fresh deployment has attested nothing, so rails - 17 and 22 are both `MISSING` and every page says so.""" + """The point of putting it on the envelope. A fresh deployment has attested no posture, so + rail 22 is `MISSING` and every page says so -- and rail 17 is never among them (#1010).""" from keel.web import api as web_api for path in ("/api/pulse", "/api/orders", "/api/holdings"): _, document = web_api.respond(_cfg(deployment), path, {}) rails = {alert["rail"] for alert in document["attestations"]} - assert rails == {"17", "22"}, path - + assert rails == {"22"}, path -def test_an_attested_deployment_raises_no_banner(deployment: tuple[str, str]) -> None: - """The negative control: without it the test above passes on a `respond` that always returns - both rails regardless of what the database says.""" - import time +def test_rail_17_state_never_raises_a_banner(deployment: tuple[str, str]) -> None: + """The negative control, both ways: neither an ancient attestation left over from the 7-day + era nor a manual suspension is a banner row. The suspension is a line on the status card; the + banner is for dates that lapse, and rail 17 has none.""" from keel.data.db import connect from keel.data.repository import Repository from keel.web import api as web_api db_path, _ = deployment repo = Repository(connect(db_path)) - now = int(time.time()) - repo.set_state("withdrawals_attested_at", str(now)) - repo.set_state("withdrawals_enabled", "true") + repo.set_state("withdrawals_attested_at", "1") + repo.set_state("withdrawals_enabled", False) web_api.close_repo(repo) _, document = web_api.respond(_cfg(deployment), "/api/pulse", {}) @@ -250,7 +251,7 @@ def test_one_expired_attestation_tones_the_whole_banner_as_an_outage() -> None: def test_every_alert_row_carries_its_remedy() -> None: - """Rejects dropping the ``: rails 17 and 22 are attested from a TTY, so a banner + """Rejects dropping the ``: rail 22 is attested from a TTY, so a banner without the command is a dead end for the one person who can act on it.""" body = _decl(_code("render.js"), "attestationLine") assert body.count('el("code", "remedy", plain(alert.remedy))') == 1 @@ -291,10 +292,10 @@ def test_the_banner_has_words_for_every_state_the_model_can_be_in() -> None: assert states - (set(table) | {attestations.OK}) == set(), sorted(table) -def test_a_suspended_attestation_reads_as_the_refusal_it_is() -> None: - """Rail 17 with a fresh attestation that says NO. The expiry is in the future, so the date +def test_a_refused_attestation_reads_as_the_refusal_it_is() -> None: + """Rail 22 with a fresh attestation that says NO. The expiry is in the future, so the date that means anything is when it was asked.""" - definition = attestations.definition("withdrawal") + definition = attestations.definition("cash_posture") refused = attestations.Attestation( key=definition.key, rail=definition.rail, diff --git a/tests/web/test_client_assets.py b/tests/web/test_client_assets.py index d36517f3..5c7a5d02 100644 --- a/tests/web/test_client_assets.py +++ b/tests/web/test_client_assets.py @@ -1370,7 +1370,7 @@ def test_the_pulse_key_scan_found_the_keys_it_claims_to_check() -> None: keys = _view_keys("pulseView", "data") assert "equity.high_water_mark" in keys assert "autonomy.live" in keys - assert "withdrawal_attestation.state" in keys + assert "withdrawal_attestation.suspended" in keys assert len(keys) >= 15, keys # The two filters, asserted rather than assumed -- both were added because this test failed diff --git a/tests/web/test_payload.py b/tests/web/test_payload.py index c050d670..923daeef 100644 --- a/tests/web/test_payload.py +++ b/tests/web/test_payload.py @@ -93,13 +93,7 @@ def _status_report(**overrides: Any) -> StatusReport: max_total_dd_pct=Decimal("0.20"), max_weekly_dd_pct=Decimal("0.08"), rail11_status="ok", - withdrawal_attestation=WithdrawalAttestationStatus( - state="attested", - enabled=True, - attested_at=NOW_TS - 86400, - expires_in_sec=6 * 86400, - expired_for_sec=None, - ), + withdrawal_attestation=WithdrawalAttestationStatus(suspended=False), paper_cash_usdc=Decimal("955.25"), open_positions=[ OpenPositionStatus( @@ -973,23 +967,25 @@ def test_an_unknown_rail_state_is_unknown_and_not_quietly_good() -> None: assert ok["drawdown"]["rail11"]["state"] == "good" -def test_an_expired_withdrawal_attestation_reads_as_bad() -> None: - """Rail 17's state is the one an operator must act on within a TTL; it arrives judged.""" - expired = _pulse_payload( - _status_report( - withdrawal_attestation=WithdrawalAttestationStatus( - state="expired", - enabled=True, - attested_at=NOW_TS - 12 * 86400, - expires_in_sec=None, - expired_for_sec=5 * 86400, - ) - ) +@pytest.mark.parametrize( + ("suspended", "value", "display", "state"), + [ + (True, "on", "ON — entries halted", "bad"), + (False, "off", "off", "neutral"), + (None, "unknown", "unknown (store unreadable)", "warn"), + ], +) +def test_rail_17_manual_suspension_arrives_judged_and_tri_state( + suspended: bool | None, value: str, display: str, state: str +) -> None: + """#1010: rail 17 has no expiry to show. The one thing an operator sets arrives with its word + and its tone, and an unreadable store is `unknown`, never `off`.""" + rail17 = _pulse_payload( + _status_report(withdrawal_attestation=WithdrawalAttestationStatus(suspended)) )["withdrawal_attestation"] - assert expired["state"]["state"] == "bad" - # `_human_age`, exactly as `_rail17_line` prints it -- "EXPIRED 5d ago", not "5d". - assert expired["expired_for"]["display"] == "5d ago" + assert set(rail17) == {"suspended"} + assert rail17["suspended"] == {"value": value, "display": display, "state": state} # -- presentation-ready: displays are built here, never derivable there --------------------------- @@ -1360,13 +1356,7 @@ def test_every_builder_survives_a_completely_empty_report() -> None: drawdown_total_pct=None, drawdown_weekly_pct=None, rail11_status="unknown", - withdrawal_attestation=WithdrawalAttestationStatus( - state="unattested", - enabled=None, - attested_at=None, - expires_in_sec=None, - expired_for_sec=None, - ), + withdrawal_attestation=WithdrawalAttestationStatus(suspended=False), paper_cash_usdc=None, open_positions=[], rule_counts={}, diff --git a/tests/web/test_view_sections.py b/tests/web/test_view_sections.py index 54b4aa10..9bad4c33 100644 --- a/tests/web/test_view_sections.py +++ b/tests/web/test_view_sections.py @@ -637,11 +637,7 @@ def test_the_job_panel_is_page_level_and_not_inside_any_section() -> None: "unbracketed_count", "why_not", "why_not_none", - "withdrawal_attestation.attested_at", - "withdrawal_attestation.enabled", - "withdrawal_attestation.expired_for", - "withdrawal_attestation.expires_in", - "withdrawal_attestation.state", + "withdrawal_attestation.suspended", ) From f7560c92f17f33a6d109ac99bec2347eb6575401 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 14:17:54 -0500 Subject: [PATCH 09/18] feat(dca): retry a skipped buy daily until the rule's next window (#1010) Dca.detect gains a ledger-fed catch_up state (DcaCatchUp): a window whose buy did not fill is retried on every later daily cycle until it fills or the next window opens, at most one fill per rule per window. Backtests, the sim and research never set it and keep window-open-day behaviour. The agent reads each rule's own BUYs in the orders.mode the cycle writes (live/paper); an unreadable ledger or a rule row with no timestamps disables catch-up for that rule. The same-UTC-day DCA characterization test is rewritten, as its header asks, now that a dedupe exists. Co-Authored-By: Claude Opus 5.5 --- keel/agent.py | 109 ++++++++++- keel/strategy/rules/dca.py | 72 ++++++- tests/strategy/test_dca_cadence.py | 133 ++++++++++++- tests/test_agent.py | 291 +++++++++++++++++++++++++++-- 4 files changed, 581 insertions(+), 24 deletions(-) diff --git a/keel/agent.py b/keel/agent.py index de9f2e85..9d0e4961 100644 --- a/keel/agent.py +++ b/keel/agent.py @@ -94,7 +94,7 @@ from keel.strategy.rules import sleeve_exit as sleeve_exit_rule from keel.strategy.rules.base import Action, Rule, Setup, Signal, completed_days from keel.strategy.rules.cusum_event import CusumEvent -from keel.strategy.rules.dca import Dca +from keel.strategy.rules.dca import Dca, DcaCatchUp, window_start_day from keel.strategy.rules.profit_take import ProfitTake from keel.strategy.rules.pullback_continuation import PullbackContinuation from keel.strategy.rules.reverse_dca import ReverseDca @@ -1188,6 +1188,99 @@ def _bought_today(repo: Repository, product_id: str, now_ts: int, *, mode: str) ) +def _dca_order_bar_day(order: dict[str, Any], *, mode: str) -> int: + """The epoch day of the daily BAR a DCA BUY row in the `mode` ledger bought. + + The two writers stamp `created_at` differently, so the bar is read per mode: a live row is + stamped with the cycle's clock (`executor._order_row`'s `now_ts`), and the cycle on day + `d + 1` buys bar `d` (`Dca.detect` reads the latest COMPLETED bar, and the stale-bar gate + withholds every entry whose bar is not the one just closed) -- so `created_at // 86400 - 1`. + A paper row is stamped with the signal's own ts (`PaperTrader._enter`), which IS the bar's + -- so `created_at // 86400`. Reading a paper fill with the live offset would put a + window-open buy in the previous window and buy the window again the next day. + + A row with no `created_at` raises: the ledger cannot say which window it filled, and a + catch-up is never fired on a ledger the cycle cannot read (`_attach_dca_catch_up`). + """ + created_at = order.get("created_at") + if created_at is None: + raise ValueError(f"order {order.get('id')} has no created_at; its window is unknown") + day = int(created_at) // _DAY + return day if mode == "paper" else day - 1 + + +def _dca_catch_up_state( + repo: Repository, rule: Dca, rule_row: dict[str, Any], *, mode: str +) -> DcaCatchUp: + """What `mode`'s own ledger says about DCA `rule` for its catch-up (#1010; `DcaCatchUp`). + + `mode` is the `orders.mode` value this cycle WRITES -- `"live"` for a live cycle (whatever + its executor mode: `executor._order_row` always writes `"live"`), `"paper"` for a paper one + (`PaperTrader._enter`) -- so a paper rehearsal of a rule never reads the live fills as its + own, nor the reverse. Only this rule's (`orders.rule_id`) BUYs count. + + - `since_day`: `promoted_at // 86400`, else `created_at // 86400`, else `None` (catch-up + off: a row that cannot say when it went live is never read as "live since forever"). + - `filled_window_day`: the window of the rule's most recent FILLED BUY, by the bar it + bought (`_dca_order_bar_day`), not by row order. + - `in_flight`: a BUY of the rule in `executor.RESTING_STATUSES`. + + Raises on an unreadable ledger; `_attach_dca_catch_up` turns that into "catch-up off". + """ + stamp = rule_row.get("promoted_at") + if stamp is None: + stamp = rule_row.get("created_at") + since_day = None if stamp is None else int(stamp) // _DAY + cadence_days = int(rule.params["cadence_days"]) + buys = [ + order + for order in repo.get_orders(mode=mode) + if order.get("rule_id") == rule.rule_id and order["side"] == Side.BUY.value + ] + filled_bar_days = [ + _dca_order_bar_day(order, mode=mode) for order in buys if order["status"] == "filled" + ] + return DcaCatchUp( + since_day=since_day, + filled_window_day=( + window_start_day(max(filled_bar_days), cadence_days) if filled_bar_days else None + ), + in_flight=any(order["status"] in executor.RESTING_STATUSES for order in buys), + ) + + +def _attach_dca_catch_up( + repo: Repository, rules: list[Rule], rule_rows: list[dict[str, Any]], *, mode: str +) -> None: + """Set `catch_up` on every loaded `Dca` from `mode`'s ledger, before this cycle's + `engine.evaluate` and `_dca_fires_today` read `Dca.detect` (#1010). + + Set on EVERY cycle, `None` included, so an instance reused across cycles never carries a + previous cycle's answer. A rule with no `rules.id` gets `None` (it has no ledger to read). + A lookup that raises costs that rule its catch-up THIS cycle -- `None`, the window-open day + only -- and is logged; it never fires a catch-up on a ledger it could not read, and never + costs the cycle or another rule's buy. + """ + rows_by_id = {row.get("id"): row for row in rule_rows} + for rule in rules: + if not isinstance(rule, Dca): + continue + rule.catch_up = None + row = rows_by_id.get(rule.rule_id) + if rule.rule_id is None or row is None: + continue + try: + rule.catch_up = _dca_catch_up_state(repo, rule, row, mode=mode) + except Exception: # noqa: BLE001 -- an unreadable ledger disables catch-up, not the cycle + log_exception( + logger, + "agent.dca_catch_up_unreadable", + product=rule.product_id, + rule_id=rule.rule_id, + mode=mode, + ) + + def _dca_fires_today( product_rules: list[Rule], candles_by_tf: dict[Granularity, list[Any]], @@ -2314,13 +2407,10 @@ def run_once( # row is built, and loaded on their own by `_sleeve_rules` (which also reads `paper` # rows in a live cycle, R16). Their products join the poll set, so the reduction step # at the end of the cycle has candles to decide on. + rule_rows = [row for row in repo.get_rules(rule_status) if not _is_sleeve_kind(row["kind"])] rules = [ rule - for rule in ( - _build_rule(row) - for row in repo.get_rules(rule_status) - if not _is_sleeve_kind(row["kind"]) - ) + for rule in (_build_rule(row) for row in rule_rows) if promotion.promotion_class_of(rule) != promotion.SLEEVE_SELL ] # `_sleeve_rules` reads `repo.get_rules("paper")` even in a LIVE cycle (R16), and a @@ -2378,6 +2468,13 @@ def run_once( # real executor `mode`; only what we surface to the user changes. reported_mode = "paper" if paper_trader is not None else mode log_event(logger, logging.INFO, "agent.mode_resolved", mode=reported_mode) + # DCA catch-up (#1010): each `Dca` reads THIS cycle's ledger -- the `orders.mode` the + # cycle writes: "paper" on the paper path, and "live" for BOTH executor modes + # (`executor._order_row`), never `mode`/`reported_mode`'s "confirm"/"autonomous" -- before + # `engine.evaluate` and `_dca_fires_today` (sleeve reductions) call its `detect`. + _attach_dca_catch_up( + repo, rules, rule_rows, mode="paper" if paper_trader is not None else "live" + ) max_age_sec = config.auto_trade.interval_sec * FEED_STALENESS_CYCLES finest = _finest_granularity(granularities) diff --git a/keel/strategy/rules/dca.py b/keel/strategy/rules/dca.py index ca2c4fc5..76a8656b 100644 --- a/keel/strategy/rules/dca.py +++ b/keel/strategy/rules/dca.py @@ -14,10 +14,19 @@ kill-switch and every other rail -- among them the per-order, per-day, correlated-size, cash and monthly-buy rails (2, 3, 5, 13, 14). `execution/guards.py`'s module docstring is the authoritative list; this rule (like every Phase 2 rule) only emits *intents*. + +**Catch-up until the next window (#1010).** A buy that was due and did not fill -- any rail +veto, a failed cycle, a venue error, the machine being off -- is attempted again on every later +daily cycle until it fills or the rule's next window opens, with at most one fill per rule per +window. The window arithmetic stays here and pure; WHAT the ledger says (`DcaCatchUp`) is read +by the live/paper cycle (`agent._dca_catch_up_state`) and set on the instance as `catch_up`. +Backtests, the account sim and research never set it, so they keep the window-open-day-only +behaviour exactly. """ from __future__ import annotations +from dataclasses import dataclass from decimal import Decimal from keel.strategy.rules.base import Rule, Setup, completed_days @@ -46,6 +55,32 @@ def next_cadence_day_ts(now_ts: int, cadence_days: int) -> int: return (day if remainder == 0 else day + cadence_days - remainder) * _SECONDS_PER_DAY +def window_start_day(day: int, cadence_days: int) -> int: + """The epoch day the cadence window containing `day` opened on: the last cadence day on or + before it (`is_cadence_day`'s expression, solved for the window rather than tested).""" + if cadence_days <= 0: + raise ValueError("cadence_days must be positive") + return day - day % cadence_days + + +@dataclass(frozen=True) +class DcaCatchUp: + """What the cycle's own ledger says about one DCA rule, for `Dca.detect`'s catch-up (#1010). + + - `since_day`: the epoch day the rule went to its current status (`promoted_at // 86400`, + else `created_at // 86400`). `None` when the row has neither: catch-up is OFF for that + rule (window-open day only) -- a missing timestamp is never read as "since forever". + - `filled_window_day`: `window_start_day` of the rule's most recent FILLED BUY in the + cycle's mode, or `None` when it has none. + - `in_flight`: a BUY of the rule is still resting (`executor.RESTING_STATUSES`); it blocks + every buy of the rule, so two are never in flight. + """ + + since_day: int | None + filled_window_day: int | None + in_flight: bool + + class Dca(Rule): """Cadence-based dip-buy accumulation rule. @@ -83,6 +118,10 @@ class docstring and the help system cannot drift apart. #: (`_is_dca_setup`) and rests nothing behind the buy. Declared so the bracket column reads #: "no stop by design" for its tranches rather than WARNing on every one (#961). places_bracket = False + #: The ledger's answer for catch-up (`DcaCatchUp`), set per cycle by the live/paper agent. + #: `None` -- the class default, and all a backtest, the sim or research ever sees -- is the + #: window-open-day-only behaviour this rule always had. + catch_up: DcaCatchUp | None = None def __init__( self, @@ -123,6 +162,19 @@ def __init__( def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None: """Pure. On a cadence boundary, a market-buy long `Setup` sized to `budget_usd` (scaled up on dips); off-cadence (or with no daily candles), `None`. + + With `catch_up` set (live/paper only), on the latest completed bar's day `d` and its + window `w = window_start_day(d, cadence_days)`, in this order: + + 1. a BUY in flight -> `None` (never two buys in flight); + 2. the ledger already filled window `w` -> `None` -- the window-open day too, so a cycle + that re-runs after the buy filled never re-buys it; + 3. `d == w` -> the ordinary window-open buy (`context["catch_up"]` False); + 4. no `since_day`, or the window's buying cycle (day `w + 1`, which buys bar `w`) ran + before the rule went to its status (`w + 1 < since_day`) -> `None`: a rule promoted + mid-window waits for the next one; + 5. otherwise -> a catch-up buy (`context["catch_up"]` True), sized from bar `d`, not the + window-opening bar. """ # COMPLETED days only (#821): the account sim's last daily bar is the still-forming day, # stored with the completed day's OHLC -- reading it from hour 0 bought at a close that @@ -134,8 +186,10 @@ def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None latest = candles[-1] cadence_days = self.params["cadence_days"] - if not is_cadence_day(latest.ts, cadence_days): - return None # off-cadence: no buy today + day = latest.ts // _SECONDS_PER_DAY + window_day = window_start_day(day, cadence_days) + if not self._due(day, window_day): + return None budget_usd: Decimal = self.params["budget_usd"] dip_bonus_pct: Decimal = self.params["dip_bonus_pct"] @@ -161,6 +215,8 @@ def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None "recent_high": recent_high, "size_usd": size_usd, "no_stop": True, + "catch_up": day != window_day, + "window_start_day": window_day, } return Setup( @@ -177,6 +233,18 @@ def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None ts=latest.ts, ) + def _due(self, day: int, window_day: int) -> bool: + """`detect`'s buy-or-not decision for bar `day` in window `window_day` (its docstring + has the order). `day == window_day` is `is_cadence_day`'s expression on that bar.""" + catch_up = self.catch_up + if catch_up is None: + return day == window_day # backtests/sim: the window-open day only + if catch_up.in_flight or catch_up.filled_window_day == window_day: + return False + if day == window_day: + return True + return catch_up.since_day is not None and window_day + 1 >= catch_up.since_day + def exit_signal(self, held: Setup, candles_by_tf: dict[Granularity, list[Candle]]) -> bool: """DCA never exits on a signal -- it's accumulation, not a risk-defined trade.""" return False diff --git a/tests/strategy/test_dca_cadence.py b/tests/strategy/test_dca_cadence.py index 9af26e46..e29fea18 100644 --- a/tests/strategy/test_dca_cadence.py +++ b/tests/strategy/test_dca_cadence.py @@ -7,7 +7,13 @@ import pytest -from keel.strategy.rules.dca import Dca, is_cadence_day, next_cadence_day_ts +from keel.strategy.rules.dca import ( + Dca, + DcaCatchUp, + is_cadence_day, + next_cadence_day_ts, + window_start_day, +) from keel.types import Candle, Granularity _DAY = 86_400 @@ -50,3 +56,128 @@ def test_the_helpers_refuse_a_non_positive_cadence() -> None: is_cadence_day(0, bad) with pytest.raises(ValueError): next_cadence_day_ts(0, bad) + + +# -- catch-up until the next window (#1010) --------------------------------------------------- + +_K = 1_400 # an arbitrary window index, far from day 0 so no lookback edge is involved + + +def _bars_ending(day: int) -> dict[Granularity, list[Candle]]: + return {Granularity.ONE_DAY: [_candle(d) for d in range(day - 3, day + 1)]} + + +def _rule(cadence: int, catch_up: DcaCatchUp | None) -> Dca: + rule = Dca("BTC-USD", cadence_days=cadence) + rule.catch_up = catch_up + return rule + + +def test_window_start_day_is_the_last_cadence_day_on_or_before_the_day() -> None: + assert window_start_day(14 * _K, 14) == 14 * _K + assert window_start_day(14 * _K + 13, 14) == 14 * _K + assert window_start_day(14 * _K + 14, 14) == 14 * (_K + 1) + for day in range(98, 120): + assert is_cadence_day(window_start_day(day, 7) * _DAY, 7) + + +def test_no_catch_up_state_is_unchanged_behaviour() -> None: + """Backtests and the sim never set `catch_up`: off-window days stay silent.""" + assert Dca("BTC-USD", cadence_days=7).catch_up is None + assert Dca("BTC-USD", cadence_days=7).detect(_bars_ending(7 * _K + 3)) is None + setup = Dca("BTC-USD", cadence_days=7).detect(_bars_ending(7 * _K)) + assert setup is not None + assert setup.context["catch_up"] is False and setup.context["window_start_day"] == 7 * _K + + +def test_catch_up_fires_on_a_missed_window_day() -> None: + rule = _rule(14, DcaCatchUp(since_day=0, filled_window_day=None, in_flight=False)) + setup = rule.detect(_bars_ending(14 * _K + 5)) + assert setup is not None + assert setup.context["catch_up"] is True + assert setup.context["window_start_day"] == 14 * _K + assert setup.ts == (14 * _K + 5) * _DAY, "sized and stamped from the catch-up day's bar" + + +def test_catch_up_sizes_from_the_catch_up_days_bar_not_the_window_opening_bar() -> None: + w = 14 * _K + bars = [_candle(d) for d in range(w - 3, w + 6)] + # The window-open bar is at 100; the catch-up day's bar closes at 80 (20% below the high). + last = bars[-1] + p = Decimal("80") + bars[-1] = Candle(ts=last.ts, open=p, high=p, low=p, close=p, volume=Decimal("1")) + rule = Dca("BTC-USD", cadence_days=14, dip_bonus_pct=Decimal("1")) + rule.catch_up = DcaCatchUp(since_day=0, filled_window_day=None, in_flight=False) + setup = rule.detect({Granularity.ONE_DAY: bars}) + assert setup is not None + assert setup.entry == Decimal("80") + assert setup.context["drawdown_pct"] == Decimal("20") + assert setup.context["size_usd"] == Decimal("50") * Decimal("1.2") + + +def test_catch_up_stops_once_the_window_filled() -> None: + rule = _rule(14, DcaCatchUp(0, filled_window_day=14 * _K, in_flight=False)) + assert rule.detect(_bars_ending(14 * _K + 5)) is None + assert rule.detect(_bars_ending(14 * _K + 13)) is None + + +def test_window_open_day_after_fill_in_same_window_does_not_rebuy() -> None: + """Review Focus 4: a cycle that re-runs on the window-open bar after the buy filled.""" + rule = _rule(14, DcaCatchUp(0, filled_window_day=14 * _K, in_flight=False)) + assert rule.detect(_bars_ending(14 * _K)) is None + + +def test_a_fill_in_the_previous_window_does_not_block_this_one() -> None: + rule = _rule(14, DcaCatchUp(0, filled_window_day=14 * (_K - 1), in_flight=False)) + assert rule.detect(_bars_ending(14 * _K)) is not None + assert rule.detect(_bars_ending(14 * _K + 3)) is not None + + +def test_in_flight_order_blocks_catch_up() -> None: + """Two buys are never in flight: a resting BUY blocks the retry AND the window-open buy.""" + rule = _rule(14, DcaCatchUp(0, filled_window_day=None, in_flight=True)) + assert rule.detect(_bars_ending(14 * _K + 5)) is None + assert rule.detect(_bars_ending(14 * _K)) is None + + +def test_rule_promoted_mid_window_waits_for_next_window() -> None: + """Review Focus 3: the window's buying cycle (day w + 1) ran before the rule went live.""" + rule = _rule(14, DcaCatchUp(since_day=14 * _K + 4, filled_window_day=None, in_flight=False)) + assert rule.detect(_bars_ending(14 * _K + 6)) is None + assert rule.detect(_bars_ending(14 * (_K + 1))) is not None + + +def test_rule_live_on_the_windows_buying_day_still_catches_up() -> None: + """since_day == w + 1: the rule was live on the day the window's buying cycle ran.""" + rule = _rule(14, DcaCatchUp(since_day=14 * _K + 1, filled_window_day=None, in_flight=False)) + setup = rule.detect(_bars_ending(14 * _K + 6)) + assert setup is not None and setup.context["catch_up"] is True + + +def test_next_window_open_day_is_a_normal_buy() -> None: + rule = _rule(14, DcaCatchUp(0, filled_window_day=14 * _K, in_flight=False)) + setup = rule.detect(_bars_ending(14 * (_K + 1))) + assert setup is not None + assert setup.context["catch_up"] is False + assert setup.context["window_start_day"] == 14 * (_K + 1) + + +def test_a_catch_up_with_no_since_day_never_catches_up() -> None: + """A rule with no `promoted_at`/`created_at` is never read as "live since forever": only the + window-open day buys, and the ledger guards still apply to it.""" + rule = _rule(14, DcaCatchUp(since_day=None, filled_window_day=None, in_flight=False)) + assert rule.detect(_bars_ending(14 * _K + 5)) is None + assert rule.detect(_bars_ending(14 * _K)) is not None + rule.catch_up = DcaCatchUp(since_day=None, filled_window_day=14 * _K, in_flight=False) + assert rule.detect(_bars_ending(14 * _K)) is None + + +def test_every_day_of_a_missed_window_fires_until_the_next_one() -> None: + """The spread: with nothing filled, every day of the window buys (catch-up after day w).""" + rule = _rule(7, DcaCatchUp(since_day=0, filled_window_day=None, in_flight=False)) + flags = [] + for d in range(7 * _K, 7 * _K + 7): + setup = rule.detect(_bars_ending(d)) + assert setup is not None + flags.append(setup.context["catch_up"]) + assert flags == [False] + [True] * 6 diff --git a/tests/test_agent.py b/tests/test_agent.py index 4c0afe7a..efdf9808 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -50,7 +50,7 @@ from keel.execution import executor, sleeve from keel.strategy.reduction import Reduction from keel.strategy.rules.base import Action, Rule, Setup, Signal, completed_days -from keel.strategy.rules.dca import Dca +from keel.strategy.rules.dca import Dca, DcaCatchUp from keel.strategy.rules.pullback_continuation import PullbackContinuation from keel.strategy.rules.sleeve_exit import SleeveExit from keel.strategy.rules.turtle_breakout import TurtleBreakout @@ -3942,7 +3942,7 @@ def get_balances(self): assert equity == Decimal("500"), f"cash for a held product's quote leg was missed: {equity!r}" -# -- CHARACTERIZATION: same-UTC-day re-entry is NOT deduped on the live path ------------------- +# -- CHARACTERIZATION: same-UTC-day re-entry is NOT deduped on the live path (except DCA) ------ # # These two tests are deliberately NOT regression tests for a bug fix -- they PIN a hazard that # exists today in the live path, on purpose, so it cannot regress into "nobody noticed it was @@ -3975,18 +3975,25 @@ def get_balances(self): # not just the shell script's stamp), these two tests SHOULD start failing. That failure is the # signal to REWRITE them to assert a single entry, not to delete them: until that rail exists, # this is the only thing in the test suite that would notice its absence. +# +# The DCA half HAS since been deduped (#1010): a DCA rule's buy is decided against its own ledger +# (`agent._dca_catch_up_state` -- at most one fill per rule per cadence window), so its test is +# now `test_two_cycles_in_one_utc_day_place_one_dca_order`. The risk-defined half below still +# pins the hazard: nothing dedupes a Turtle re-entry. -def test_two_cycles_in_one_utc_day_place_two_dca_orders(repo): - """CHARACTERIZATION, not a spec: pins the DCA half of the live day-stamp hazard described - in the section header above. See that comment for the full mechanism. +def test_two_cycles_in_one_utc_day_place_one_dca_order(repo): + """REWRITTEN, as the section header asks, now that the DCA half of the hazard is deduped + (#1010): this was `..._place_two_dca_orders`, pinning that two cycles in one UTC day bought + the same DCA window twice. The DCA catch-up reads the rule's own ledger + (`agent._dca_catch_up_state`), and a window the ledger already filled is never bought again + -- the window-open day included -- so the second cycle detects nothing. Template: `test_run_once_polls_evaluates_and_executes_a_real_dca_rule` (single day-0 daily candle, `Dca`'s cadence-boundary fixture -- `latest.ts // 86_400 % cadence_days == 0` with `cadence_days=7`, and `0 % 7 == 0`). `market_feed._latest_closed_ts(now_ts, 86_400)` resolves to that same day-0 candle for ANY `now_ts` inside calendar day 1 (`[86_400, 172_800)`), so - both cycles below evaluate the identical `Setup` off the identical candle -- the rule does - not "know" it already fired. + both cycles below see the identical candle. `EARLY`/`LATE` model the runner's own two-trigger-in-one-day shape (an hourly-launchd detector re-armed after a missed day-stamp -- see `keel-live-run.sh`'s header): `EARLY` is @@ -4005,17 +4012,13 @@ def test_two_cycles_in_one_utc_day_place_two_dca_orders(repo): first = run_once(broker, repo, config, now_ts=EARLY) second = run_once(broker, repo, config, now_ts=LATE) - # The hazard, stated as an assertion: BOTH cycles place. A real dedupe would make the second - # `placed is False` (or the signal never fire at all); today it does not. assert first.enter_results[0].placed is True - assert second.enter_results[0].placed is True + assert second.enter_signals == [], "the second cycle re-detected a window already filled" + assert second.stale_products == [], "fixture: the second cycle must reach the rule at all" orders = repo.get_orders(mode="live", product_id=PRODUCT) - assert len(orders) == 2, ( - "expected the SAME dca signal to place twice in one UTC day -- if this is 1, a dedupe " - "now exists and this test must be REWRITTEN (not deleted) to assert that instead" - ) - assert all(o["side"] == "BUY" for o in orders) + assert len(orders) == 1 + assert orders[0]["side"] == "BUY" def test_two_cycles_in_one_utc_day_reenter_the_same_turtle_breakout(repo, monkeypatch): @@ -7024,3 +7027,261 @@ def test_a_sleeve_exit_rules_longer_window_sizes_the_backfill_on_a_sleeve_only_p assert result.sleeve_only_products == [_PAXG], "fixture: PAXG is a sleeve-only product" cached = repo.get_candles(_PAXG, Granularity.ONE_DAY) assert (len(cached), cached[-1].ts) == (300, 399 * DAY) + + +# -- DCA catch-up until the next window (#1010) ------------------------------------------------- + + +def _dca_series(last_day: int, first_day: int = 0) -> list[Candle]: + return [_candle(d * DAY, "100") for d in range(first_day, last_day + 1)] + + +def test_vetoed_dca_window_buys_on_next_cycle_then_not_again(repo): + """Review Focus 2. Cycle 1 sees the window-open bar (day 14) and rail 17 vetoes the buy (the + venue read fails). Cycle 2, the next day, buys it -- once. Cycle 3 buys nothing: the ledger, + not the candle, says the window is filled. The next window opens as an ordinary buy.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0 + ) + series = _dca_series(14) + _seed_history(repo, series) + unreadable = FakeBroker( + series={(PRODUCT, Granularity.ONE_DAY): series}, + withdrawals_raises=RuntimeError("venue down"), + ) + + first = run_once(unreadable, repo, _config(), now_ts=15 * DAY + 3_600) + + assert [r.placed for r in first.enter_results] == [False], "fixture: the buy must be vetoed" + assert unreadable.withdrawals_calls >= 1, "fixture: rail 17 must have read the venue" + assert repo.get_orders(mode="live") == [] + + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + series.append(_candle(15 * DAY, "100")) + second = run_once(broker, repo, _config(), now_ts=16 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in second.enter_signals] == [True] + assert [s.setup.context["window_start_day"] for s in second.enter_signals] == [14] + assert [o["rule_id"] for o in repo.get_orders(mode="live", status="filled")] == [rule_id] + + series.append(_candle(16 * DAY, "100")) + third = run_once(broker, repo, _config(), now_ts=17 * DAY + 3_600) + + assert third.enter_signals == [] + assert [o["rule_id"] for o in repo.get_orders(mode="live", status="filled")] == [rule_id] + assert len(broker.place_calls) == 1 + + series.extend(_candle(d * DAY, "100") for d in range(17, 22)) + fourth = run_once(broker, repo, _config(), now_ts=22 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in fourth.enter_signals] == [False] + assert len(repo.get_orders(mode="live", status="filled")) == 2 + + +def test_a_window_bought_on_its_open_day_is_not_bought_again_after_a_rerun(repo): + """Anti-goal: a cycle that re-runs the next day (or after a crash) never re-buys a window + the window-open buy already filled.""" + repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + assert len(broker.place_calls) == 1, "fixture: the window-open buy must fill" + + rerun = run_once(broker, repo, _config(), now_ts=15 * DAY + 7_200) + series.append(_candle(15 * DAY, "100")) + next_day = run_once(broker, repo, _config(), now_ts=16 * DAY + 3_600) + + assert rerun.enter_signals == [] and next_day.enter_signals == [] + assert len(broker.place_calls) == 1 + + +def test_a_window_missed_entirely_is_bought_by_the_first_cycle_after_it(repo): + """The machine was off on the window's buying day: the first later cycle in the window buys.""" + repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0) + series = _dca_series(18) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=19 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in result.enter_signals] == [True] + assert len(broker.place_calls) == 1 + + +def test_a_rule_promoted_after_the_windows_buying_day_waits_for_the_next_window(repo): + """Review Focus 3, through the cycle: `promoted_at` (day 17) is after window 14's buying + cycle (day 15), so day 18's bar buys nothing -- even though `created_at` is day 0.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0 + ) + repo._conn.execute("UPDATE rules SET promoted_at = ? WHERE id = ?", (17 * DAY + 5, rule_id)) + repo._conn.commit() + series = _dca_series(18) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=19 * DAY + 3_600) + + assert result.enter_signals == [] and broker.place_calls == [] + + +def test_paper_window_buy_is_not_re_bought_the_next_day(repo): + """The paper ledger stamps a fill with its BAR's ts (`PaperTrader._enter`), not the cycle's + clock, so its window is read off that day -- else day 15 would read day 14's fill as the + previous window's and buy again.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="paper", now_ts=0 + ) + series = _dca_series(14) + _seed_history(repo, series) + broker = _MarketDataOnlyBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + cfg = _paper_config(paper=PaperConfig(starting_equity_usd=Decimal("100000"))) + + run_once(broker, repo, cfg, now_ts=15 * DAY + 3_600) + buys = [o for o in repo.get_orders(mode="paper") if o["side"] == Side.BUY.value] + assert [o["rule_id"] for o in buys] == [rule_id], "fixture: the paper window buy must fill" + + series.append(_candle(15 * DAY, "100")) + result = run_once(broker, repo, cfg, now_ts=16 * DAY + 3_600) + + assert result.enter_signals == [] + + +def test_paper_missed_window_catches_up_from_the_paper_ledger(repo): + """The paper path reads its OWN ledger: a live fill of the same rule does not fill the paper + window.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="paper", now_ts=0 + ) + repo.insert_order( + { + "mode": "live", + "product_id": PRODUCT, + "side": Side.BUY.value, + "order_type": "market", + "qty": Decimal("0.5"), + "status": "filled", + "rule_id": rule_id, + "created_at": 15 * DAY + 3_600, + "updated_at": 15 * DAY + 3_600, + } + ) + series = _dca_series(16) + _seed_history(repo, series) + broker = _MarketDataOnlyBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + cfg = _paper_config(paper=PaperConfig(starting_equity_usd=Decimal("100000"))) + + result = run_once(broker, repo, cfg, now_ts=17 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in result.enter_signals] == [True] + + +def test_an_unreadable_ledger_disables_catch_up_and_logs_it(repo, monkeypatch, caplog): + """Controller ruling 4: never fire a catch-up on a ledger the cycle could not read. The rule + falls back to the window-open day alone, and the failure is logged.""" + repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0) + calls: list[int] = [] + + def _boom(*args: Any, **kwargs: Any) -> DcaCatchUp: + calls.append(1) + raise RuntimeError("database is locked") + + monkeypatch.setattr(agent, "_dca_catch_up_state", _boom) + series = _dca_series(16) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + with caplog.at_level(logging.WARNING): + off_window = run_once(broker, repo, _config(), now_ts=17 * DAY + 3_600) + + assert calls, "the lookup was never attempted -- this test proves nothing" + assert off_window.enter_signals == [] + assert "agent.dca_catch_up_unreadable" in [r.getMessage() for r in caplog.records] + + series.extend(_candle(d * DAY, "100") for d in range(17, 22)) + window_open = run_once(broker, repo, _config(), now_ts=22 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in window_open.enter_signals] == [False] + + +def test_catch_up_state_reads_this_rules_buys_in_this_mode(repo): + rule_id = repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, now_ts=3 * DAY) + other_id = repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, now_ts=0) + [row] = [r for r in repo.get_rules() if r["id"] == rule_id] + rule = _build_rule(row) + + def _order(mode: str, rid: int, status: str, created_at: int, side: str = "BUY") -> None: + repo.insert_order( + { + "mode": mode, + "product_id": PRODUCT, + "side": side, + "order_type": "market", + "qty": Decimal("0.5"), + "status": status, + "rule_id": rid, + "created_at": created_at, + "updated_at": created_at, + } + ) + + empty = agent._dca_catch_up_state(repo, rule, row, mode="live") + assert empty == DcaCatchUp(since_day=3, filled_window_day=None, in_flight=False) + + _order("live", rule_id, "filled", 8 * DAY + 4_800) # cycle on day 8 buys bar 7: window 7 + _order("live", rule_id, "filled", 14 * DAY + 4_800) # cycle on day 14 buys bar 13: window 7 + _order("live", rule_id, "filled", 7 * DAY + 4_800) # older (bar 6, window 0), inserted last + _order("live", other_id, "filled", 22 * DAY + 4_800) # another rule + _order("paper", rule_id, "filled", 28 * DAY) # another mode + _order("live", rule_id, "filled", 29 * DAY + 4_800, side="SELL") # not a buy + _order("live", rule_id, "rejected", 30 * DAY + 4_800) # not filled + + state = agent._dca_catch_up_state(repo, rule, row, mode="live") + assert state == DcaCatchUp(since_day=3, filled_window_day=7, in_flight=False) + + paper = agent._dca_catch_up_state(repo, rule, row, mode="paper") + assert paper == DcaCatchUp(since_day=3, filled_window_day=28, in_flight=False), ( + "a paper fill is stamped with its bar's ts, so its window is its own day's" + ) + + for status in executor.RESTING_STATUSES: + _order("live", rule_id, status, 31 * DAY) + assert agent._dca_catch_up_state(repo, rule, row, mode="live").in_flight is True + assert agent._dca_catch_up_state(repo, rule, row, mode="paper").in_flight is False + + +def test_catch_up_since_day_prefers_promoted_at_and_never_guesses_a_missing_one(repo): + rule = Dca(PRODUCT, cadence_days=7) + rule.rule_id = 1 + row = {"id": 1, "created_at": 2 * DAY + 5, "promoted_at": 9 * DAY + 5} + assert agent._dca_catch_up_state(repo, rule, row, mode="live").since_day == 9 + row["promoted_at"] = None + assert agent._dca_catch_up_state(repo, rule, row, mode="live").since_day == 2 + row["created_at"] = None + assert agent._dca_catch_up_state(repo, rule, row, mode="live").since_day is None + + +def test_catch_up_day_counts_as_dca_day_for_sleeve_exclusion(repo, monkeypatch): + """Spec §3.4 + catch-up: day 40 is no 7-day window-open day (window 35), but window 35 never + filled, so day 40 is a DCA buy day -- even when rail 17 vetoes that buy, the same-day sale + beside it is refused.""" + _seed_rules( + repo, + monkeypatch, + (Dca(product_id=PRODUCT, cadence_days=7), "live"), + (_AlwaysReduceRule(PRODUCT), "live"), + ) + repo._conn.execute("UPDATE rules SET created_at = 0") + repo._conn.commit() + _held_since_day_0(repo) + now = _history(repo, 41) + broker = _HoldingBroker(withdrawals_raises=RuntimeError("venue down")) + + result = run_once(broker, repo, _config(), now_ts=now) + + assert [s.setup.context["catch_up"] for s in result.enter_signals] == [True] + assert [r.placed for r in result.enter_results] == [False], "fixture: the buy must be vetoed" + [row] = repo.get_sell_proposals() + assert (row["decision"], row["rails"]["sleeve"]) == ("vetoed", "same_day_dca") From 6e6161eac73338ad58f4f9bce36d0ca364200474 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 14:31:57 -0500 Subject: [PATCH 10/18] feat(alerts): macOS notification for every skipped live DCA buy (#1010) A vetoed live DCA buy is a zero-exit cycle, so neither the wrapper's alerts nor the unconfigured webhook reached the operator. keel now posts a macOS notification itself, once per rule per UTC day, for every due buy that did not go out: vetoed, feed stale, entries withheld, or blocked by a resting order (named, with how to resolve it). Live cycles only; the sink is inert off macOS and under pytest and never fails a cycle. Co-Authored-By: Claude Opus 5.5 --- keel/agent.py | 186 ++++++++++++++- keel/strategy/rules/dca.py | 26 +- packages/keel-core/keel_core/macos_notify.py | 58 +++++ tests/core/test_macos_notify.py | 85 +++++++ tests/test_agent.py | 235 +++++++++++++++++++ 5 files changed, 587 insertions(+), 3 deletions(-) create mode 100644 packages/keel-core/keel_core/macos_notify.py create mode 100644 tests/core/test_macos_notify.py diff --git a/keel/agent.py b/keel/agent.py index 9d0e4961..0af18e39 100644 --- a/keel/agent.py +++ b/keel/agent.py @@ -66,10 +66,12 @@ import time from collections.abc import Callable from dataclasses import dataclass, field, replace +from datetime import UTC, datetime from decimal import Decimal from typing import Any, Literal from keel_broker_api.results import MarketSchedule, SessionState +from keel_core import macos_notify from keel_core.products import quote_currency_of from keel_core.telemetry import ( bind_cycle, @@ -1240,12 +1242,16 @@ def _dca_catch_up_state( filled_bar_days = [ _dca_order_bar_day(order, mode=mode) for order in buys if order["status"] == "filled" ] + resting = tuple( + int(order["id"]) for order in buys if order["status"] in executor.RESTING_STATUSES + ) return DcaCatchUp( since_day=since_day, filled_window_day=( window_start_day(max(filled_bar_days), cadence_days) if filled_bar_days else None ), - in_flight=any(order["status"] in executor.RESTING_STATUSES for order in buys), + in_flight=bool(resting), + resting_order_ids=resting, ) @@ -1281,6 +1287,134 @@ def _attach_dca_catch_up( ) +#: Characters of a skip reason that reach the notification banner; the full reason is in the +#: cycle's log line. +_SKIP_REASON_MAX_CHARS = 180 + +#: The notification title for a skipped buy. +SKIPPED_BUY_TITLE = "keel-live: DCA buy skipped" + + +def _epoch_day_date(day: int) -> str: + return datetime.fromtimestamp(day * _DAY, tz=UTC).strftime("%Y-%m-%d") + + +def _skipped_dca_buys( + rules: list[Rule], + granularities: list[Granularity], + candles_by_tf_by_product: dict[str, dict[Granularity, list[Any]]], + repo: Repository, + *, + enter_signals: list[Signal], + enter_results: list[ExecutionResult], + stale_products: list[str], + blocked_entries: list[BlockedEntry], + entries_allowed: bool, +) -> list[SkippedBuy]: + """Every LIVE DCA buy that was due this cycle and did not go out, one per rule (#1010). + + "Due" is `Dca`'s own decision, so a rule that is simply between windows (or whose window + already filled) is never reported. Four ways a due buy fails to go out, each a skip: + + - it was signalled and not placed: the rails that vetoed it (`vetoed_by`, verbatim -- rail + 17's `withdrawal_capability: ...` names the wallet or the unreadable venue), else the + executor's `reason`; + - the product's feed was stale, so it was never evaluated; + - the whole cycle's entries were withheld (`entries_allowed` is false -- some product's gating + bar was not ready), so it was never evaluated; + - a resting BUY of the rule blocks it (`DcaCatchUp.in_flight`): reported with the local order + id and what to do, because nothing else will ever say so. + + The caller runs this on a LIVE cycle only. A rule with no `rules.id` is skipped: the + notification is per rule. + """ + signal_result = { + sig.rule_id: res + for sig, res in zip(enter_signals, enter_results, strict=False) + if sig.rule_id is not None + } + signals_by_rule = {sig.rule_id: sig for sig in enter_signals if sig.rule_id is not None} + skipped: list[SkippedBuy] = [] + for rule in rules: + if not isinstance(rule, Dca) or rule.rule_id is None: + continue + cadence_days = int(rule.params["cadence_days"]) + signal = signals_by_rule.get(rule.rule_id) + window_day: int | None + reason: str | None + if signal is not None: + result = signal_result[rule.rule_id] + if result.placed: + continue + reason = "; ".join(result.vetoed_by) or result.reason + window_day = ( + signal.setup.context.get("window_start_day") if signal.setup is not None else None + ) + else: + candles_by_tf = candles_by_tf_by_product.get(rule.product_id) + if candles_by_tf is None: # a stale product's series was never loaded + candles_by_tf = {g: repo.get_candles(rule.product_id, g) for g in granularities} + window_day = rule.due_window(candles_by_tf, ignore_in_flight=True) + if window_day is None: + continue + if rule.product_id in stale_products: + reason = f"{rule.product_id} feed is stale; the buy was not evaluated" + elif not entries_allowed: + lagging = ", ".join(sorted({f"{b.product} ({b.reason})" for b in blocked_entries})) + reason = f"entries withheld this cycle (gating bar not ready: {lagging})" + elif rule.catch_up is not None and rule.catch_up.in_flight: + ids = ", ".join(f"#{i}" for i in rule.catch_up.resting_order_ids) + reason = ( + f"previous DCA order {ids} still pending at the venue; " + "resolve it (reconcile or cancel) to resume buying" + ) + else: + continue # due, not blocked, no signal: nothing this function can name + skipped.append( + SkippedBuy( + rule_id=rule.rule_id, + product=rule.product_id, + reason=reason, + retry_until=( + None if window_day is None else _epoch_day_date(window_day + cadence_days) + ), + ) + ) + return skipped + + +def _post_skipped_buys( + repo: Repository, + skipped: list[SkippedBuy], + now_ts: int, + *, + post: Callable[[str, str], bool] = macos_notify.post, +) -> int: + """Post a macOS notification for each skipped buy, at most once per rule per UTC day. + Returns how many were posted (#1010). + + "Once per rule per day, repeating every day the buy stays skipped" is the spec, so this is + deliberately NOT #793's `unreported()` once-per-window dedupe, which would go quiet after the + first alert. The memory is `agent_state` `dca_skip_notified:` = the UTC date it was + last said; nothing reads it but this function, so a wrong value costs one repeat or one + silent day. It is written only after a SUCCESSFUL post: a banner that never showed is not + "said", and the next cycle that day tries again. + """ + today = datetime.fromtimestamp(now_ts, tz=UTC).strftime("%Y-%m-%d") + posted = 0 + for skip in skipped: + key = f"dca_skip_notified:{skip.rule_id}" + if repo.get_state(key) == today: + continue + message = f"{skip.product} (rule {skip.rule_id}): {skip.reason[:_SKIP_REASON_MAX_CHARS]}." + if skip.retry_until is not None: + message += f" Retrying daily until {skip.retry_until}." + if post(SKIPPED_BUY_TITLE, message): + repo.set_state(key, today) + posted += 1 + return posted + + def _dca_fires_today( product_rules: list[Rule], candles_by_tf: dict[Granularity, list[Any]], @@ -2235,6 +2369,22 @@ def _slot_market_closed(repo: Repository, config: Config, now_ts: int, venue: st # -- one cycle ----------------------------------------------------------------------------- +@dataclass(frozen=True) +class SkippedBuy: + """A live DCA buy that was due this cycle and did not go out (#1010), and why. + + `reason` is for the operator's screen: the rails that vetoed it verbatim, or the executor's + own reason, or what withheld it before an order was ever built. `retry_until` is the UTC date + (`YYYY-MM-DD`) the rule's NEXT window opens -- the day the daily retries stop -- or `None` + when the cycle could not say. + """ + + rule_id: int + product: str + reason: str + retry_until: str | None = None + + @dataclass(frozen=True) class LoopResult: """The outcome of one `run_once` cycle.""" @@ -2272,6 +2422,10 @@ class LoopResult: paper_equity: Decimal | None = None drawdown_total_pct: Decimal | None = None drawdown_weekly_pct: Decimal | None = None + # #1010: the live DCA buys that were due this cycle and did not go out, one per rule. Always + # empty on a paper cycle (nothing there can be "skipped" in a way the operator must hear + # about). Defaulted so every existing `LoopResult(...)` construction stays valid. + skipped_buys: list[SkippedBuy] = field(default_factory=list) def _effective_mode( @@ -2985,6 +3139,26 @@ def run_once( except Exception: # noqa: BLE001 -- the monitor must never cost the cycle log_exception(logger, "agent.exit_watch_failed") + # #1010: the live DCA buys due this cycle that did not go out. LIVE only -- a paper cycle + # has no operator to tell. Wrapped: building the list is a diagnostic, and a failure there + # must cost neither the cycle nor (below) the notification of nothing. + skipped_buys: list[SkippedBuy] = [] + if paper_trader is None: + try: + skipped_buys = _skipped_dca_buys( + rules, + granularities, + candles_by_tf_by_product, + repo, + enter_signals=enter_signals, + enter_results=enter_results, + stale_products=stale_products, + blocked_entries=blocked_entries, + entries_allowed=entries_allowed, + ) + except Exception: # noqa: BLE001 -- a diagnostic must never cost the cycle + log_exception(logger, "agent.skipped_buys_failed") + cycle_result = LoopResult( ts=now_ts, skipped=False, @@ -3003,6 +3177,7 @@ def run_once( paper_equity=result_paper_equity, drawdown_total_pct=result_drawdown_total_pct, drawdown_weekly_pct=result_drawdown_weekly_pct, + skipped_buys=skipped_buys, ) # Post-cycle notifications (#444): derive the opted-in events from the same seams @@ -3015,6 +3190,15 @@ def run_once( notify_after_cycle(repo, config, cycle_result, now_ts) + # Then the macOS banner for each skipped live DCA buy (#1010) -- the channel that reaches + # the operator on a zero-exit cycle, which `notify_after_cycle` does not on a deployment + # with no webhook. LAST, so nothing above waits on it, and wrapped on its own: a + # notification failure (the post, or the dedupe write) never fails the cycle. + try: + _post_skipped_buys(repo, cycle_result.skipped_buys, now_ts) + except Exception: # noqa: BLE001 -- a notification must never cost the cycle + log_exception(logger, "agent.skipped_buy_notify_failed") + return cycle_result finally: unbind_cycle(cycle_token) diff --git a/keel/strategy/rules/dca.py b/keel/strategy/rules/dca.py index 76a8656b..7614120f 100644 --- a/keel/strategy/rules/dca.py +++ b/keel/strategy/rules/dca.py @@ -74,11 +74,15 @@ class DcaCatchUp: cycle's mode, or `None` when it has none. - `in_flight`: a BUY of the rule is still resting (`executor.RESTING_STATUSES`); it blocks every buy of the rule, so two are never in flight. + - `resting_order_ids`: the local `orders.id`s behind `in_flight`, for the operator-facing + reason a blocked buy is reported with. Defaulted, and never read by `detect`, so every + caller that builds a `DcaCatchUp` without it keeps its meaning. """ since_day: int | None filled_window_day: int | None in_flight: bool + resting_order_ids: tuple[int, ...] = () class Dca(Rule): @@ -233,13 +237,31 @@ def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None ts=latest.ts, ) - def _due(self, day: int, window_day: int) -> bool: + def due_window( + self, candles_by_tf: dict[Granularity, list[Candle]], *, ignore_in_flight: bool = False + ) -> int | None: + """Pure. The window (its opening epoch day) `detect` would buy on these candles, or + `None` when it would not. `ignore_in_flight` answers "would it buy but for a resting + order?" -- the question the cycle asks to report a BLOCKED buy rather than stay silent + about it. The same `_due` decision `detect` makes, so the two cannot drift.""" + candles = completed_days(candles_by_tf) + if not candles: + return None + day = candles[-1].ts // _SECONDS_PER_DAY + window_day = window_start_day(day, self.params["cadence_days"]) + if self._due(day, window_day, ignore_in_flight=ignore_in_flight): + return window_day + return None + + def _due(self, day: int, window_day: int, *, ignore_in_flight: bool = False) -> bool: """`detect`'s buy-or-not decision for bar `day` in window `window_day` (its docstring has the order). `day == window_day` is `is_cadence_day`'s expression on that bar.""" catch_up = self.catch_up if catch_up is None: return day == window_day # backtests/sim: the window-open day only - if catch_up.in_flight or catch_up.filled_window_day == window_day: + if (catch_up.in_flight and not ignore_in_flight) or ( + catch_up.filled_window_day == window_day + ): return False if day == window_day: return True diff --git a/packages/keel-core/keel_core/macos_notify.py b/packages/keel-core/keel_core/macos_notify.py new file mode 100644 index 00000000..7200be1c --- /dev/null +++ b/packages/keel-core/keel_core/macos_notify.py @@ -0,0 +1,58 @@ +"""A macOS notification posted by keel itself (#1010). + +Why this exists beside `keel_core.alerting` and `keel.notifications`: a vetoed live DCA buy is a +normal zero-exit cycle. The wrapper's `osascript` alerts fire only on non-zero exits, and the +webhook path is opt-in and unconfigured on the live deployment, so the operator found out days +late, more than once. This is the channel that reaches the screen on the cycle that skipped. + +**Inert where it cannot help.** Off macOS, with `KEEL_DISABLE_MACOS_NOTIFY` set, and under pytest +without an injected `runner`, `post` does nothing and returns `False` -- a test run must never +pop a banner on the developer's Mac, and a Linux CI box has no `osascript`. + +**Never breaks the caller.** Every error is swallowed and reported as `False`; a notification +must not cost a trading cycle. The 10 s timeout bounds a hung Notification Centre. +""" + +from __future__ import annotations + +import os +import subprocess +import sys +from collections.abc import Callable +from typing import Any + +#: Set (to anything) to silence every post, e.g. on a deployment that does not want banners. +DISABLE_ENV = "KEEL_DISABLE_MACOS_NOTIFY" + +OSASCRIPT = "/usr/bin/osascript" +TIMEOUT_SEC = 10 + + +def _escape(text: str) -> str: + """Make `text` safe inside an AppleScript string literal: `\\` and `"` escaped, and line + breaks flattened to spaces (a raw newline would end the literal and break the script).""" + flat = text.replace("\r", " ").replace("\n", " ") + return flat.replace("\\", "\\\\").replace('"', '\\"') + + +def _run(argv: list[str]) -> Any: + return subprocess.run(argv, timeout=TIMEOUT_SEC, check=True, capture_output=True) + + +def post( + title: str, + message: str, + *, + runner: Callable[[list[str]], Any] | None = None, +) -> bool: + """Post a macOS notification; `True` only when `osascript` ran cleanly. Never raises.""" + try: + if sys.platform != "darwin" or os.environ.get(DISABLE_ENV): + return False + if runner is None and os.environ.get("PYTEST_CURRENT_TEST"): + return False + script = f'display notification "{_escape(message)}" with title "{_escape(title)}"' + (runner or _run)([OSASCRIPT, "-e", script]) + return True + except Exception: # noqa: BLE001 -- a notification must never cost the caller + return False diff --git a/tests/core/test_macos_notify.py b/tests/core/test_macos_notify.py new file mode 100644 index 00000000..184626a3 --- /dev/null +++ b/tests/core/test_macos_notify.py @@ -0,0 +1,85 @@ +"""Tests for the macOS notification sink (#1010). + +The sink is the one channel that reaches the operator on a zero-exit cycle: a vetoed live DCA buy +is a normal cycle, so neither the wrapper's non-zero-exit alerts nor an unconfigured webhook ever +fire. It must be inert off macOS and under pytest, and must never raise into the cycle. +""" + +from __future__ import annotations + +import sys +from typing import Any + +import pytest +from keel_core.macos_notify import DISABLE_ENV, post + + +def _boom(_argv: list[str]) -> Any: + raise AssertionError("the runner must not be called") + + +def _raises(_argv: list[str]) -> Any: + raise OSError("osascript missing") + + +@pytest.fixture(autouse=True) +def _darwin(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(sys, "platform", "darwin") + monkeypatch.delenv(DISABLE_ENV, raising=False) + + +def test_post_is_noop_off_darwin(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(sys, "platform", "linux") + assert post("t", "m", runner=_boom) is False + + +def test_post_is_noop_when_disabled_by_env(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv(DISABLE_ENV, "1") + assert post("t", "m", runner=_boom) is False + + +def test_post_is_noop_under_pytest_without_an_injected_runner() -> None: + # PYTEST_CURRENT_TEST is set by pytest itself while this test runs; with no runner the + # default osascript runner must never be reached. + assert post("t", "m") is False + + +def test_post_runs_osascript_with_the_script() -> None: + calls: list[list[str]] = [] + assert post("keel-live: DCA buy skipped", "PAXG-USD (rule 9): x", runner=calls.append) is True + [argv] = calls + assert argv[:2] == ["/usr/bin/osascript", "-e"] + assert argv[2] == ( + 'display notification "PAXG-USD (rule 9): x" with title "keel-live: DCA buy skipped"' + ) + + +def test_post_escapes_quotes_and_backslashes_in_both_strings() -> None: + calls: list[list[str]] = [] + post('a"b', "c\\d", runner=calls.append) + assert calls[0][2] == 'display notification "c\\\\d" with title "a\\"b"' + + +def test_post_flattens_newlines_so_a_reason_cannot_break_the_script() -> None: + calls: list[list[str]] = [] + post("t", "line one\nline two", runner=calls.append) + assert "\n" not in calls[0][2] + + +def test_post_swallows_runner_errors() -> None: + assert post("t", "m", runner=_raises) is False + + +def test_default_runner_uses_a_ten_second_timeout(monkeypatch: pytest.MonkeyPatch) -> None: + import subprocess + + seen: dict[str, Any] = {} + + def fake_run(argv: list[str], **kwargs: Any) -> Any: + seen.update(argv=argv, **kwargs) + + monkeypatch.setattr(subprocess, "run", fake_run) + monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False) + assert post("t", "m") is True + assert seen["timeout"] == 10 + assert seen["check"] is True diff --git a/tests/test_agent.py b/tests/test_agent.py index efdf9808..009f330a 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -7285,3 +7285,238 @@ def test_catch_up_day_counts_as_dca_day_for_sleeve_exclusion(repo, monkeypatch): assert [r.placed for r in result.enter_results] == [False], "fixture: the buy must be vetoed" [row] = repo.get_sell_proposals() assert (row["decision"], row["rails"]["sleeve"]) == ("vetoed", "same_day_dca") + + +# -- a macOS notification for every skipped live DCA buy (#1010) --------------------------------- + + +def _spy_post(monkeypatch) -> list[list[agent.SkippedBuy]]: + """Replace the cycle's notifier so no test posts anything; records what it was handed.""" + seen: list[list[agent.SkippedBuy]] = [] + + def _record(repo, skipped, now_ts, **_kw): + seen.append(list(skipped)) + return 0 + + monkeypatch.setattr(agent, "_post_skipped_buys", _record) + return seen + + +def _live_dca_rule(repo) -> int: + return repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0 + ) + + +def test_skipped_live_dca_buy_posts_once_per_rule_per_day(repo): + """Review Focus 5. Once per rule per UTC day -- NOT #793's once-per-window `unreported()`, + which would silence every repeat -- and a failed post is not remembered as said.""" + posted: list[tuple[str, str]] = [] + + def _ok(title: str, message: str) -> bool: + posted.append((title, message)) + return True + + reason = "withdrawal_capability: UNKNOWN (venue down)" + skip = agent.SkippedBuy(9, "PAXG-USD", reason, retry_until="1970-01-22") + now = 15 * DAY + 3_600 + + assert agent._post_skipped_buys(repo, [skip, skip], now, post=_ok) == 1 + assert agent._post_skipped_buys(repo, [skip], now + 3_600, post=_ok) == 0 + assert len(posted) == 1 + assert posted[0] == ( + "keel-live: DCA buy skipped", + f"PAXG-USD (rule 9): {reason}. Retrying daily until 1970-01-22.", + ) + assert repo.get_state("dca_skip_notified:9") == "1970-01-16" + + assert agent._post_skipped_buys(repo, [skip], now + DAY, post=_ok) == 1 + assert len(posted) == 2 + + other = agent.SkippedBuy(10, "BTC-USD", "max_per_day") + assert agent._post_skipped_buys(repo, [other], now + DAY, post=_ok) == 1 + + +def test_skipped_buy_reason_is_cut_to_180_characters(repo): + posted: list[str] = [] + skip = agent.SkippedBuy(1, "BTC-USD", "x" * 500, retry_until="1970-01-22") + agent._post_skipped_buys(repo, [skip], DAY, post=lambda t, m: posted.append(m) or True) + assert posted == ["BTC-USD (rule 1): " + "x" * 180 + ". Retrying daily until 1970-01-22."] + + +def test_a_failed_post_is_not_remembered_so_the_next_cycle_tries_again(repo): + skip = agent.SkippedBuy(1, "BTC-USD", "max_per_day") + assert agent._post_skipped_buys(repo, [skip], DAY, post=lambda t, m: False) == 0 + assert repo.get_state("dca_skip_notified:1") is None + assert agent._post_skipped_buys(repo, [skip], DAY + 60, post=lambda t, m: True) == 1 + + +def test_cycle_with_vetoed_dca_records_skipped_buy(repo, monkeypatch): + seen = _spy_post(monkeypatch) + rule_id = _live_dca_rule(repo) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker( + series={(PRODUCT, Granularity.ONE_DAY): series}, + withdrawals_raises=RuntimeError("venue down"), + ) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert [r.placed for r in result.enter_results] == [False], "fixture: the buy is vetoed" + [skip] = result.skipped_buys + assert (skip.rule_id, skip.product) == (rule_id, PRODUCT) + assert skip.reason.startswith("withdrawal_capability:") + assert skip.retry_until == "1970-01-22", "the next window (day 21) opens on 1970-01-22" + assert seen == [[skip]], "the cycle hands its skipped buys to the notifier" + + +def test_a_placed_dca_buy_records_no_skipped_buy(repo, monkeypatch): + seen = _spy_post(monkeypatch) + _live_dca_rule(repo) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert [r.placed for r in result.enter_results] == [True], "fixture: the buy places" + assert result.skipped_buys == [] + assert seen in ([], [[]]) + + +def test_a_day_the_rule_is_not_due_records_no_skipped_buy(repo, monkeypatch): + _spy_post(monkeypatch) + _live_dca_rule(repo) + series = _dca_series(17) # day 17 is mid-window and the window-14 buy has filled + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) # window 14 fills + assert len(broker.place_calls) == 1, "fixture: the window filled" + + result = run_once(broker, repo, _config(), now_ts=18 * DAY + 3_600) + + assert result.enter_signals == [] and result.skipped_buys == [] + + +def test_paper_cycle_records_no_skipped_buys(repo, monkeypatch): + seen = _spy_post(monkeypatch) + repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 7}, status="paper", now_ts=0) + series = _dca_series(14) + _seed_history(repo, series) + broker = _MarketDataOnlyBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + # No synthetic equity -> the paper buy is skipped (placed False): live would record it. + cfg = _paper_config(paper=PaperConfig(starting_equity_usd=Decimal("0.01"))) + + result = run_once(broker, repo, cfg, now_ts=15 * DAY + 3_600) + + assert [s.setup.context["window_start_day"] for s in result.enter_signals] == [14] + assert result.skipped_buys == [] + assert all(batch == [] for batch in seen) + + +def test_notifier_failure_does_not_fail_cycle(repo, monkeypatch, caplog): + def _explode(*_a, **_k): + raise RuntimeError("notification centre on fire") + + monkeypatch.setattr(agent, "_post_skipped_buys", _explode) + _live_dca_rule(repo) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker( + series={(PRODUCT, Granularity.ONE_DAY): series}, + withdrawals_raises=RuntimeError("venue down"), + ) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert len(result.skipped_buys) == 1 + assert "agent.skipped_buy_notify_failed" in [r.getMessage() for r in caplog.records] + + +def test_building_the_skipped_buys_failing_does_not_fail_cycle(repo, monkeypatch): + def _explode(*_a, **_k): + raise RuntimeError("cannot build") + + monkeypatch.setattr(agent, "_skipped_dca_buys", _explode) + _live_dca_rule(repo) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert [r.placed for r in result.enter_results] == [True] + assert result.skipped_buys == [] + + +def test_an_in_flight_order_blocking_a_due_buy_is_a_skip_naming_the_order(repo, monkeypatch): + """Controller ruling 1: a due buy blocked by a resting order is NOT silent. The reason names + the local order id and says to resolve it, and it notifies every day like any other skip.""" + _spy_post(monkeypatch) + rule_id = _live_dca_rule(repo) + order_id = repo.insert_order( + { + "mode": "live", + "product_id": PRODUCT, + "side": Side.BUY.value, + "order_type": "market", + "qty": Decimal("0.5"), + "status": "pending", + "rule_id": rule_id, + "created_at": 8 * DAY + 3_600, + "updated_at": 8 * DAY + 3_600, + } + ) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert result.enter_signals == [] and broker.place_calls == [], "fixture: the buy is blocked" + [skip] = result.skipped_buys + assert (skip.rule_id, skip.product) == (rule_id, PRODUCT) + assert f"#{order_id}" in skip.reason + assert "resolve" in skip.reason + assert skip.retry_until == "1970-01-22" + + +def test_withheld_entries_make_every_due_dca_rule_a_skip(repo, monkeypatch): + """Controller ruling 2: the whole cycle's entries are withheld (a lagging product's bar), and + the due DCA rule beside it is a skip with that reason -- the very silence the operator hit.""" + _spy_post(monkeypatch) + config = _blocked_cycle_config() + now_ts = 11 * DAY + ready_id = repo.insert_rule("dca", {"product_id": PRODUCT, "cadence_days": 1}, status="live") + repo.upsert_candles(PRODUCT, Granularity.ONE_DAY, [_candle(10 * DAY, "100")]) + repo.insert_rule("dca", {"product_id": _XLM, "cadence_days": 1}, status="live") + repo.upsert_candles(_XLM, Granularity.ONE_DAY, [_candle(9 * DAY, "100")]) + broker = FakeBroker( + series={ + (PRODUCT, Granularity.ONE_DAY): [_candle(10 * DAY, "100")], + (_XLM, Granularity.ONE_DAY): [_candle(9 * DAY, "100")], + } + ) + + result = run_once(broker, repo, config, now_ts=now_ts) + + assert result.enter_signals == [], "fixture: entries are withheld" + by_product = {s.product: s for s in result.skipped_buys} + assert PRODUCT in by_product and by_product[PRODUCT].rule_id == ready_id + assert "withheld" in by_product[PRODUCT].reason + + +def test_a_stale_feed_makes_the_due_dca_rule_a_skip(repo, monkeypatch): + _spy_post(monkeypatch) + _live_dca_rule(repo) + series = _dca_series(14) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + # 30 days later the stored series is long stale (interval_sec=50_000 -> ~42h tolerance). + result = run_once(broker, repo, _config(), now_ts=45 * DAY + 3_600) + + assert PRODUCT in result.stale_products, "fixture: the product is stale" + [skip] = result.skipped_buys + assert PRODUCT == skip.product and "stale" in skip.reason From 082d1f23315e5bd9ad31e2b2deb38f2a56e14d32 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 14:34:53 -0500 Subject: [PATCH 11/18] fix(alerts): judge a skipped DCA buy by today, not a stale feed's last bar (#1010) Co-Authored-By: Claude Opus 5.5 --- keel/agent.py | 15 +++++---------- keel/strategy/rules/dca.py | 19 ++++++++----------- tests/test_agent.py | 25 +++++++++++++++++++++++++ 3 files changed, 38 insertions(+), 21 deletions(-) diff --git a/keel/agent.py b/keel/agent.py index 0af18e39..302af50a 100644 --- a/keel/agent.py +++ b/keel/agent.py @@ -1301,9 +1301,7 @@ def _epoch_day_date(day: int) -> str: def _skipped_dca_buys( rules: list[Rule], - granularities: list[Granularity], - candles_by_tf_by_product: dict[str, dict[Granularity, list[Any]]], - repo: Repository, + now_ts: int, *, enter_signals: list[Signal], enter_results: list[ExecutionResult], @@ -1351,10 +1349,9 @@ def _skipped_dca_buys( signal.setup.context.get("window_start_day") if signal.setup is not None else None ) else: - candles_by_tf = candles_by_tf_by_product.get(rule.product_id) - if candles_by_tf is None: # a stale product's series was never loaded - candles_by_tf = {g: repo.get_candles(rule.product_id, g) for g in granularities} - window_day = rule.due_window(candles_by_tf, ignore_in_flight=True) + # By the CLOCK, not the stored series: a stale feed's last bar would judge a window + # that closed long ago (silent skip, past retry date). The cycle on day D buys bar D-1. + window_day = rule.due_window(now_ts // _DAY - 1, ignore_in_flight=True) if window_day is None: continue if rule.product_id in stale_products: @@ -3147,9 +3144,7 @@ def run_once( try: skipped_buys = _skipped_dca_buys( rules, - granularities, - candles_by_tf_by_product, - repo, + now_ts, enter_signals=enter_signals, enter_results=enter_results, stale_products=stale_products, diff --git a/keel/strategy/rules/dca.py b/keel/strategy/rules/dca.py index 7614120f..cbda860a 100644 --- a/keel/strategy/rules/dca.py +++ b/keel/strategy/rules/dca.py @@ -237,17 +237,14 @@ def detect(self, candles_by_tf: dict[Granularity, list[Candle]]) -> Setup | None ts=latest.ts, ) - def due_window( - self, candles_by_tf: dict[Granularity, list[Candle]], *, ignore_in_flight: bool = False - ) -> int | None: - """Pure. The window (its opening epoch day) `detect` would buy on these candles, or - `None` when it would not. `ignore_in_flight` answers "would it buy but for a resting - order?" -- the question the cycle asks to report a BLOCKED buy rather than stay silent - about it. The same `_due` decision `detect` makes, so the two cannot drift.""" - candles = completed_days(candles_by_tf) - if not candles: - return None - day = candles[-1].ts // _SECONDS_PER_DAY + def due_window(self, day: int, *, ignore_in_flight: bool = False) -> int | None: + """Pure. The window (its opening epoch day) a buy off the daily bar of epoch day `day` + would be due for, or `None` when it would not be. `ignore_in_flight` answers "would it buy + but for a resting order?". The same `_due` decision `detect` makes, so they cannot drift. + + Takes a DAY, not candles: the cycle asks it for the last completed day by the CLOCK, to + report a buy it never evaluated (stale feed, withheld entries, resting order) -- reading + the stored series there would judge a stale feed by its stale last bar.""" window_day = window_start_day(day, self.params["cadence_days"]) if self._due(day, window_day, ignore_in_flight=ignore_in_flight): return window_day diff --git a/tests/test_agent.py b/tests/test_agent.py index 009f330a..e9f61e18 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -7505,6 +7505,7 @@ def test_withheld_entries_make_every_due_dca_rule_a_skip(repo, monkeypatch): by_product = {s.product: s for s in result.skipped_buys} assert PRODUCT in by_product and by_product[PRODUCT].rule_id == ready_id assert "withheld" in by_product[PRODUCT].reason + assert by_product[PRODUCT].retry_until == "1970-01-12", "cadence 1: day 10's next window is 11" def test_a_stale_feed_makes_the_due_dca_rule_a_skip(repo, monkeypatch): @@ -7520,3 +7521,27 @@ def test_a_stale_feed_makes_the_due_dca_rule_a_skip(repo, monkeypatch): assert PRODUCT in result.stale_products, "fixture: the product is stale" [skip] = result.skipped_buys assert PRODUCT == skip.product and "stale" in skip.reason + # Judged by TODAY, not by the stale series: cycle day 45 buys bar 44, window 42, next 49. + assert skip.retry_until == "1970-02-19" + + +def test_a_stale_feed_whose_last_bar_window_already_filled_still_reports_the_current_window( + repo, monkeypatch +): + """The last stored bar (day 17) sits in window 14, which FILLED -- judged by that bar the rule + is not due and the skip would go silent for as long as the feed stays stale. The cycle on day + 22 is due for window 21, so it must report that, with window 21's next-window date.""" + _spy_post(monkeypatch) + _live_dca_rule(repo) + series = _dca_series(17) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + assert len(broker.place_calls) == 1, "fixture: window 14 filled" + + result = run_once(broker, repo, _config(), now_ts=22 * DAY + 3_600) + + assert PRODUCT in result.stale_products, "fixture: the product is stale" + [skip] = result.skipped_buys + assert "stale" in skip.reason + assert skip.retry_until == "1970-01-29", "window 21 -> the next window opens on day 28" From 924d4cfe9ccb27c724a334d890a0968527ff5d54 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 14:47:29 -0500 Subject: [PATCH 12/18] docs(rail-17): venue-read withdrawal capability, DCA catch-up, skip alerts (#1010) Co-Authored-By: Claude Opus 5.5 --- README.md | 4 +- docs/glossary.md | 10 +++ docs/go-live-runbook.md | 6 +- docs/operator-runbook.md | 62 +++++++--------- docs/rails/rail-17-withdrawal-attestation.md | 65 ----------------- docs/rails/rail-17-withdrawal-capability.md | 74 ++++++++++++++++++++ keel/commands/dca_plan.py | 9 +++ tests/commands/test_dca_plan.py | 12 ++++ 8 files changed, 134 insertions(+), 108 deletions(-) delete mode 100644 docs/rails/rail-17-withdrawal-attestation.md create mode 100644 docs/rails/rail-17-withdrawal-capability.md diff --git a/README.md b/README.md index 53b0389b..75e23354 100644 --- a/README.md +++ b/README.md @@ -200,9 +200,9 @@ applies the confirm/autonomy gate, then places and logs. There is deliberately n per-day spend caps, exposure and concentration caps, correlation-aware sizing, a minimum-move floor, no-martingale/no-stop-widening, a fails-closed kill-switch, total and weekly drawdown breakers, a consecutive-loss/edge-decay breaker, feed-staleness and - quote-balance checks, venue **subscription/withdrawal attestations** — rail 17 encodes + quote-balance checks, venue **subscription attestation**, and rail 17, which encodes §65.4 *qabd*: an asset that cannot be withdrawn may not have been validly possessed, so - withdrawal capability is attested and enforced, not assumed. A rail veto names itself and + every live buy reads the venue's own withdrawal flag and fails closed if it cannot. A rail veto names itself and the command that clears it. Beside the rails sits one routing-time check that needs the venue's own book, which a broker-less rail cannot see: the **max-spread entry gate** (#350) refuses a live BUY whose previewed `(best_ask − best_bid) / mid` is at or beyond diff --git a/docs/glossary.md b/docs/glossary.md index 7a9de52e..4a25dbbc 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -34,6 +34,16 @@ breakers, the allowlist, settlement-currency, spot-shape and trade-scope checks. Source: keel's own vocabulary -- docs/fiqh-basis.md's rails table (the prudential rails 2-14, 16, 20) plus its prose sections for rails 1, 17, 18 and 19 +## withdrawal capability + +Whether the account's wallets allow withdrawals right now, the test rail 17 applies on every live +BUY. keel reads it from the venue (Coinbase `allow_withdrawals` for the order's quote wallet and +the bought coin's wallet) rather than from an operator attestation; unreadable means veto. The +flag has only been observed `true`. `keel withdrawals attest --suspended` is a manual brake with +no expiry. + +Source: docs/rails/rail-17-withdrawal-capability.md + ## attestation market facts are computed, Shariah classifications are **ATTESTED, never inferred**. diff --git a/docs/go-live-runbook.md b/docs/go-live-runbook.md index 5623a169..ef857d49 100644 --- a/docs/go-live-runbook.md +++ b/docs/go-live-runbook.md @@ -70,12 +70,12 @@ unattested venue or asset is treated as unknown, not as fine. ```bash keel assets list # what is attested, and what is not keel subscription show # rail 14's spend allowance comes from this record -keel withdrawals show # rail 17: withdrawal capability is a compliance precondition +keel withdrawals show # rail 17: manual suspension + the venue's per-wallet answer keel scope show # rail 20: can this venue's credential place a live ENTRY? ``` Attest anything missing (`keel assets attest`, `keel subscription attest`, -`keel withdrawals attest`, `keel scope attest --trading`) before continuing. If a rail vetoes +`keel scope attest --trading`); rail 17 needs no attestation, it reads the venue on every live BUY before continuing. If a rail vetoes later, its message names the command that fixes it. Rail 20 exists because a credential that *reads* fine is not evidence it can *trade*: a @@ -235,7 +235,7 @@ unreadable state halts trading rather than permitting it. | `keel resume-entries` | a consecutive-loss halt (rail 16) | | `keel record-flow --amount ±N` | rebases rail 11's high-water mark | | `keel reset-hwm` | rail 11's high-water mark, clearing a stuck drawdown halt | -| `keel withdrawals attest --enabled` | rail 17's entry halt | +| `keel withdrawals attest --enabled` | rail 17's manual entry halt (set by `--suspended`; no expiry) | The de-risking direction of each is always allowed and needs no terminal: `keel kill`, `keel withdrawals attest --suspended`, `keel autonomy off`. diff --git a/docs/operator-runbook.md b/docs/operator-runbook.md index a5c4a22d..50f43fdb 100644 --- a/docs/operator-runbook.md +++ b/docs/operator-runbook.md @@ -19,8 +19,7 @@ Design home: `docs/superpowers/specs/2026-07-16-keel-broker-abstraction-design.m ## Pre-live checklist Run through this **before arming the agent for live trading**, and re-check after any change to the -Coinbase account. Item 3 is not a one-time check: it recurs **weekly** for as long as the agent -runs. +Coinbase account. Item 3 is not a checklist step: it says what to do when a DCA buy is skipped. ### 1. ⛔ Disable interest / rewards on idle balances — **required** @@ -70,39 +69,25 @@ A zakat estimate (~2.5% of holdings' market value per lunar year) is a **positiv item 1's prohibition, and it is informational: keel reports, you decide and discharge it. Tracked at KB §33.1. No pre-live action; noted here so the account-level obligation set is complete in one place. -### 3. Withdrawal-capability attestation — **weekly refresh** (rail 17) +### 3. A macOS notification says a DCA buy was skipped -**Why.** Rail 17 (§65.4 *qabd*) halts all BUY entries unless the withdrawal-capability attestation is -fresh, and "fresh" means a **7-day TTL** (`WITHDRAWAL_ATTESTATION_TTL_SEC`, -`keel/execution/executor.py`). An expired attestation reads as UNKNOWN, and rail 17 fails closed on -unknown — live DCA buys are vetoed. That is not hypothetical: on **2026-08-14** it vetoed the only -live DCA signal because the attestation had lapsed, and as of 2026-08-17 every deployment's -attestation was weeks stale, so rail 17 was halting entries on the **live** deployment (it is a -`LIVE_STATE` rail, skipped in paper, where a stale attestation matters only to the status display). -Each deployment carries its own attestation (they do not share a database), so the live one must be -refreshed for the rail and the paper ones to quiet their status lines. +keel itself posts a notification titled "keel-live: DCA buy skipped" when a live DCA buy was due +and did not place, at most once per rule per UTC day, repeating each day while it stays skipped. +Set `KEEL_DISABLE_MACOS_NOTIFY` to silence it. A DCA order stuck `pending` is reported daily too, +naming the order id. -Rail 17 fails closed by itself — what it cannot do is refresh its own input, and that input is -deliberately human (see the warning below). This entry is the cadence obligation the rail cannot -enforce, which is why it lives here rather than in `guards.py`. +**What to do.** Read the reason in the notification. Most reasons (a rail veto, a stale feed, a +cash shortfall) clear on their own. keel retries the buy on each later daily cycle until it fills +or the rule's next window opens, at most one fill per rule per window; a resting +(`pending`/`partially_filled`) DCA order blocks the retry until it fills or is cancelled. Catch-up +only covers windows whose buying cycle ran after the rule went live. `keel dca-plan` states that +catch-up can add up to one buy per rule to a calendar month; rail 14 still bounds the total. -**How to verify.** `keel status` prints the rail-17 line with days-to-expiry (`attested, expires in -3d`, `EXPIRED 12d ago`, or `never attested`) — staleness is visible there *before* it vetoes, not -only in the veto log. `keel withdrawals show` reads the same state with the age to one decimal. - -**Cadence.** Re-attest **weekly** — a calendar reminder is the intended mechanism. Confirm the -balances really are withdrawable on demand, then, per deployment: - -```bash -keel withdrawals attest --enabled -keel --config config.live-sandbox.yaml --db keel-live.db withdrawals attest --enabled -``` - -> ⚠️ **The typed confirmation is deliberately human.** `--enabled` RELEASES a rail-17 entry halt and -> demands a typed `yes` at a terminal — so that a scheduled job can never release a §65.4 halt, the -> same posture as `keel autonomy on`. Do not script this command and do not pipe a `yes` into it: -> the weekly habit is the fix for staleness, and automating the release would undo the rail. If a -> calendar reminder ever feels like it should be a cron job, re-read this warning. +Rail 17 (withdrawal capability) needs no weekly chore: it reads the venue's `allow_withdrawals` +flag for the order's wallets on every live BUY (`docs/rails/rail-17-withdrawal-capability.md`). +`keel withdrawals show` prints the manual suspension and the venue's per-wallet answer; +`keel withdrawals attest --suspended` is a manual brake with no expiry, and `--enabled` clears it +(typed `yes` at a terminal). --- @@ -723,11 +708,12 @@ candidates carries the purification obligation in their own records. an ACATS transfer to another broker. It is attested like any venue: ```bash -keel --config config.paper-equities.yaml --db keel-equities.db withdrawals attest --enabled +keel --config config.paper-equities.yaml --db keel-equities.db withdrawals show ``` -Rail 17 is a live-state rail (skipped in paper), so this is recorded for the day live is ever -considered, and it lapses weekly like every deployment's attestation. +Rail 17 is a live-state rail (skipped in paper). It reads the venue's own withdrawal flag, not an +attestation, so there is nothing to record or renew; the venue read for an equities broker is +whatever that adapter supports, and a broker without it fails closed on a live BUY. ### T+1 settlement × daily cadence @@ -1110,7 +1096,7 @@ no handler attached at all: an offline install makes zero network calls. This ch unchanged by the notification layer below and stays independent of it. **Opt-in event notifications (default OFF).** The events an operator most needs are silent -precisely because they are not errors — rail 17's attestation nearing expiry fails CLOSED and +precisely because they are not errors — a lapsed attestation fails CLOSED and quietly vetoed a real setup for weeks before anyone opened the TUI, the console of the day (#541). The notification layer (#444) delivers those over the SAME generic webhook, per-event opt-in in Freqtrade's `notification_settings` shape: @@ -1121,7 +1107,7 @@ quietly vetoed a real setup for weeks before anyone opened the TUI, the console notifications: format: plain # plain = generic JSON (default); slack = {"text": ...} chat payload events: - attestation.expiring: true + attestation.expiring: true # rail 22 (cash posture) attestation only rail.armed: true setup.unplaced: true allowance.nearing_exhaustion: true @@ -1135,7 +1121,7 @@ doctor's own findings, so the alert and the diagnostic can never disagree): | Event key | Fires when | Why it is not a CRITICAL log | |---|---|---| -| `attestation.expiring` | rail-17 withdrawal attestation has ≤2 of its 7 TTL days left (or has expired, or was never attested) | an expired attestation silently vetoes every entry — cycles keep running, nothing errors | +| `attestation.expiring` | the rail-22 cash-posture attestation is near expiry, expired, or never made (rail 17 no longer has an attestation or an expiry) | an expired attestation silently vetoes every entry — cycles keep running, nothing errors | | `rail.armed` | rail 16's consecutive-loss halt arms, or rail 11's drawdown reaches 20% | a halt is a correct state, not a fault; the kill switch is deliberately absent (you engaged it at a terminal — you know) | | `setup.unplaced` | a cycle detected an entry setup and could not place it | the veto is a WARNING; the rail-17 incident looked like a quiet week | | `allowance.nearing_exhaustion` | month-to-date BUY spend reaches 80% of the in-force rail-14 allowance — or there is spend against an allowance of 0 (no subscription in force: lapsed or never attested) | rail 14 only speaks when it vetoes, which is too late to re-tier | diff --git a/docs/rails/rail-17-withdrawal-attestation.md b/docs/rails/rail-17-withdrawal-attestation.md deleted file mode 100644 index 8324158c..00000000 --- a/docs/rails/rail-17-withdrawal-attestation.md +++ /dev/null @@ -1,65 +0,0 @@ -# Rail 17 — withdrawal capability (the `qabd` rail) - -Rail 17 is one of keel's hard rails: checks in `keel/execution/guards.py` that run before every -order, in every mode, and cannot be switched off or widened from `config.yaml`. This one blocks -new **BUY** entries unless a *fresh attestation* says the account's balances can be withdrawn on -demand right now. - -## What the rail permits - -A BUY (a new entry, DCA buys included) passes rail 17 only when the operator has **recently -attested** that the account is in a withdrawable state. Everything else is vetoed: - -- **No attestation at all, or one older than 7 days** — the rail reads the state as `UNKNOWN` - and vetoes the BUY. An unverified state is not evidence that possession holds. -- **Attested `--suspended`** — withdrawals are known to be restricted, so new entries are - halted. -- **SELLs are never blocked.** Existing holdings are already owned; forcing a sale to "fix" a - withdrawal freeze would be strictly worse than holding through it. Exits, stop-outs and - protective orders are deliberately unaffected. - -In paper trading the rail is *skipped* (it describes the real account, which a paper rehearsal -cannot see) — but it is recorded as skipped, never silently omitted, so a paper track record is -honest about its own gaps. - -## Why it exists - -The fiqh basis is KB §65.4, Ayub's constructive-possession test (*qabd*): possession of a bought -asset holds only while *"there is nothing to prevent the buyer from taking physical possession -whenever he desires."* An asset sitting in an account that cannot be withdrawn from may not have -been validly possessed at all — so **acquiring more of it** is the thing to stop. The rail turns -that into a mechanical rule: funds enter the account's holdings only under a deliberate, -recently-renewed human act, never on an old assertion. - -Because the attestation is about the account's *current* state, and a withdrawal freeze can -appear at any time, the TTL is deliberately short: **7 days** -(`WITHDRAWAL_ATTESTATION_TTL_SEC` in `keel/execution/executor.py`). A stale attestation is -treated as no better than none. - -## What happens when it expires - -Nothing dramatic and nothing automatic — the rail simply starts vetoing. An expired attestation -resolves to `UNKNOWN` (distinct from an attested suspension: "nobody has checked recently" is a -different claim than "the broker says withdrawals are restricted"), and the rail **fails closed** -on unknown. On 2026-08-14 this was not hypothetical: a lapsed attestation vetoed the only live -DCA signal of the day. `keel status` shows the freshness line with days-to-expiry, so staleness -is visible *before* it vetoes. - -## Renewing it - -Confirm in the venue's own app that the BTC/ETH/PAXG/USDC balances really are withdrawable on -demand, then, per deployment (each has its own database and its own attestation): - -```bash -keel withdrawals attest --enabled -keel --config config.live-sandbox.yaml --db keel-live.db withdrawals attest --enabled -``` - -`--enabled` **releases an entry halt**, so it demands a typed `yes` at a terminal — a cron line -must not be able to clear a rail-17 halt and let the next cycle place live orders with no human -in the loop. `--suspended` only ever reduces capability and is ungated. To inspect the current -state: `keel withdrawals show`, or the rail-17 line in `keel status`. - -The intended cadence is **weekly** — see item 3 of `docs/operator-runbook.md`. The rail fails -closed by itself; what it cannot do is refresh its own input, and that input is deliberately -human. diff --git a/docs/rails/rail-17-withdrawal-capability.md b/docs/rails/rail-17-withdrawal-capability.md new file mode 100644 index 00000000..51af366d --- /dev/null +++ b/docs/rails/rail-17-withdrawal-capability.md @@ -0,0 +1,74 @@ +# Rail 17 — withdrawal capability (the `qabd` rail) + +Rail 17 is one of keel's hard rails: checks in `keel/execution/guards.py` that run before every +order, in every mode, and cannot be switched off or widened from `config.yaml`. This one blocks +new **BUY** entries unless the venue itself says the account's balances can be withdrawn right +now. + +## What the rail reads + +On every live BUY the executor asks Coinbase (`GET /v2/accounts`, the `allow_withdrawals` flag) +about two wallets: the order's **quote wallet** (the cash being spent) and the **wallet of the coin +being bought**. The existing API key is view + trade with `can_transfer: false`, so this is a +read the key can already make; nothing new is granted. + +- **Both wallets allow withdrawals** — the BUY passes. +- **Either wallet has withdrawals off** — the BUY is vetoed, and the veto names the wallet. +- **The coin's wallet does not exist yet** (first purchase of that coin) — the decision rests on + the quote wallet alone. +- **The venue cannot be read, the quote wallet is missing, the broker adapter has no such method, + or the manual-suspension store cannot be read** — the BUY is vetoed. An unverified state is not + evidence that possession holds, so the rail fails closed. +- **SELLs are never blocked.** Existing holdings are already owned; forcing a sale to "fix" a + withdrawal freeze would be strictly worse than holding through it. Exits, stop-outs and + protective orders are unaffected. + +In paper trading the rail is *skipped* (it describes the real account, which a paper rehearsal +cannot see) — recorded as skipped, never silently omitted. SELLs and paper never read the venue. + +## What has and has not been observed + +`allow_withdrawals` has only ever been seen `true`. It has **not** been observed flipping to +`false` during a real withdrawal freeze, so it is not known that Coinbase sets it in that +situation. The rail is correct if it does and silent if it does not. That is why the manual +brake below exists, and why this page does not claim the venue read is a complete detector. + +## The manual brake + +`keel withdrawals attest --suspended` halts all new entries whatever the venue says. It has **no +expiry**: it stays until someone clears it. `keel withdrawals attest --enabled` clears it, and +still demands a typed `yes` at a terminal — a scheduled job must not be able to release a +rail-17 halt and let the next cycle place live orders with no human in the loop. `--suspended` +only ever reduces capability and is ungated. + +Per deployment (each has its own database): + +```bash +keel withdrawals attest --suspended +keel --config config.live-sandbox.yaml --db keel-live.db withdrawals attest --enabled +``` + +`keel withdrawals show` prints the manual suspension state and, for each wallet, the venue's +current answer. If the venue cannot be reached it says so and the suspension line is still shown. + +## What was removed + +Rail 17 used to rest on an operator attestation with a 7-day TTL. A lapsed attestation read as +unknown and vetoed BUYs; on 2026-08-14 that vetoed the only live DCA signal of the day. The TTL +(`WITHDRAWAL_ATTESTATION_TTL_SEC`), the weekly renewal chore and every expiry warning are gone. +Rail 22 (cash posture) and rail 14 (subscription) keep their attestations, unchanged. + +## Why it exists + +The fiqh basis is KB §65.4, Ayub's constructive-possession test (*qabd*): possession of a bought +asset holds only while *"there is nothing to prevent the buyer from taking physical possession +whenever he desires."* An asset sitting in an account that cannot be withdrawn from may not have +been validly possessed at all — so **acquiring more of it** is the thing to stop. See +`docs/fiqh-basis.md`. + +## A skipped buy is retried + +A rail-17 veto skips that day's DCA buy. DCA catch-up retries a buy that was due and did not fill +on each later daily cycle until it fills or the rule's next window opens, so a short freeze does +not cost the window. A skipped live DCA buy also posts a macOS notification ("keel-live: DCA buy +skipped") with the reason; see `docs/operator-runbook.md`. diff --git a/keel/commands/dca_plan.py b/keel/commands/dca_plan.py index 52c2c88e..4cae88c3 100644 --- a/keel/commands/dca_plan.py +++ b/keel/commands/dca_plan.py @@ -1156,6 +1156,14 @@ def build_dca_plan( "order pays the venue's fee." ) +#: A buy that was due and did not fill is retried on later daily cycles until the rule's next +#: window opens, so a month can carry one more fill per rule than the schedule below counts. +#: Rail 14's monthly cap still bounds the total. +CATCHUP_NOTE = ( + "Catch-up can add up to one buy per rule to a calendar month (a missed buy is retried " + "until the next window); rail 14's monthly cap still bounds the total." +) + def worst_month_text(plan: DcaPlan) -> str: """What rail 14's cap was checked against (R6, amended #847), as ONE sentence both @@ -1184,6 +1192,7 @@ def render_dca_plan(plan: DcaPlan) -> list[str]: # a total recomputed (and possibly larger) than what the blocker check actually used. f" checked against the cap: {worst_month_text(plan)}", f" {RAIL14_NOTE}", + f" {CATCHUP_NOTE}", "", "== Schedule ==", f" {'asset':<8} {'weight':>7} {'cadence':<14} {'per buy':>10} {'monthly':>10} " diff --git a/tests/commands/test_dca_plan.py b/tests/commands/test_dca_plan.py index ca1566ef..8c777fee 100644 --- a/tests/commands/test_dca_plan.py +++ b/tests/commands/test_dca_plan.py @@ -19,6 +19,7 @@ from keel.commands import dca_plan as dca_mod from keel.commands.dca_plan import ( + CATCHUP_NOTE, RAIL14_NOTE, DcaPlanError, DcaPlanRefused, @@ -1553,3 +1554,14 @@ def flaky(*args, **kwargs): # Non-vacuous: the patch reached the two calls `apply_dca_plan`'s write loop made before # the refusal (one delegated to the real writer, one raised). assert calls["n"] == 2 + + +def test_the_rendered_plan_says_catch_up_can_add_a_buy_and_rail_14_still_bounds_it( + valid_config_path: Path, +) -> None: + lines = render_dca_plan(_plan(valid_config_path)) + assert sum(CATCHUP_NOTE in line for line in lines) == 1 + assert "up to one buy per rule" in CATCHUP_NOTE + assert "rail 14" in CATCHUP_NOTE.lower() + # It sits in the cap block, before the schedule, next to the rail 14 note. + assert lines.index(f" {CATCHUP_NOTE}") < lines.index("== Schedule ==") From e6bf1dfecd03563cde7dd62e6f0289467886e30d Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 15:10:29 -0500 Subject: [PATCH 13/18] fix(dca): a stuck pending buy has a way out; blocked skips say blocked (#1010) F1: `keel orders cancel N` marks a live entry left `pending` with no venue order id canceled LOCALLY after a typed `yes` at a terminal; the prompt says to check the Coinbase app first, the broker is never built, and the row's raw_response records why under `local_cancel`. A row with a venue id keeps the venue path unchanged. Registered in keel.capabilities. F4: an in-flight skip names that command and says "Blocked until that order is resolved" instead of a false "Retrying daily until". F2: an in-flight DCA BUY counts as a DCA day for the same-day sleeve-sell exclusion. F3: catch-up state is read after reconcile, so a row reconcile settles this cycle is not counted in flight. F11: the runbook names what clears each kind of skip. Co-Authored-By: Claude Opus 5.5 --- docs/operator-runbook.md | 28 +++-- keel/agent.py | 53 +++++++-- keel/capabilities.py | 15 ++- keel/commands/orders.py | 81 ++++++++++++- keel/web/__init__.py | 2 +- keel/web/gates.py | 2 +- keel/web/security.py | 4 +- keel/web/server.py | 6 +- keel/web/static/js/api.js | 2 +- keel/web/static/js/render.js | 2 +- tests/commands/test_orders_cancel.py | 95 +++++++++++++++ tests/test_agent.py | 157 ++++++++++++++++++++++++- tests/test_capabilities.py | 2 +- tests/test_runbook_console.py | 15 ++- tests/web/test_web_prose_stays_true.py | 2 +- 15 files changed, 430 insertions(+), 36 deletions(-) diff --git a/docs/operator-runbook.md b/docs/operator-runbook.md index 50f43fdb..1ebdfd23 100644 --- a/docs/operator-runbook.md +++ b/docs/operator-runbook.md @@ -76,11 +76,24 @@ and did not place, at most once per rule per UTC day, repeating each day while i Set `KEEL_DISABLE_MACOS_NOTIFY` to silence it. A DCA order stuck `pending` is reported daily too, naming the order id. -**What to do.** Read the reason in the notification. Most reasons (a rail veto, a stale feed, a -cash shortfall) clear on their own. keel retries the buy on each later daily cycle until it fills -or the rule's next window opens, at most one fill per rule per window; a resting -(`pending`/`partially_filled`) DCA order blocks the retry until it fills or is cancelled. Catch-up -only covers windows whose buying cycle ran after the rule went live. `keel dca-plan` states that +**What to do.** Read the reason in the notification; what clears it depends on the reason. + +- **A stale feed or a cash shortfall** clears on its own once the feed catches up or the cash + arrives. keel retries the buy on each later daily cycle until it fills or the rule's next window + opens ("Retrying daily until ..."), at most one fill per rule per window. +- **A rail veto** names the rail and what clears it, verbatim; follow that. Some rails clear on + their own (an unreadable venue on rail 17), others need you (an attestation, a halt release). +- **A stuck order** ("previous DCA order #N is still pending ... Blocked until that order is + resolved") does NOT retry: a resting (`pending`/`partially_filled`) DCA order blocks the rule + until it is resolved. Reconcile settles one the venue gave an id on its own cycle. One the venue + never gave an id (the placement raised, e.g. a timeout) is settled by nothing, so run + `keel orders cancel N` at a terminal: first check in the Coinbase app that no such order exists + and nothing filled, then type `yes`. keel marks the row canceled locally without contacting the + venue, and the rule catches up on the next cycle. +- **A manual suspension** (rail 17's `--suspended` switch) stays until you clear it with + `keel withdrawals attest --enabled`. + +Catch-up only covers windows whose buying cycle ran after the rule went live. `keel dca-plan` states that catch-up can add up to one buy per rule to a calendar month; rail 14 still bounds the total. Rail 17 (withdrawal capability) needs no weekly chore: it reads the venue's `allow_withdrawals` @@ -1021,8 +1034,9 @@ covers the rest from a plain shell. **What the browser cannot do, and that is deliberate.** Every capability-increasing action -- arming autonomy, releasing the kill-switch, clearing a consecutive-loss halt, re-seeding the drawdown high-water mark, declaring a deposit or withdrawal, attesting withdrawal capability, scope -or posture, declaring an out-of-band close, and replacing the binary -- is a CLI command behind a -typed confirmation at an interactive terminal. `keel capabilities` lists all ten with the gate +or posture, declaring an out-of-band close, cancelling a stuck order the venue never gave an id, +and replacing the binary -- is a CLI command behind a typed confirmation at an interactive +terminal. `keel capabilities` lists all eleven with the gate covering each. The one exception is the console's HALT button, which engages the kill-switch (stopping is never gated). The server's gated routes under `/api/gates/` can also release the kill-switch and clear the consecutive-loss halt, but only diff --git a/keel/agent.py b/keel/agent.py index 302af50a..d97113e6 100644 --- a/keel/agent.py +++ b/keel/agent.py @@ -1321,7 +1321,7 @@ def _skipped_dca_buys( - the whole cycle's entries were withheld (`entries_allowed` is false -- some product's gating bar was not ready), so it was never evaluated; - a resting BUY of the rule blocks it (`DcaCatchUp.in_flight`): reported with the local order - id and what to do, because nothing else will ever say so. + id and the command that resolves it (`blocked_by`), because nothing else will ever say so. The caller runs this on a LIVE cycle only. A rule with no `rules.id` is skipped: the notification is per rule. @@ -1337,6 +1337,7 @@ def _skipped_dca_buys( if not isinstance(rule, Dca) or rule.rule_id is None: continue cadence_days = int(rule.params["cadence_days"]) + blocked_by: tuple[int, ...] = () signal = signals_by_rule.get(rule.rule_id) window_day: int | None reason: str | None @@ -1360,10 +1361,16 @@ def _skipped_dca_buys( lagging = ", ".join(sorted({f"{b.product} ({b.reason})" for b in blocked_entries})) reason = f"entries withheld this cycle (gating bar not ready: {lagging})" elif rule.catch_up is not None and rule.catch_up.in_flight: - ids = ", ".join(f"#{i}" for i in rule.catch_up.resting_order_ids) + blocked_by = rule.catch_up.resting_order_ids + ids = ", ".join(f"#{i}" for i in blocked_by) + commands = "; ".join(f"`keel orders cancel {i}`" for i in blocked_by) + # The one command that resolves it either way (#1010 F1): a row with a venue id is + # cancelled AT the venue, one with none -- a placement that raised -- is marked + # canceled locally after a typed `yes`. Reconcile settles the first kind on its + # own; nothing settles the second. reason = ( - f"previous DCA order {ids} still pending at the venue; " - "resolve it (reconcile or cancel) to resume buying" + f"previous DCA order {ids} is still pending; nothing is bought until it is " + f"resolved ({commands})" ) else: continue # due, not blocked, no signal: nothing this function can name @@ -1375,6 +1382,7 @@ def _skipped_dca_buys( retry_until=( None if window_day is None else _epoch_day_date(window_day + cadence_days) ), + blocked_by=blocked_by, ) ) return skipped @@ -1404,7 +1412,13 @@ def _post_skipped_buys( if repo.get_state(key) == today: continue message = f"{skip.product} (rule {skip.rule_id}): {skip.reason[:_SKIP_REASON_MAX_CHARS]}." - if skip.retry_until is not None: + if skip.blocked_by: + # Blocked, not retrying: the rule buys nothing until the order is resolved, so a + # "Retrying daily" promise would be false (#1010 F4). Outside the reason's cut, so + # the command always reaches the banner. + commands = "; ".join(f"keel orders cancel {i}" for i in skip.blocked_by) + message += f" Blocked until that order is resolved: {commands}." + elif skip.retry_until is not None: message += f" Retrying daily until {skip.retry_until}." if post(SKIPPED_BUY_TITLE, message): repo.set_state(key, today) @@ -1428,11 +1442,15 @@ def _dca_fires_today( the buy was vetoed or withheld (the day is still a buy day, and a sale beside it would be the round trip the exclusion exists to prevent); the ledger answers it for a buy that went out under a rule this cycle did not load. A DCA whose `detect` RAISES counts as firing -- the - refusal is the direction that costs nothing. + refusal is the direction that costs nothing. So does one with a BUY still in flight + (`DcaCatchUp.in_flight`, #1010): `detect` answers `None` for it, but a sale beside a buy + that may yet fill is the same round trip. """ for rule in product_rules: if not isinstance(rule, Dca): continue + if rule.catch_up is not None and rule.catch_up.in_flight: + return True try: if rule.detect(candles_by_tf) is not None: return True @@ -2380,6 +2398,11 @@ class SkippedBuy: product: str reason: str retry_until: str | None = None + #: The local `orders.id`s of a resting BUY blocking the rule (`DcaCatchUp.in_flight`). A + #: blocked rule does NOT retry daily -- it buys nothing until the order is resolved -- so the + #: banner says that instead of `retry_until`'s "Retrying daily until" (#1010 F4). `()` for + #: every other skip. + blocked_by: tuple[int, ...] = () @dataclass(frozen=True) @@ -2619,13 +2642,6 @@ def run_once( # real executor `mode`; only what we surface to the user changes. reported_mode = "paper" if paper_trader is not None else mode log_event(logger, logging.INFO, "agent.mode_resolved", mode=reported_mode) - # DCA catch-up (#1010): each `Dca` reads THIS cycle's ledger -- the `orders.mode` the - # cycle writes: "paper" on the paper path, and "live" for BOTH executor modes - # (`executor._order_row`), never `mode`/`reported_mode`'s "confirm"/"autonomous" -- before - # `engine.evaluate` and `_dca_fires_today` (sleeve reductions) call its `detect`. - _attach_dca_catch_up( - repo, rules, rule_rows, mode="paper" if paper_trader is not None else "live" - ) max_age_sec = config.auto_trade.interval_sec * FEED_STALENESS_CYCLES finest = _finest_granularity(granularities) @@ -2661,6 +2677,17 @@ def run_once( # already resting. Neither subsumes the other.) reconcile.sweep_orphan_brackets(broker, repo, now_ts) + # DCA catch-up (#1010): each `Dca` reads THIS cycle's ledger -- the `orders.mode` the + # cycle writes: "paper" on the paper path, and "live" for BOTH executor modes + # (`executor._order_row`), never `mode`/`reported_mode`'s "confirm"/"autonomous" -- before + # `engine.evaluate` and `_dca_fires_today` (sleeve reductions) call its `detect`. AFTER + # the three reconcile passes above, which never call `detect`: a BUY that filled since the + # last cycle is booked there, and read before it the settled row would count as in flight + # and block (and report) a buy that is free to go out. + _attach_dca_catch_up( + repo, rules, rule_rows, mode="paper" if paper_trader is not None else "live" + ) + # Rail 11's inputs, refreshed BEFORE any entry this cycle. `poll_once` above has already # written the candles, so every product's latest price is readable here -- and it has to # be done now, not after the loop: `guards.check` reads these scalars from inside diff --git a/keel/capabilities.py b/keel/capabilities.py index 0e71d94c..3c7aef7b 100644 --- a/keel/capabilities.py +++ b/keel/capabilities.py @@ -21,7 +21,7 @@ and those four rows went with their call sites. `mirrors` stays on the dataclass: it describes a shape this registry must be able to express the moment a second front-end gates anything again. -**Eight of the ten capability increases in this build are a CLI command run by a person at a +**Nine of the eleven capability increases in this build are a CLI command run by a person at a terminal, and the browser can perform none of those.** That is not a property of what the client draws -- "a client that hides a button is not a gate" -- it is a property of the server, which implements no verb that would reach one, asserted by scans over `keel/web/`'s own source. @@ -98,7 +98,7 @@ class Gate: #: **`BROWSER` is a SECOND gate and never a seam in the first.** `_is_interactive` is untouched, #: still has no env-var or flag override, and every CLI path still requires a real terminal. Two #: kinds of evidence for one fact -- a human, present, who meant this -- and the browser's kind -#: is admitted for two actions out of ten. +#: is admitted for two actions out of eleven. GATES: tuple[Gate, ...] = (TTY, BROWSER) @@ -229,6 +229,17 @@ def key(self) -> tuple[str, str]: "made, and a wrong `--price` books a wrong P&L row into rail 16's streak" ), ), + Capability( + module="keel.commands.orders", + function="_cancel_locally", + surface="cli", + invocation="keel orders cancel (a live entry left pending with no venue order id)", + increases=( + "a live BUY row the venue never gave an id is marked canceled without asking the " + "venue, so the DCA rule it was blocking buys again on the next cycle. Gated because " + "the venue cannot be consulted: if that order did in fact fill, the rule buys twice" + ), + ), Capability( module="keel.commands.update", function="typed_update_gate", diff --git a/keel/commands/orders.py b/keel/commands/orders.py index 2ea23358..a5842770 100644 --- a/keel/commands/orders.py +++ b/keel/commands/orders.py @@ -57,7 +57,7 @@ from typing import Any import click -from keel_core.telemetry import log_exception +from keel_core.telemetry import log_event, log_exception from keel.commands._common import _open_repo_ro, with_disclaimer from keel.data.repository import Repository @@ -1115,6 +1115,77 @@ def orders_list(ctx: click.Context, scope: str, limit: int) -> None: CANCEL_EXIT_PHRASE = "remove protection from order {order_id}" +#: The typed gate's wording for the LOCAL-ONLY cancel (#1010): a live entry left `pending` with +#: no venue order id. Module-level so the docs and the in-flight skip banner name one command. +LOCAL_CANCEL_ACTION = "mark order {order_id} canceled locally, without asking the venue" +LOCAL_CANCEL_DETAIL = ( + "Order {order_id} has no venue order id, so keel cannot ask the venue about it. First check " + "in the Coinbase app that no such order exists and nothing filled -- if it did fill, do not " + "cancel it here. Marking it canceled lets its DCA rule buy again on the next cycle." +) +#: What the row's `raw_response` records about why it is canceled. Under `local_cancel`, never +#: `order_id`: `executor._native_order_id` reads that key alone, so the note can never be +#: mistaken for an id the venue issued. +LOCAL_CANCEL_NOTE = ( + "canceled locally by the operator via `keel orders cancel`: the placement left no venue " + "order id, and the operator confirmed at a terminal that the venue holds no such order" +) + + +def _is_local_only_cancel(decision: CancelDecision, row: dict[str, Any]) -> bool: + """A live ENTRY still `pending` that the venue never gave an id (#1010). + + `executor._run_order` inserts the row BEFORE `broker.place_order`, so a placement that raises + (a timeout) leaves it here: `reconcile` cannot ask about it and `_cancel_at_exchange` refuses + it, and a resting BUY blocks its DCA rule (`DcaCatchUp.in_flight`) for good. ENTRIES ONLY: an + exit or a protective leg marked canceled locally would have reconcile re-place protection + over it, so those keep the venue path and its named refusal. `pending` only: a + `partially_filled` row is one the venue answered about, so it has an id to cancel by. + """ + from keel.execution.executor import _native_order_id + + return ( + decision.kind == "entry" + and row.get("mode") == "live" + and row.get("status") == "pending" + and _native_order_id(row) is None + ) + + +def _cancel_locally(repo: Repository, order_id: int) -> None: + """Mark a local-only `pending` entry canceled without touching the venue, after a typed `yes`. + + The one place this command writes a terminal status the venue did not confirm, so it takes + the heavier gate (`_require_interactive_confirmation`, the full word `yes`, a terminal), and + the detail tells the operator to settle the question the venue cannot be asked first. RE-READ + after the prompt, like the venue path: a row that gained an id or moved while the operator + answered is a refusal, not a proceed. No broker is built. + """ + from keel.commands._common import _require_interactive_confirmation + + _require_interactive_confirmation( + LOCAL_CANCEL_ACTION.format(order_id=order_id), + LOCAL_CANCEL_DETAIL.format(order_id=order_id), + ) + row = repo.get_order(order_id) + if row is None: + raise click.ClickException(f"order {order_id} vanished from the book while you answered.") + fresh = classify_cancel(repo, order_id, row=row) + if not fresh.cancellable or not _is_local_only_cancel(fresh, row): + raise click.ClickException( + f"not cancelling order {order_id}: it changed while you were answering." + ) + now_ts = int(time.time()) + repo.update_order( + order_id, + status="canceled", + raw_response=json.dumps({"local_cancel": LOCAL_CANCEL_NOTE}), + updated_at=now_ts, + ) + log_event(logger, logging.WARNING, "orders.cancelled_locally", order_id=order_id) + click.echo(f"marked order {order_id} canceled locally; the venue was not contacted.") + + @orders_cmd.command("cancel") @click.argument("order_id", type=int) @click.pass_context @@ -1133,6 +1204,10 @@ def orders_cancel(ctx: click.Context, order_id: int) -> None: Needs a terminal either way. The venue is reached only after the confirmation, and only the venue's own confirmation lets the local row be marked -- `executor._cancel_at_exchange` owns that rule and this calls it rather than restating it. + + ONE EXCEPTION, and it is gated harder rather than softer: a live entry still `pending` with + no venue order id (`_is_local_only_cancel`) has nothing the venue can be asked about, so it + is marked canceled locally after a typed `yes` (`_cancel_locally`) and no broker is built. """ from keel.commands._common import _build_broker, _is_interactive, _load_cfg, _open_repo from keel.execution.executor import CancelPending, CancelUnavailable, _cancel_at_exchange @@ -1150,6 +1225,10 @@ def orders_cancel(ctx: click.Context, order_id: int) -> None: raise click.ClickException(decision.reason) click.echo(f"order {order_id}: {decision.kind} on {decision.product_id}") + row = repo.get_order(order_id) + if row is not None and _is_local_only_cancel(decision, row): + _cancel_locally(repo, order_id) + return if decision.typed: phrase = CANCEL_EXIT_PHRASE.format(order_id=order_id) click.echo( diff --git a/keel/web/__init__.py b/keel/web/__init__.py index 1595a94f..d67e655b 100644 --- a/keel/web/__init__.py +++ b/keel/web/__init__.py @@ -20,7 +20,7 @@ machine with no terminal can create a deployment, and `/api/gates/*` through the gate table (the halt and the two releases below). "No POST at all" was a clean property that was also satisfied by a server which could not set anything up. -* **Eight of the ten capability-increasing actions in `keel/capabilities.py` are unreachable +* **Nine of the eleven capability-increasing actions in `keel/capabilities.py` are unreachable from this package, and the other two are reachable only as the gate table declares them**, asserted by tests that scan this source rather than by inspection. The server cannot arm or spend, and it can release exactly two halts -- through `gates.TIER1_ACTIONS`, which holds the diff --git a/keel/web/gates.py b/keel/web/gates.py index 7748eec0..e0ff7984 100644 --- a/keel/web/gates.py +++ b/keel/web/gates.py @@ -12,7 +12,7 @@ **Not a bypass, and not a seam in the TTY gate.** `_is_interactive` is untouched, has no env-var or flag override, and the CLI path still requires a real terminal. This is a SECOND kind of evidence for the same fact -- a human, present, who typed something a script would not. The other -eight are refused here as firmly as they are refused to cron. +nine are refused here as firmly as they are refused to cron. **Not the whole gate either.** The typed phrase proves intent, not locality, and on its own it is the weakest of the seven checks a release must pass. `security.gated_action_permitted` decides diff --git a/keel/web/security.py b/keel/web/security.py index c765545f..ad4cf6e5 100644 --- a/keel/web/security.py +++ b/keel/web/security.py @@ -38,8 +38,8 @@ machine with no terminal has to be able to create a deployment somehow), and it routes through `keel.commands.setup.ACTIONS`, every member of which is a step declared `MECHANICAL`, is idempotent and is never destructive, or through the gate table (`/api/gates/*`: the halt and - the two Tier 1 releases, see `gates_token` and `halt_token` below). Of the ten - capability-increasing actions in `keel/capabilities.py`, eight are unreachable and the other + the two Tier 1 releases, see `gates_token` and `halt_token` below). Of the eleven + capability-increasing actions in `keel/capabilities.py`, nine are unreachable and the other two are reachable only as that table declares them, and a test asserts it. That is a narrower guarantee than "no POST at all" was, and a more useful one: "no POST" would have been satisfied by a server that could not set anything up, while this is satisfied only by one diff --git a/keel/web/server.py b/keel/web/server.py index ea155aa4..e6f92d1c 100644 --- a/keel/web/server.py +++ b/keel/web/server.py @@ -28,8 +28,8 @@ switch, ungated on purpose -- refusing a stop is the one failure it must not have) and the gated halt releases in `gates.TIER1_ACTIONS` (`resume`, `resume-entries`), which demand a loopback session on a loopback bind, the gate token (which no endpoint serves, #968) and a typed phrase; -the halt takes a halt token of its own. The guarantee is that **eight of the -ten capability-increasing actions in `keel/capabilities.py` are unreachable from this package and +the halt takes a halt token of its own. The guarantee is that **nine of the +eleven capability-increasing actions in `keel/capabilities.py` are unreachable from this package and the other two only as that gate table declares them**, asserted by a test that scans this source rather than by inspection. "No POST" said the server could not write; this says it cannot arm, attest or spend -- which is the property anyone actually cares about. @@ -243,7 +243,7 @@ def ensure_schema(db_path: str) -> None: #: #: The action SET is unchanged by all of this, and that is the invariant that matters more than the #: path: `keel.commands.setup.ACTIONS` still contains only idempotent, non-destructive, -#: `MECHANICAL` steps, and a test still asserts it is disjoint from the ten capability- +#: `MECHANICAL` steps, and a test still asserts it is disjoint from the eleven capability- #: increasing actions in `keel/capabilities.py`. A browser can set a deployment up. It still cannot #: arm a rule, attest an asset or enable autonomy. API_SETUP_PREFIX = "/api/setup/" diff --git a/keel/web/static/js/api.js b/keel/web/static/js/api.js index 410c787b..72664e0e 100644 --- a/keel/web/static/js/api.js +++ b/keel/web/static/js/api.js @@ -295,7 +295,7 @@ function isField(value) { * ── WHAT MAKES THIS SAFE IS NOT IN THIS FILE ──────────────────────────────────────────────── * `key` reaches the server as a path segment and is looked up in `keel.commands.setup.ACTIONS`, * a closed registry of idempotent, non-destructive, `MECHANICAL` steps that a test asserts is - * disjoint from the ten capability-increasing actions. Nothing this function is called with + * disjoint from the eleven capability-increasing actions. Nothing this function is called with * can widen that: an unknown key is a 404, and a known one runs the same step `keel setup` runs. * The client cannot arm a rule, attest an asset or enable autonomy, and it is the SERVER that * makes that true. diff --git a/keel/web/static/js/render.js b/keel/web/static/js/render.js index 1dbce885..5b3fb5ac 100644 --- a/keel/web/static/js/render.js +++ b/keel/web/static/js/render.js @@ -1545,7 +1545,7 @@ function stepCard(step, actions, notAutomated) { * * **The gate did not move with the button, and that is the property to keep in view.** What may be * performed here is `keel.commands.setup.ACTIONS` and nothing else: idempotent, non-destructive, - * `MECHANICAL` steps, asserted disjoint from the ten capability-increasing actions. A client + * `MECHANICAL` steps, asserted disjoint from the eleven capability-increasing actions. A client * that hides a button is not a gate, so nothing is hidden -- and nothing needed to be, because the * server refuses what is not in that set regardless of what this file draws. * diff --git a/tests/commands/test_orders_cancel.py b/tests/commands/test_orders_cancel.py index 490fded2..1cc756ba 100644 --- a/tests/commands/test_orders_cancel.py +++ b/tests/commands/test_orders_cancel.py @@ -668,3 +668,98 @@ def test_a_venue_that_refuses_the_cancel_is_a_message_not_a_traceback( conn = connect(str(db_path)) migrate(conn) assert Repository(conn).get_order(entry)["status"] == "pending" + + +# -- a pending BUY with no venue order id: the local-only cancel (#1010 F1) ----------------------- +# +# `executor._run_order` inserts the `pending` row BEFORE `broker.place_order`, so a placement +# that raises (a timeout) leaves a row with no venue id. Reconcile cannot ask about it and +# `_cancel_at_exchange` refuses it, so without this path the row blocks its DCA rule forever. + + +def _stuck(deployment, **overrides: Any) -> int: + repo = _book(deployment) + order_id = _order(repo, **{"side": "buy", "raw_response": None, **overrides}) + repo._conn.close() # noqa: SLF001 + return order_id + + +def _count_broker_builds(monkeypatch: pytest.MonkeyPatch, broker: Any) -> list[int]: + built: list[int] = [] + + def _build(_cfg: Any) -> Any: + built.append(1) + return broker + + monkeypatch.setattr("keel.commands._common._build_broker", _build) + return built + + +def test_a_pending_buy_with_no_venue_id_is_marked_canceled_locally_after_a_typed_yes( + deployment, monkeypatch +) -> None: + built = _count_broker_builds(monkeypatch, deployment[2]) + order_id = _stuck(deployment) + + result = _run(deployment, ["orders", "cancel", str(order_id)], stdin="yes\n") + + assert result.exit_code == 0, result.output + assert "Coinbase app" in result.output, "the prompt must say to check the venue first" + assert deployment[2].cancelled == [] and built == [], "the venue is never reached" + row = _book(deployment).get_order(order_id) + assert row["status"] == "canceled" + import json + + from keel.execution.executor import _native_order_id + + assert json.loads(row["raw_response"])["local_cancel"], "why it was cancelled is recorded" + assert _native_order_id(row) is None, "the note must never read as a venue order id" + + +def test_the_local_cancel_wants_the_word_yes_not_the_entry_prompts_y(deployment) -> None: + order_id = _stuck(deployment) + + result = _run(deployment, ["orders", "cancel", str(order_id)], stdin="y\n") + + assert result.exit_code != 0 + assert "confirmation not given" in result.output + assert _book(deployment).get_order(order_id)["status"] == "pending" + + +def test_the_local_cancel_is_refused_off_a_terminal(deployment, monkeypatch) -> None: + monkeypatch.setattr("keel.commands._common._is_interactive", lambda: False) + order_id = _stuck(deployment) + + result = _run(deployment, ["orders", "cancel", str(order_id)], stdin="yes\n") + + assert result.exit_code != 0 + assert "interactive terminal" in result.output + assert _book(deployment).get_order(order_id)["status"] == "pending" + + +def test_a_pending_buy_that_has_a_venue_id_still_goes_to_the_venue(deployment) -> None: + """The local path is for a row with NO venue id only; one the venue issued an id for is + cancelled AT the venue exactly as before, whatever the operator types.""" + order_id = _stuck(deployment, **{"raw_response": '{"order_id": "venue-1"}'}) + + result = _run(deployment, ["orders", "cancel", str(order_id)], stdin="yes\n") + + assert result.exit_code == 0, result.output + assert "Coinbase app" not in result.output + assert deployment[2].cancelled == ["venue-1"] + assert "cancelled order" in result.output and "at the venue" in result.output + + +def test_a_SELL_with_no_venue_id_is_not_offered_the_local_cancel(deployment) -> None: + """Narrow on purpose: an exit or a protective leg marked canceled locally would have + reconcile re-place protection over it. Only an ENTRY with no venue id takes this path.""" + repo = _book(deployment) + order_id = _order(repo, side="sell", raw_response=None) + repo._conn.close() # noqa: SLF001 + + phrase = orders_mod.CANCEL_EXIT_PHRASE.format(order_id=order_id) + result = _run(deployment, ["orders", "cancel", str(order_id)], stdin=phrase + "\n") + + assert result.exit_code != 0 + assert "the venue did not cancel" in result.output + assert _book(deployment).get_order(order_id)["status"] == "pending" diff --git a/tests/test_agent.py b/tests/test_agent.py index e9f61e18..da93a133 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -7287,6 +7287,43 @@ def test_catch_up_day_counts_as_dca_day_for_sleeve_exclusion(repo, monkeypatch): assert (row["decision"], row["rails"]["sleeve"]) == ("vetoed", "same_day_dca") +def test_an_in_flight_dca_buy_still_counts_as_a_dca_day_for_sleeve_exclusion(repo, monkeypatch): + """F2, spec §3.4: a DCA BUY of the rule still resting makes `detect` answer `None`, but the + day is still a DCA day -- a same-day sale beside an in-flight buy is the round trip the + exclusion exists to prevent.""" + _seed_rules( + repo, + monkeypatch, + (Dca(product_id=PRODUCT, cadence_days=7), "live"), + (_AlwaysReduceRule(PRODUCT), "live"), + ) + repo._conn.execute("UPDATE rules SET created_at = 0") + repo._conn.commit() + [dca_row] = [row for row in repo.get_rules("live") if row["kind"] == "dca"] + repo.insert_order( + { + "mode": "live", + "product_id": PRODUCT, + "side": Side.BUY.value, + "order_type": "market", + "qty": Decimal("0.5"), + "status": "pending", + "rule_id": dca_row["id"], + "created_at": 37 * DAY + 3_600, + "updated_at": 37 * DAY + 3_600, + } + ) + _held_since_day_0(repo) + now = _history(repo, 41) + broker = _HoldingBroker() + + result = run_once(broker, repo, _config(), now_ts=now) + + assert result.enter_signals == [], "fixture: the in-flight buy blocks the rule" + [row] = repo.get_sell_proposals() + assert (row["decision"], row["rails"]["sleeve"]) == ("vetoed", "same_day_dca") + + # -- a macOS notification for every skipped live DCA buy (#1010) --------------------------------- @@ -7478,10 +7515,128 @@ def test_an_in_flight_order_blocking_a_due_buy_is_a_skip_naming_the_order(repo, [skip] = result.skipped_buys assert (skip.rule_id, skip.product) == (rule_id, PRODUCT) assert f"#{order_id}" in skip.reason - assert "resolve" in skip.reason + assert f"keel orders cancel {order_id}" in skip.reason, "the reason names the way out (F1)" + assert skip.blocked_by == (order_id,) assert skip.retry_until == "1970-01-22" +def test_a_blocked_skip_says_blocked_and_an_ordinary_skip_says_retrying(repo): + """F4: a rule blocked by a resting order does NOT retry daily -- it buys nothing until the + order is resolved -- so its banner must not promise a retry. Every other skip keeps it.""" + posted: list[str] = [] + + def _ok(_title: str, message: str) -> bool: + posted.append(message) + return True + + blocked = agent.SkippedBuy( + 7, PRODUCT, "previous DCA order #5 is still pending", "1970-01-22", blocked_by=(5,) + ) + vetoed = agent.SkippedBuy(8, PRODUCT, "max_per_day", "1970-01-22") + + assert agent._post_skipped_buys(repo, [blocked, vetoed], 15 * DAY, post=_ok) == 2 + assert posted[0] == ( + f"{PRODUCT} (rule 7): previous DCA order #5 is still pending. " + "Blocked until that order is resolved: keel orders cancel 5." + ) + assert "Retrying" not in posted[0] + assert posted[1] == f"{PRODUCT} (rule 8): max_per_day. Retrying daily until 1970-01-22." + + +def test_a_stuck_order_cancelled_locally_lets_the_rule_catch_up_next_cycle(repo, monkeypatch): + """F1: a placement that raised leaves a `pending` row with no venue id, which blocks the rule. + After `keel orders cancel N` marks it canceled locally, the NEXT cycle buys the window.""" + import json + + from keel.commands import orders as orders_mod + + _spy_post(monkeypatch) + rule_id = _live_dca_rule(repo) + stuck = repo.insert_order( + { + "mode": "live", + "product_id": PRODUCT, + "side": Side.BUY.value, + "order_type": "market", + "qty": Decimal("0.5"), + "status": "pending", + "rule_id": rule_id, + "raw_response": None, + "created_at": 15 * DAY + 3_600, + "updated_at": 15 * DAY + 3_600, + } + ) + series = _dca_series(16) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + blocked = run_once(broker, repo, _config(), now_ts=16 * DAY + 3_600) + assert broker.place_calls == [] and [s.blocked_by for s in blocked.skipped_buys] == [ + (stuck,) + ], "fixture: the stuck row blocks the rule" + + # What `keel orders cancel N` writes on the local-only path (`orders._cancel_locally`). + repo.update_order( + stuck, + status="canceled", + raw_response=json.dumps({"local_cancel": orders_mod.LOCAL_CANCEL_NOTE}), + updated_at=16 * DAY + 7_200, + ) + series = _dca_series(17) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=17 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in result.enter_signals] == [True] + assert [r.placed for r in result.enter_results] == [True] + assert len(broker.place_calls) == 1 + assert result.skipped_buys == [] + + +def test_a_pending_buy_reconcile_fills_this_cycle_is_not_counted_in_flight(repo, monkeypatch): + """F3: catch-up reads the ledger AFTER reconcile. The previous window's BUY was still + `pending` when the cycle began, and the venue says it FILLED; reconcile books that at the top + of the cycle, so this cycle's window-14 buy goes out -- read before reconcile, the settled row + would have blocked the rule and reported an in-flight skip.""" + _spy_post(monkeypatch) + rule_id = _live_dca_rule(repo) + order_id = repo.insert_order( + { + "mode": "live", + "product_id": PRODUCT, + "side": Side.BUY.value, + "order_type": "market", + "qty": Decimal("0.5"), + "status": "pending", + "rule_id": rule_id, + "raw_response": '{"order_id": "venue-prev"}', + "created_at": 8 * DAY + 3_600, + "updated_at": 8 * DAY + 3_600, + } + ) + series = _dca_series(14) + _seed_history(repo, series) + + class _FilledAtVenue(FakeBroker): + def get_order(self, native_id: str) -> OrderStatus: + return OrderStatus( + order_id=native_id, + status="FILLED", + filled_size=Decimal("0.5"), + average_filled_price=Decimal("100"), + total_fees=Decimal("0.45"), + ) + + broker = _FilledAtVenue(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=15 * DAY + 3_600) + + assert repo.get_order(order_id)["status"] == "filled", "fixture: reconcile settled it" + assert result.skipped_buys == [] + assert [r.placed for r in result.enter_results] == [True] + + def test_withheld_entries_make_every_due_dca_rule_a_skip(repo, monkeypatch): """Controller ruling 2: the whole cycle's entries are withheld (a lagging product's bar), and the due DCA rule beside it is a skip with that reason -- the very silence the operator hit.""" diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index 4ba4d423..8d2339ce 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -264,7 +264,7 @@ def test_the_gate_vocabulary_is_stated_and_bounded() -> None: def test_only_the_two_halt_releases_are_reachable_from_the_browser() -> None: """**Which actions the second gate covers, named, so widening it is an edit here.** - Eight of the ten remain CLI-only. The two that are not are the halt releases, and each + Nine of the eleven remain CLI-only. The two that are not are the halt releases, and each appears twice in the inventory -- once as its CLI row, once as the `web` row mirroring it.""" web = {cap.mirrors for cap in CAPABILITIES if cap.gate == BROWSER.name} diff --git a/tests/test_runbook_console.py b/tests/test_runbook_console.py index e4884a30..de77ff93 100644 --- a/tests/test_runbook_console.py +++ b/tests/test_runbook_console.py @@ -148,7 +148,20 @@ def test_the_runbook_states_the_capability_count_the_registry_declares() -> None mirror nothing, the same rule the `keel/web` prose is held to.""" from keel.capabilities import CAPABILITIES - words = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"] + words = [ + "zero", + "one", + "two", + "three", + "four", + "five", + "six", + "seven", + "eight", + "nine", + "ten", + "eleven", + ] stated = re.findall(r"`keel capabilities` lists all (\w+)", _section()) assert len(stated) == 1, stated assert stated[0] in words, stated diff --git a/tests/web/test_web_prose_stays_true.py b/tests/web/test_web_prose_stays_true.py index 33c7f5f2..941b85ac 100644 --- a/tests/web/test_web_prose_stays_true.py +++ b/tests/web/test_web_prose_stays_true.py @@ -219,7 +219,7 @@ def _stated_count(text: str, pattern: str) -> int: def test_the_gates_module_counts_the_actions_it_refuses_from_the_inventory() -> None: """`gates`' docstring says how many capability increases the browser gate refuses outright. - That is every action in `keel.capabilities` that no `web` row mirrors -- eight of the ten + That is every action in `keel.capabilities` that no `web` row mirrors -- nine of the eleven since the inventory grew, while the docstring still said seven.""" browser_mirrored = {cap.mirrors for cap in CAPABILITIES if cap.gate == BROWSER.name} terminal_only = [ From f05356d1368fbfcc2464543ce806adb5bfdeeac1 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 15:16:39 -0500 Subject: [PATCH 14/18] fix(rail-17): truthful messages, next-buy day honours catch-up (#1010) F5: rail 17's UNKNOWN veto promises a retry only to a DCA buy. F6: `keel withdrawals show` names the exception type when the venue is unreadable (still exit 0). F7: the doctor source-scan test is now a behaviour test: a garbage or expired `withdrawals_attested_at` leaves the rail-17 finding unchanged. F8: Pulse's next DCA buy is derived from `Dca.due_window` on the clock with the ledger's catch-up state, so a pending catch-up shows as the next cycle's bar; the console column reads "next buy day (UTC)". F9: `keel status --help` and the withdrawals capability row describe the venue read and the manual suspension, not an expiring attestation. Co-Authored-By: Claude Opus 5.5 --- keel/capabilities.py | 5 ++- keel/commands/pulse.py | 78 +++++++++++++++++++++++++++++++--- keel/commands/status.py | 9 ++-- keel/commands/withdrawals.py | 11 +++-- keel/execution/guards.py | 10 ++++- keel/web/static/js/render.js | 2 +- tests/commands/test_doctor.py | 26 +++++++++--- tests/commands/test_pulse.py | 42 ++++++++++++++++++ tests/commands/test_status.py | 15 +++++++ tests/execution/test_guards.py | 10 +++++ tests/test_capabilities.py | 9 ++++ tests/test_cli.py | 6 +-- 12 files changed, 195 insertions(+), 28 deletions(-) diff --git a/keel/capabilities.py b/keel/capabilities.py index 3c7aef7b..5728697c 100644 --- a/keel/capabilities.py +++ b/keel/capabilities.py @@ -191,8 +191,9 @@ def key(self) -> tuple[str, str]: surface="cli", invocation="keel withdrawals attest --enabled", increases=( - "rail 17 stops vetoing: the operator attests that withdrawals are possible at the " - "venue, which is what makes holding a balance there permissible rather than assumed" + "rail 17's manual suspension is cleared, so new entries are no longer halted by the " + "operator's own brake; rail 17 still reads the venue's withdrawal flag on every live " + "buy and vetoes it when the venue says no or cannot be read" ), ), Capability( diff --git a/keel/commands/pulse.py b/keel/commands/pulse.py index bcf1926d..e3335a8e 100644 --- a/keel/commands/pulse.py +++ b/keel/commands/pulse.py @@ -2,7 +2,7 @@ A VIEW over `StatusReport`, `ActivityFeed` and `existing_dca_rules`: it reads nothing the CLI cannot and computes only what no existing report holds -- today's sums, the why-not list, the -next DCA cadence day, the unbracketed roll-call. The halt is not here: `keel/web/gates.py` owns +next DCA buy day, the unbracketed roll-call. The halt is not here: `keel/web/gates.py` owns it ("stopping must never be slower than starting"), and this module only names the command that undoes it (`RESUME_COMMAND`), so the client never has to. @@ -24,9 +24,13 @@ from __future__ import annotations +import logging +from collections.abc import Mapping from dataclasses import dataclass from decimal import Decimal +from keel_core.telemetry import log_exception + from keel.commands.activity import ActivityFeed from keel.commands.dca_plan import ExistingDcaRule, existing_dca_rules from keel.commands.status import ( @@ -39,7 +43,11 @@ from keel.commands.trading import KILL_ENGAGED_LINE from keel.config import Config from keel.data.repository import Repository -from keel.strategy.rules.dca import next_cadence_day_ts +from keel.strategy.rules.dca import Dca, DcaCatchUp, next_cadence_day_ts + +logger = logging.getLogger(__name__) + +_DAY = 86_400 #: What undoes a halt. The CLI's own command name; the console shows it and never performs it. RESUME_COMMAND = "keel resume" @@ -80,7 +88,10 @@ class NextDcaBuy: product_id: str cadence_days: int budget_usd: Decimal - #: 00:00 UTC of the cadence day (`dca.next_cadence_day_ts`) -- a day, not an order time. + #: 00:00 UTC of the daily BAR the rule's next buy is off -- a day, not an order time: the buy + #: is placed by the first cycle after that bar closes. A pending catch-up (#1010) is the + #: next cycle's bar (yesterday's, or today's); otherwise the next cadence day + #: (`dca.next_cadence_day_ts`). See `_next_buy_day_ts`. next_cadence_day_ts: int #: `ExistingDcaRule.budget_inferred`, carried through: the budget is the rule's default, not #: one the row stored. @@ -208,14 +219,39 @@ def _why_not(status: StatusReport, feed: ActivityFeed) -> tuple[str, ...]: return tuple(reasons) -def _next_dca(rules: tuple[ExistingDcaRule, ...], now_ts: int) -> tuple[NextDcaBuy, ...]: +def _next_buy_day_ts(rule: ExistingDcaRule, catch_up: DcaCatchUp | None, now_ts: int) -> int: + """00:00 UTC of the bar the rule's next buy is off, by `Dca.due_window` on the CLOCK (#1010). + + The cycle on day `D` buys bar `D - 1`, so with the ledger's `catch_up` in hand: bar + `today - 1` when it is due (the next cycle buys it -- a catch-up, or a window-open bar not yet + bought), else bar `today` when it is due (tomorrow's cycle), else the next cadence day. The + same `_due` decision the engine makes, so the console cannot promise a day the engine skips. + Without `catch_up` (an unreadable ledger, or a caller that passed none) it is the cadence + day alone, which is all a window-open-only rule ever had. + """ + if catch_up is not None: + dca = Dca(rule.product_id, cadence_days=rule.cadence_days) + dca.catch_up = catch_up + today = now_ts // _DAY + for day in (today - 1, today): + if dca.due_window(day) is not None: + return day * _DAY + return next_cadence_day_ts(now_ts, rule.cadence_days) + + +def _next_dca( + rules: tuple[ExistingDcaRule, ...], + now_ts: int, + catch_up: Mapping[int, DcaCatchUp] | None = None, +) -> tuple[NextDcaBuy, ...]: + catch_up = catch_up or {} return tuple( NextDcaBuy( rule_id=r.rule_id, product_id=r.product_id, cadence_days=r.cadence_days, budget_usd=r.budget_usd, - next_cadence_day_ts=next_cadence_day_ts(now_ts, r.cadence_days), + next_cadence_day_ts=_next_buy_day_ts(r, catch_up.get(r.rule_id), now_ts), budget_inferred=r.budget_inferred, ) for r in rules @@ -223,12 +259,35 @@ def _next_dca(rules: tuple[ExistingDcaRule, ...], now_ts: int) -> tuple[NextDcaB ) +def _dca_catch_ups(repo: Repository, rules: tuple[ExistingDcaRule, ...]) -> dict[int, DcaCatchUp]: + """The live ledger's `DcaCatchUp` per LIVE DCA rule, by the cycle's own reader + (`agent._dca_catch_up_state`, mode `"live"`: a `live`-status rule is traded by a live cycle). + A rule whose ledger read raises is left out -- its next buy falls back to the cadence day -- + and logged, never costing the view.""" + from keel import agent + + rows = {row.get("id"): row for row in repo.get_rules("live")} + found: dict[int, DcaCatchUp] = {} + for rule in rules: + row = rows.get(rule.rule_id) + if rule.status != "live" or row is None: + continue + try: + dca = Dca(rule.product_id, cadence_days=rule.cadence_days) + dca.rule_id = rule.rule_id + found[rule.rule_id] = agent._dca_catch_up_state(repo, dca, row, mode="live") + except Exception: # noqa: BLE001 -- a view degrades to the cadence day, never fails + log_exception(logger, "pulse.dca_catch_up_unreadable", rule_id=rule.rule_id) + return found + + def build_pulse( status: StatusReport, feed: ActivityFeed, dca_rules: tuple[ExistingDcaRule, ...], *, now_ts: int, + catch_up: Mapping[int, DcaCatchUp] | None = None, ) -> PulseReport: """`gather_pulse`'s composition over inputs already in hand. PURE.""" # `bracket_verdict` decides, as it does for `keel status` and the bracket cells: paper never @@ -256,7 +315,7 @@ def build_pulse( no_cycles_line="" if feed.cycles else _NO_CYCLES_LINE[basis], last_cycle_ts=_last_cycle_ts(feed), why_not=_why_not(status, feed), - next_dca=_next_dca(dca_rules, now_ts), + next_dca=_next_dca(dca_rules, now_ts, catch_up), unbracketed=unbracketed, unbracketed_count=len(unbracketed), resume_command=RESUME_COMMAND if status.kill_switch_engaged else "", @@ -281,6 +340,11 @@ def gather_pulse( ) -> PulseReport: """The Pulse view's report: `gather_status` and `existing_dca_rules` read from the ledger, `feed` (scoped to "today" by the caller) read from the log. No broker, no network.""" + dca_rules = existing_dca_rules(repo) return build_pulse( - gather_status(repo, config, now_ts), feed, existing_dca_rules(repo), now_ts=now_ts + gather_status(repo, config, now_ts), + feed, + dca_rules, + now_ts=now_ts, + catch_up=_dca_catch_ups(repo, dca_rules), ) diff --git a/keel/commands/status.py b/keel/commands/status.py index 20854d18..704b24c0 100644 --- a/keel/commands/status.py +++ b/keel/commands/status.py @@ -4,7 +4,7 @@ ("A dedicated `keel status` command is deferred as a follow-up"); this is that follow-up. Its job is narrow: let an operator running a paper-forward see the agent's state at a glance, **purely from the local DB and config** -- mode, kill-switch, autonomy, Rail 11 drawdown/equity -state, rail 17's withdrawal-attestation freshness, open positions, rule counts, and per-product +state, rail 17's manual withdrawal suspension, open positions, rule counts, and per-product data freshness. It NEVER calls the broker; that is the whole point (`monitor`/`agent` are the commands that touch the network). @@ -627,9 +627,10 @@ def _report_to_jsonable(report: StatusReport) -> dict[str, Any]: def status_cmd(ctx: click.Context, as_json: bool) -> None: """Operator-observability snapshot of the agent's state -- read-only, no broker call. - Shows mode, the kill-switch, autonomy, Rail 11 drawdown/equity state, rail 17's - withdrawal-attestation state (fresh with time remaining, or EXPIRED with how long ago and - the command that re-attests), open positions, rule counts, and per-product data freshness, + Shows mode, the kill-switch, autonomy, Rail 11 drawdown/equity state, rail 17's manual + withdrawal suspension (off, ON with the command that clears it, or unknown -- rail 17 itself + reads the venue's withdrawal flag on every live buy, so there is no expiry to show), open + positions, rule counts, and per-product data freshness, all read straight from the local DB and config. This is the interim of the deferred `keel status` TUI: same underlying report (`--json` is its forward-compatible shape), just rendered to the terminal. diff --git a/keel/commands/withdrawals.py b/keel/commands/withdrawals.py index c2d3ab40..59eaab7d 100644 --- a/keel/commands/withdrawals.py +++ b/keel/commands/withdrawals.py @@ -125,10 +125,15 @@ def withdrawals_show(ctx: click.Context) -> None: config = _load_cfg(ctx) broker = _build_broker(config, timeout=SHOW_VENUE_TIMEOUT_SEC, repo=repo) answers = broker.withdrawals_allowed(_wallet_currencies(repo)) - except Exception: + except Exception as exc: # A viewer, not a gate: a venue that cannot be asked is reported, never raised, so the - # operator can still read the manual-suspension line above it. - click.echo("venue unreadable -- rail 17 refuses entries while it cannot read the venue") + # operator can still read the manual-suspension line above it. The exception TYPE, as + # the rail's own veto says it (`venue unreadable: `), and not its message -- + # an adapter's message can carry a request URL or a credential name. + click.echo( + f"venue unreadable: {type(exc).__name__} -- rail 17 refuses entries while it cannot " + "read the venue" + ) return for currency, allowed in answers.items(): click.echo(f" {currency:<8}{_WALLET_WORDS[allowed]}") diff --git a/keel/execution/guards.py b/keel/execution/guards.py index 8e1c29d5..5eb3106d 100644 --- a/keel/execution/guards.py +++ b/keel/execution/guards.py @@ -915,11 +915,17 @@ def check( # Fails CLOSED on None, like rails 12/13 — silence is not evidence of possession. if is_buy and not offline: if intent.withdrawals_enabled is None: + # Catch-up (#1010) is DCA-only: only a DCA buy is retried within its window, so only + # its message says so. + retry = ( + " Buy skipped; it is retried next cycle within its window." + if intent.is_dca + else " Buy skipped." + ) violations.append( f"withdrawal_capability: UNKNOWN ({intent.withdrawals_detail}). Under §65.4 " "possession requires that nothing prevents withdrawal on demand; an unverified " - "state is not evidence that it holds. Buy skipped; it is retried next cycle " - "within its window." + f"state is not evidence that it holds.{retry}" ) elif intent.withdrawals_enabled is False: violations.append( diff --git a/keel/web/static/js/render.js b/keel/web/static/js/render.js index 5b3fb5ac..5437f160 100644 --- a/keel/web/static/js/render.js +++ b/keel/web/static/js/render.js @@ -861,7 +861,7 @@ function pulseNextDca(data) { { label: "cadence", numeric: false }, { label: "budget", numeric: true }, { label: "budget basis", numeric: false }, - { label: "next cadence day (UTC)", numeric: false }, + { label: "next buy day (UTC)", numeric: false }, ], data.next_dca.map( /** @param {any} row */ (row) => [ diff --git a/tests/commands/test_doctor.py b/tests/commands/test_doctor.py index 5d08b0a8..c9cc7c38 100644 --- a/tests/commands/test_doctor.py +++ b/tests/commands/test_doctor.py @@ -99,12 +99,26 @@ def test_doctor_rail17_warns_when_the_store_is_unreadable() -> None: assert "unreadable" in (rail17.headline + rail17.detail).lower() -def test_doctor_never_reads_the_old_attestation_timestamp() -> None: - import inspect - - from keel.commands import doctor - - assert '"withdrawals_attested_at"' not in inspect.getsource(doctor) +def test_doctor_rail17_ignores_a_garbage_or_expired_old_attestation_timestamp( + tmp_path, valid_config_path +) -> None: + """F7, by behaviour rather than a source scan: rail 17 has no clock (#1010). A leftover + `withdrawals_attested_at` -- unparseable, or a week and more in the past, which the old + expiry would have failed -- changes nothing: with no suspension the finding passes.""" + config = load_config(valid_config_path) + seen: list[tuple[str, str, str]] = [] + for leftover in ("not-a-timestamp", NOW - 30 * DAY): + repo = _seeded_repo(tmp_path / f"keel-{len(seen)}.db") + repo.set_state("withdrawals_enabled", True) + repo.set_state("withdrawals_attested_at", leftover) + assert repo.get_state("withdrawals_attested_at") == leftover, "fixture reached the store" + (rail17,) = [ + f for f in gather_findings(repo, config, [], NOW) if f.name == "attest.withdrawals" + ] + seen.append((rail17.status, rail17.headline, rail17.detail)) + baseline = _rail17(False) + assert seen == [(baseline.status, baseline.headline, baseline.detail)] * 2 + assert baseline.status == OK def test_rail14_absent_fails_naming_the_venue() -> None: diff --git a/tests/commands/test_pulse.py b/tests/commands/test_pulse.py index 28fbe93e..dc004b47 100644 --- a/tests/commands/test_pulse.py +++ b/tests/commands/test_pulse.py @@ -29,6 +29,7 @@ from keel.commands.trading import KILL_ENGAGED_LINE from keel.data.db import connect, migrate from keel.data.repository import Repository +from keel.types import Side from tests.commands.test_activity import _line, _quiet_cycle_lines, _rail_veto_cycle_lines from tests.commands.test_dca_plan import _insert_dca from tests.web.test_payload import _status_report @@ -239,6 +240,47 @@ def test_next_dca_is_today_when_today_is_a_cadence_day() -> None: assert report.next_dca[0].next_cadence_day_ts == 105 * _DAY +def _live_since_day_0(repo: Repository) -> int: + rule_id = _insert_dca(repo, "BTC-USD", "live") + repo._conn.execute("UPDATE rules SET created_at = 0, promoted_at = 0") # noqa: SLF001 + repo._conn.commit() # noqa: SLF001 + return rule_id + + +def test_next_dca_shows_a_pending_catch_up_as_the_next_cycle(valid_config_path: Path) -> None: + """F8: window 98 (days 98-104) never filled and the rule was live before it opened, so the + next cycle buys bar 99 as a catch-up -- "next DCA buy" must say day 99 (placed by the first + cycle after day 99's bar closes), not the next cadence day 105.""" + repo = _repo() + _live_since_day_0(repo) + config = load_config(str(valid_config_path)) + + [buy] = gather_pulse(repo, config, _today_feed(), now_ts=NOW).next_dca + + assert buy.next_cadence_day_ts == 99 * _DAY + + +def test_next_dca_is_the_next_cadence_day_once_the_window_filled(valid_config_path: Path) -> None: + repo = _repo() + rule_id = _live_since_day_0(repo) + repo.insert_order( + { + "mode": "live", + "product_id": "BTC-USD", + "side": Side.BUY.value, + "qty": Decimal("0.1"), + "status": "filled", + "rule_id": rule_id, + "created_at": 99 * _DAY + 3600, # the day-99 cycle bought bar 98, window 98 + } + ) + config = load_config(str(valid_config_path)) + + [buy] = gather_pulse(repo, config, _today_feed(), now_ts=NOW).next_dca + + assert buy.next_cadence_day_ts == 105 * _DAY + + # -- unbracketed roll-call ---------------------------------------------------------------------- diff --git a/tests/commands/test_status.py b/tests/commands/test_status.py index 7eb11c24..797e7b02 100644 --- a/tests/commands/test_status.py +++ b/tests/commands/test_status.py @@ -634,3 +634,18 @@ def test_status_command_json_flag_emits_parseable_json(tmp_path, valid_config_pa assert "open_positions" in payload assert "rule_counts" in payload assert "data_freshness" in payload + + +def test_status_help_describes_the_venue_read_and_the_manual_suspension() -> None: + """F9: rail 17 has no expiry since #1010, so `keel status --help` must not promise a + "fresh with time remaining, or EXPIRED" state; it shows the manual suspension.""" + from click.testing import CliRunner + + from keel.cli import cli + + result = CliRunner().invoke(cli, ["status", "--help"]) + text = " ".join(result.output.split()) + + assert result.exit_code == 0, result.output + assert "EXPIRED" not in text and "time remaining" not in text + assert "manual withdrawal suspension" in text diff --git a/tests/execution/test_guards.py b/tests/execution/test_guards.py index ecf65ed3..581aeb1b 100644 --- a/tests/execution/test_guards.py +++ b/tests/execution/test_guards.py @@ -1537,6 +1537,16 @@ def test_rail17_messages_carry_the_venue_detail_and_no_attestation_advice( assert "attest" not in msg +def test_rail17_unknown_promises_a_retry_only_to_a_dca_buy(repo: Repository) -> None: + """F5: catch-up is DCA-only. A DCA buy refused on an unreadable venue is retried within its + window; any other BUY is not, so its message must not say it is.""" + for is_dca, promised in ((True, True), (False, False)): + intent = _intent(withdrawals_enabled=None, withdrawals_detail="venue down", is_dca=is_dca) + result = check(intent, repo, _config(), NOW_TS) + (msg,) = [v for v in result.violations if v.startswith("withdrawal_capability")] + assert ("retried" in msg) is promised, msg + + def test_rail17_passes_when_withdrawals_are_attested_enabled(repo: Repository) -> None: result = check(_intent(withdrawals_enabled=True), repo, _config(), NOW_TS) assert "withdrawal_capability" not in _keys(result) diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index 8d2339ce..48b4bfa9 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -366,3 +366,12 @@ def test_render_lines_names_the_surfaces_that_read_it() -> None: assert importers, "render_lines has a reader; this test reads the import graph to find it" doc = render_lines.__doc__ or "" assert sorted(m for m in importers if f"`{m}`" not in doc) == [] + + +def test_the_withdrawals_row_describes_the_manual_suspension_not_an_attestation() -> None: + """F9: since #1010 the venue answers rail 17 on every buy; `--enabled` clears the operator's + own brake. The row must say what is released, not the retired "operator attests" model.""" + [row] = [c for c in CAPABILITIES if c.function == "withdrawals_attest"] + assert "attests that withdrawals are possible" not in row.increases + assert "manual suspension" in row.increases + assert "venue" in row.increases diff --git a/tests/test_cli.py b/tests/test_cli.py index 964ae241..f4ae94a8 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -2409,11 +2409,11 @@ def test_show_says_no_wallet_for_an_unheld_asset(tmp_path, valid_config_path, mo def test_show_reports_unreadable_venue_without_failing(tmp_path, valid_config_path, monkeypatch): - broker = FakeBroker(withdrawals_raises=RuntimeError("boom")) + broker = FakeBroker(withdrawals_raises=TimeoutError("boom")) result, _built, _db = _withdrawals_show(tmp_path, valid_config_path, monkeypatch, broker) assert result.exit_code == 0, result.output - assert "venue unreadable" in result.output + assert "venue unreadable: TimeoutError" in result.output, "F6: say what failed" assert "manual suspension: off" in result.output @@ -2423,7 +2423,7 @@ def test_show_reports_unbuildable_broker_without_failing(tmp_path, valid_config_ ) assert result.exit_code == 0, result.output - assert "venue unreadable" in result.output + assert "venue unreadable: RuntimeError" in result.output def test_show_names_a_manual_suspension(tmp_path, valid_config_path, monkeypatch): From c865a60e3226e6ee840973c7176b09345d3cdb72 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 15:17:11 -0500 Subject: [PATCH 15/18] docs(rail-17): name the venues rail 17 refuses; fix the go-live sentence (#1010) F10: the rail-17 page states that a venue whose adapter lacks `withdrawals_allowed` (Alpaca, Kraken, Robinhood) has every live BUY refused, and both manual-switch lines use the live config and database. F12: the go-live runbook's garbled rail-17 sentence is rewritten. Co-Authored-By: Claude Opus 5.5 --- docs/go-live-runbook.md | 6 +++--- docs/rails/rail-17-withdrawal-capability.md | 11 +++++++++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/go-live-runbook.md b/docs/go-live-runbook.md index ef857d49..e12f1898 100644 --- a/docs/go-live-runbook.md +++ b/docs/go-live-runbook.md @@ -74,9 +74,9 @@ keel withdrawals show # rail 17: manual suspension + the venue' keel scope show # rail 20: can this venue's credential place a live ENTRY? ``` -Attest anything missing (`keel assets attest`, `keel subscription attest`, -`keel scope attest --trading`); rail 17 needs no attestation, it reads the venue on every live BUY before continuing. If a rail vetoes -later, its message names the command that fixes it. +Attest anything missing with `keel assets attest`, `keel subscription attest` and +`keel scope attest --trading`. Rail 17 needs no attestation: it reads the venue's withdrawal flag +on every live BUY. If a rail vetoes later, its message names the command that fixes it. Rail 20 exists because a credential that *reads* fine is not evidence it can *trade*: a well-formed `ROBINHOOD_API_KEY` passed every read this deployment ever made, and the first live diff --git a/docs/rails/rail-17-withdrawal-capability.md b/docs/rails/rail-17-withdrawal-capability.md index 51af366d..8c1da1b2 100644 --- a/docs/rails/rail-17-withdrawal-capability.md +++ b/docs/rails/rail-17-withdrawal-capability.md @@ -23,6 +23,12 @@ read the key can already make; nothing new is granted. withdrawal freeze would be strictly worse than holding through it. Exits, stop-outs and protective orders are unaffected. +**Only the Coinbase adapter can answer.** `withdrawals_allowed` is implemented by the Coinbase +broker adapter alone; the Alpaca, Kraken and Robinhood adapters do not have it. On a live +deployment against any of those venues the rail cannot read the venue, so **every live BUY is +refused** (`withdrawal_capability: UNKNOWN (venue adapter cannot report withdrawals)`). That is the fail-closed rule above, not a fault to +attest around; SELLs on those venues are unaffected. + In paper trading the rail is *skipped* (it describes the real account, which a paper rehearsal cannot see) — recorded as skipped, never silently omitted. SELLs and paper never read the venue. @@ -44,7 +50,7 @@ only ever reduces capability and is ungated. Per deployment (each has its own database): ```bash -keel withdrawals attest --suspended +keel --config config.live-sandbox.yaml --db keel-live.db withdrawals attest --suspended keel --config config.live-sandbox.yaml --db keel-live.db withdrawals attest --enabled ``` @@ -70,5 +76,6 @@ been validly possessed at all — so **acquiring more of it** is the thing to st A rail-17 veto skips that day's DCA buy. DCA catch-up retries a buy that was due and did not fill on each later daily cycle until it fills or the rule's next window opens, so a short freeze does -not cost the window. A skipped live DCA buy also posts a macOS notification ("keel-live: DCA buy +not cost the window. A manual suspension is different: it is retried the same way, but nothing +clears it except `keel withdrawals attest --enabled`. A skipped live DCA buy also posts a macOS notification ("keel-live: DCA buy skipped") with the reason; see `docs/operator-runbook.md`. From 286d3158af4fce248c2fe130999331ce7640412b Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 15:23:51 -0500 Subject: [PATCH 16/18] test(mcp): the write vocabulary catches the local-only cancel (#1010) F1 registered `keel orders cancel` (local-only) as a capability; wall 1's vocabulary must trip on it, so `cancel` joins FORBIDDEN_SUBSTRINGS. Co-Authored-By: Claude Opus 5.5 --- tests/mcp/test_readonly.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/mcp/test_readonly.py b/tests/mcp/test_readonly.py index 0517e947..b674cf4f 100644 --- a/tests/mcp/test_readonly.py +++ b/tests/mcp/test_readonly.py @@ -85,6 +85,8 @@ "place", # `keel positions close` (#798, P4): declaring an out-of-band sale shrinks measured exposure. "close", + # `keel orders cancel` (#1010 F1): a local-only cancel releases a DCA rule a stuck order blocks. + "cancel", ) #: A lower->Upper boundary is where two words are glued without any separator a human would From b48f49bb4e42b7e76dac2c982a02156c23bc0df5 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 16:07:50 -0500 Subject: [PATCH 17/18] fix(dca): a buy that executed any quantity fills its window (#1010) `keel orders cancel` on a partially_filled DCA BUY leaves the row `canceled` with its executed `filled_quantity` still on it. `_dca_catch_up_state` counted only `filled`, so the next cycle caught that window up and bought it a second time. `_buy_executed` (filled, or filled_quantity > 0) now decides it for both the catch-up ledger and `_bought_today`. A terminal row that executed nothing still leaves its window to catch up. The in-flight block for resting statuses is unchanged. Co-Authored-By: Claude Opus 5.5 --- keel/agent.py | 33 +++++++++++++++---- keel/strategy/rules/dca.py | 5 +-- tests/test_agent.py | 65 +++++++++++++++++++++++++++++++++++++- 3 files changed, 93 insertions(+), 10 deletions(-) diff --git a/keel/agent.py b/keel/agent.py index d97113e6..2a8704b4 100644 --- a/keel/agent.py +++ b/keel/agent.py @@ -1180,13 +1180,31 @@ def _close_tranches( _DAY = 86_400 +def _buy_executed(order: dict[str, Any]) -> bool: + """Whether an order row bought anything: `filled`, or any status with a venue-observed + `filled_quantity` above zero. + + A `partially_filled` BUY cancelled through `keel orders cancel` ends `canceled` with its + executed quantity still on the row, and reconcile's dead-order path writes `filled` only when + the venue reports the fill. Reading `status == "filled"` alone would call that window unbought + and buy it a second time. A terminal row that executed nothing (`filled_quantity` absent or + zero) spent nothing, so its window is still owed.""" + if order["status"] == "filled": + return True + executed = order.get("filled_quantity") + return executed is not None and Decimal(executed) > 0 + + def _bought_today(repo: Repository, product_id: str, now_ts: int, *, mode: str) -> bool: - """Whether a `mode` BUY of `product_id` FILLED on this UTC day -- the half of "DCA fires - today" the ledger can see once the cycle's own buy has gone out (reductions run after it).""" + """Whether a `mode` BUY of `product_id` executed on this UTC day (`_buy_executed`) -- the + half of "DCA fires today" the ledger can see once the cycle's own buy has gone out + (reductions run after it).""" start = now_ts - now_ts % _DAY return any( - order["side"] == Side.BUY.value and int(order["created_at"]) >= start - for order in repo.get_orders(mode=mode, product_id=product_id, status="filled") + order["side"] == Side.BUY.value + and int(order["created_at"]) >= start + and _buy_executed(order) + for order in repo.get_orders(mode=mode, product_id=product_id) ) @@ -1223,8 +1241,9 @@ def _dca_catch_up_state( - `since_day`: `promoted_at // 86400`, else `created_at // 86400`, else `None` (catch-up off: a row that cannot say when it went live is never read as "live since forever"). - - `filled_window_day`: the window of the rule's most recent FILLED BUY, by the bar it - bought (`_dca_order_bar_day`), not by row order. + - `filled_window_day`: the window of the rule's most recent BUY that executed anything + (`_buy_executed`: `filled`, or a cancelled partial), by the bar it bought + (`_dca_order_bar_day`), not by row order. - `in_flight`: a BUY of the rule in `executor.RESTING_STATUSES`. Raises on an unreadable ledger; `_attach_dca_catch_up` turns that into "catch-up off". @@ -1240,7 +1259,7 @@ def _dca_catch_up_state( if order.get("rule_id") == rule.rule_id and order["side"] == Side.BUY.value ] filled_bar_days = [ - _dca_order_bar_day(order, mode=mode) for order in buys if order["status"] == "filled" + _dca_order_bar_day(order, mode=mode) for order in buys if _buy_executed(order) ] resting = tuple( int(order["id"]) for order in buys if order["status"] in executor.RESTING_STATUSES diff --git a/keel/strategy/rules/dca.py b/keel/strategy/rules/dca.py index cbda860a..fab116ab 100644 --- a/keel/strategy/rules/dca.py +++ b/keel/strategy/rules/dca.py @@ -70,8 +70,9 @@ class DcaCatchUp: - `since_day`: the epoch day the rule went to its current status (`promoted_at // 86400`, else `created_at // 86400`). `None` when the row has neither: catch-up is OFF for that rule (window-open day only) -- a missing timestamp is never read as "since forever". - - `filled_window_day`: `window_start_day` of the rule's most recent FILLED BUY in the - cycle's mode, or `None` when it has none. + - `filled_window_day`: `window_start_day` of the rule's most recent BUY in the cycle's mode + that executed anything (`filled`, or a cancelled partial: `agent._buy_executed`), or + `None` when it has none. - `in_flight`: a BUY of the rule is still resting (`executor.RESTING_STATUSES`); it blocks every buy of the rule, so two are never in flight. - `resting_order_ids`: the local `orders.id`s behind `in_flight`, for the operator-facing diff --git a/tests/test_agent.py b/tests/test_agent.py index 97b4b5c1..b909ebce 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -7277,7 +7277,14 @@ def _promoted_live_dca(repo: Repository, promoted_at: int) -> int: return rule_id -def _live_buy(repo: Repository, rule_id: int, status: str, created_at: int, mode: str = "live"): +def _live_buy( + repo: Repository, + rule_id: int, + status: str, + created_at: int, + mode: str = "live", + filled_quantity: Decimal | None = None, +): return repo.insert_order( { "mode": mode, @@ -7286,6 +7293,7 @@ def _live_buy(repo: Repository, rule_id: int, status: str, created_at: int, mode "order_type": "market", "qty": Decimal("0.5"), "status": status, + "filled_quantity": filled_quantity, "rule_id": rule_id, "created_at": created_at, "updated_at": created_at, @@ -7320,6 +7328,61 @@ def test_a_partially_filled_buy_blocks_its_windows_catch_up(repo): assert len(repo.get_orders(mode="live", product_id=PRODUCT)) == 1 +def test_a_canceled_buy_that_executed_part_fills_its_window(repo): + """A `partially_filled` DCA BUY cancelled through `keel orders cancel` ends `canceled` with + its executed `filled_quantity` still on the row (`orders.py` writes the terminal status after + the fill readback). Money was spent in window 14, so no later cycle of that window may buy + again; window 21 still opens as an ordinary buy.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0 + ) + _live_buy(repo, rule_id, "canceled", 15 * DAY + 3_600, filled_quantity=Decimal("0.2")) + series = _dca_series(15) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + in_window = run_once(broker, repo, _config(), now_ts=16 * DAY + 3_600) + + assert in_window.enter_signals == [] and broker.place_calls == [] + + series.extend(_candle(d * DAY, "100") for d in range(16, 22)) + next_window = run_once(broker, repo, _config(), now_ts=22 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in next_window.enter_signals] == [False] + assert len(broker.place_calls) == 1 + + +@pytest.mark.parametrize("status", ["canceled", "rejected"]) +@pytest.mark.parametrize("executed", [None, Decimal("0")]) +def test_a_buy_that_executed_nothing_leaves_its_window_to_catch_up(repo, status, executed): + """The other side of the rule above: a terminal BUY that executed nothing spent nothing, so + the window is still owed and the next cycle catches it up.""" + rule_id = repo.insert_rule( + "dca", {"product_id": PRODUCT, "cadence_days": 7}, status="live", now_ts=0 + ) + _live_buy(repo, rule_id, status, 15 * DAY + 3_600, filled_quantity=executed) + assert len(repo.get_orders(mode="live")) == 1, "fixture: the row reaches the ledger" + series = _dca_series(15) + _seed_history(repo, series) + broker = FakeBroker(series={(PRODUCT, Granularity.ONE_DAY): series}) + + result = run_once(broker, repo, _config(), now_ts=16 * DAY + 3_600) + + assert [s.setup.context["catch_up"] for s in result.enter_signals] == [True] + assert len(broker.place_calls) == 1 + + +def test_bought_today_counts_a_canceled_buy_that_executed_part(repo): + """The same-day-sale exclusion reads the same fact: a cancelled BUY that executed part of its + size bought today; one that executed nothing did not.""" + rule_id = repo.insert_rule("dca", {"product_id": PRODUCT}, status="live", now_ts=0) + now = 15 * DAY + 7_200 + _live_buy(repo, rule_id, "canceled", 15 * DAY + 3_600, filled_quantity=Decimal("0")) + assert agent._bought_today(repo, PRODUCT, now, mode="live") is False + _live_buy(repo, rule_id, "canceled", 15 * DAY + 3_600, filled_quantity=Decimal("0.2")) + assert agent._bought_today(repo, PRODUCT, now, mode="live") is True + + def test_paper_era_activity_does_not_make_a_live_window_due(repo): """Ported from #1013 (R13): a promoted rule keeps its id, so its paper-era rows sit beside its live ones. A paper fill for window 14 must neither fill nor open that window live: promoted From 87a4cde3ae67653e24324ad98a5b3d8805c4f8db Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Fri, 6 Nov 2026 16:13:52 -0500 Subject: [PATCH 18/18] docs: mark #1013's plan superseded; truthful rail-17 slide and ported-test docstring (#1010) Co-Authored-By: Claude Opus 5.5 --- docs/presentations/se.md | 7 ++++--- .../plans/2026-10-27-rail17-ttl-dca-cadence.md | 2 ++ tests/test_agent.py | 9 +++++---- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/presentations/se.md b/docs/presentations/se.md index 15178701..19fda96a 100644 --- a/docs/presentations/se.md +++ b/docs/presentations/se.md @@ -316,9 +316,10 @@ Four design decisions worth discussing: **Rail 17 is the one to dwell on.** Constructive possession — "the ability to dispose, not physical custody" — is tested by asking whether the asset can be withdrawn from the venue. If it -cannot, you do not effectively possess it, so acquiring more is stopped. The attestation expires -after 7 days (`WITHDRAWAL_ATTESTATION_TTL_SEC`) because "a stale attestation is no better than -none". A classical contract rule, expressed as a TTL. +cannot, you do not effectively possess it, so acquiring more is stopped. Before every live buy +keel asks the venue itself (Coinbase's per-wallet `allow_withdrawals`) and refuses the buy when +the answer is no or cannot be read. A classical contract rule, expressed as a check the venue +answers on every order. ### 6.6 `keel/execution/executor.py` — placing it diff --git a/docs/superpowers/plans/2026-10-27-rail17-ttl-dca-cadence.md b/docs/superpowers/plans/2026-10-27-rail17-ttl-dca-cadence.md index ae666003..d37cf54c 100644 --- a/docs/superpowers/plans/2026-10-27-rail17-ttl-dca-cadence.md +++ b/docs/superpowers/plans/2026-10-27-rail17-ttl-dca-cadence.md @@ -1,5 +1,7 @@ # Rail 17 TTL vs DCA Cadence Implementation Plan (#1010) +> **Superseded by #1016** (`docs/superpowers/specs/2026-11-06-rail-17-venue-read-and-dca-catchup-design.md`). Merged as #1013, then replaced on main: the TTL, its expiry warnings and the window ledger described below no longer exist. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Stop rail 17's 7-day withdrawal-attestation TTL from silently starving the 14-day DCA rules. Three surfaces warn when a live DCA rule's next buy falls after the attestation expires. A DCA window vetoed by a guard is retried on later fresh daily bars of the same window, with at most one buy per window per rule. The rail-17 veto text stops implying that DCA is exempt. diff --git a/tests/test_agent.py b/tests/test_agent.py index b909ebce..8cb452d7 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -7384,10 +7384,11 @@ def test_bought_today_counts_a_canceled_buy_that_executed_part(repo): def test_paper_era_activity_does_not_make_a_live_window_due(repo): - """Ported from #1013 (R13): a promoted rule keeps its id, so its paper-era rows sit beside its - live ones. A paper fill for window 14 must neither fill nor open that window live: promoted - on day 16, after the window's buying cycle (day 15), the rule waits for window 21. (#1013 - also seeded a paper-era signal; `catch_up` reads no signals, so that row would prove nothing.) + """Ported from #1013 (R13), promotion-boundary half: promoted on day 16, after window 14's + buying cycle (day 15), the rule does not catch window 14 up and waits for window 21. The + paper-era fill is seeded for realism only -- `since_day` alone decides this case, so it does + not prove mode isolation; `test_paper_missed_window_catches_up_from_the_paper_ledger` and the + state-level mode test do. (#1013 also seeded a paper-era signal; `catch_up` reads no signals.) """ rule_id = _promoted_live_dca(repo, 16 * DAY + 36_000) _live_buy(repo, rule_id, "filled", 14 * DAY, mode="paper")