Skip to content

fix: keep an unreadable repository out of the team comparison and merge object overrides - #109

Merged
zoeyrose merged 2 commits into
mainfrom
fix/settings-check-followups
Nov 15, 2026
Merged

zoeyrose merged 2 commits into
mainfrom
fix/settings-check-followups

Conversation

@zoeyrose

@zoeyrose zoeyrose commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes two defects in the read-only settings check that #108 added, and brings its texts in line with the shared agent guidance. A cross-check of the three settings pull requests, read together for the first time after #108 merged, found them.

  • An unreadable repository no longer shows its team grants as gone.
  • An override of an object key, such as repository, now changes only the entries it names.
  • README, AGENTS.md and the baseline note no longer tell agents to update the baseline "until the check is clean".
  • AGENTS.md says its commands are for Atrinik, and where another organization's baseline pull request goes.

Intent brief

  • Goal: the check gives no false difference and hides no real one in the two cases below, and its texts say the same thing as the guidance agents now get on every host.
  • Decisions:
    • A repository whose own read failed is left out of the team-grant comparison. One failed read is a "could not verify" line and exit 2, as the README already said; it was exit 1 with a fix text that would have removed real grants from the record.
    • Overrides merge one level deep. With whole-key replacement, recording one differing merge setting for one repository silently stopped the comparison of all its other merge settings. Atrinik's only override is a single name (classic's security configuration), so nothing changes for the current baseline.
    • "Update the baseline until the check is clean" is replaced by: record the change the task authorized, explain each remaining line, and record a difference the task did not cause only with the owner's explicit approval. "Clean" was also unreachable, because the organization-webhooks line always prints.
  • Constraints: unchanged: only gh api -X GET, exit codes 0/1/2/3, at most 550 lines (545 now), standard library only.
  • Out of scope: comparing who may bypass a repository-level ruleset, and printing paired settings as one command.

Implementation / behavior

  • check_settings.py: check_repositories adds a repository to the unread set when its own read fails; profile_for merges an object-valued override into the profile's object.
  • tests/test_check_settings.py: one test for each, both failing on the old code.
  • README.md: step 4 of "How a change is made", the sentence about fix commands, and the repositories row of the baseline table, which now says the merge is one level deep.
  • AGENTS.md: the change bullet, the scope of the run command, and where another organization's baseline pull request goes.
  • baselines/atrinik.json: the note about quoting output, and one reminder that named the owner's workstation.

Validation

  • python3 -m unittest discover -s tests: 37 tests pass, offline.
  • python3 .github/scripts/check_repository.py: no problems. python3 check_settings.py --validate baselines/atrinik.json: valid. git diff --check: clean.
  • Live read-only run against atrinik at 7058641 (the later commit changes only the README), exit 2:
could not verify: orgs/atrinik webhooks: Not Found (HTTP 404) (the login lacks the admin:org_hook scope)

That line is expected: the everyday login cannot read organization webhooks.

  • The other organization's baseline gives the same single line with this head.

Limitations / follow-up

  • An override cannot remove an entry from a profile's object; it can only set one. The merge is one level deep, so a branch_rules override must list all the parameters of a rule it names. Nothing uses either today.
  • The atrinik/agent-integrations skills still do not name this check.

🤖 Generated with Claude Code

…ge object overrides

A cross-check of the merged settings check against the two baselines and
the shared guidance found:

- When the read of one repository failed, its team grants were reported
  as gone (exit 1) instead of the run ending with one could-not-verify
  line (exit 2).
- An override of an object key such as `repository` replaced the whole
  key, so a one-field override silently stopped the comparison of every
  other merge setting. An object override now changes only the entries it
  names.
- README, AGENTS.md and the baseline note said to update the baseline
  "until the check is clean". That invited recording differences the task
  did not cause, and is unreachable while the webhooks line prints. They
  now match the other organization's wording: record only the authorized
  change, explain each remaining line, and record anything else only with
  the owner's explicit approval.
- AGENTS.md scoped its commands to Atrinik and says where another
  organization's baseline pull request goes.
- A reminder named the owner's workstation; it no longer does.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

Review of 7058641 noted that the README did not say how deep the merge
goes, so a partial `branch_rules` override could be written in the belief
that the rule's other parameters stay compared.

Co-Authored-By: Claude <noreply@anthropic.com>
@zoeyrose

Copy link
Copy Markdown
Member Author

Self-review of 583be2ade3dbcc6bbb23d731b53f480eb4b1d987.

What was reviewed

  • A fresh-context reviewer reviewed the whole diff at 7058641 and all of check_settings.py. It ran the 37 tests, the repository check, --validate, and the live read-only run (exit 2, the one expected line), and ran the two new tests against main's script to confirm both fail there. It checked that the merge does not change the meaning of the existing classic override, that validate() still agrees with profile_for, that an unread repository leaves nothing else falsely reported, and that README, AGENTS.md, CONTRIBUTING.md and the description agree with the code and with the shared guidance.
  • 7058641..583be2a is one sentence in the README's baseline table. I checked each of its three statements against profile_for and the tag_rules shape in validate(), and reran the tests and the repository check.

Findings

  1. P3 on 7058641: the README did not say the override merge is one level deep, so a partial branch_rules override could be written in the belief that the rule's other parameters stay compared. Fixed in 583be2a.

No other finding. Declined: nothing.

Where the changes came from

A cross-check of #108 with the other organization's baseline and the shared guidance, run after #108 merged, found the two code defects and the "until the check is clean" wording. An independent skeptic reproduced each one offline with this repository's fixtures before it was fixed here.

@zoeyrose
zoeyrose marked this pull request as ready for review November 14, 2026 21:51
@zoeyrose zoeyrose self-assigned this Nov 14, 2026
@zoeyrose
zoeyrose requested a review from a team November 14, 2026 21:51
@zoeyrose
zoeyrose merged commit d0c9b67 into main Nov 15, 2026
7 checks passed
@zoeyrose
zoeyrose deleted the fix/settings-check-followups branch November 15, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant