Repository navigation
Per-run cgroups on by default on the Hetzner stack - #410
Open
winding-lines wants to merge 2 commits into
Open
winding-lines wants to merge 2 commits into
winding-lines wants to merge 2 commits into
Conversation
morca-stack apply now starts the scheduler with the cgroup flags unless stack.conf says CGROUPS=0. A host without cgroup v2 or its memory controller gets the scheduler without them, and apply and plan say why; a failed remount no longer stops the scheduler (morca then turns run cgroups off itself). With cgroups on, apply (and plan, on stderr) warns for each pipeline whose p95 peak over its last 20 local runs is above its memory_gb's maximum, or what the host can grant. CGROUPS takes 0 or 1 only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #239
What changes
morca-stack applystarts the scheduler with the cgroup flags (--cgroupns=private,SYS_ADMINfor the remount, dropped before morca starts,apparmor=unconfined) unlessstack.confsaysCGROUPS=0.CGROUPSnow takes only0or1./sys/fs/cgroup(nocgroup.controllers), or without its memory controller, the scheduler starts without the flags, as withCGROUPS=0, andapply/planprintmorca-stack: run cgroups off: ....mount -o remount,rw /sys/fs/cgroup || echo ...; exec setpriv ... morca scheduler, so a failed remount no longer leaves a crash-looping scheduler: morca sees a read-only/sys/fs/cgroupand logsrun cgroups off.apply(andplan, on stderr so deploy.sh's plan parsing is untouched) warns for each pipeline whose p95 peak over its last 20 successful local runs (at least 3, asmorca.recommend) is above itsmemory_gbmaximum, else the host's memory lessMORCA_RESERVE_MEMORY_MB. Warn only; it doesn't block. The SQL was checked against a real Postgres.CGROUPS=0/1,CGROUPS=2refused, the v1 host and no-memory-controller hosts, the remount fallback, and the pre-flight warnings (fake host cgroup root and meminfo viaMORCA_HOST_CGROUP/MORCA_MEMINFO).Upgrade note
A stack that never set
CGROUPSgets its scheduler replaced on the next deploy (options: its flags, mounts or CGROUPS changed), which quiesces as any scheduler replacement does; stacks already onCGROUPS=1also get one replacement for the new entrypoint. Before deploying, read the pre-flight warnings (morca-stack plan) and raisememory_gbmaximums that are too low, ormorca-stack configure CGROUPS=0to keep the old behaviour.Not in this PR: the runner split (#232) and CPU priority (#229), which the issue lists as related.
🤖 Generated with Claude Code
https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6