Skip to content

Skip directories that hold no tracked files in the project walk - #153

Open
dduugg wants to merge 3 commits into
mainfrom
skip-untracked-directories
Open

dduugg wants to merge 3 commits into
mainfrom
skip-untracked-directories

Conversation

@dduugg

@dduugg dduugg commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #152.

What

  • The project walk now skips a directory unless it holds a tracked file, could hold a team file, or is the project root.
  • The version goes to 0.7.1, so merging this cuts a patch release.

Why

Since #110 turned off git's ignore rules for the walk, it enters every directory outside the top-level ignore_dirs and drops untracked files one at a time. On a developer checkout with large gitignored directories, the walk took most of the run time. #152 has the details.

How

  • tracked_files::tracked_directories builds the set of tracked paths and the directories above them. It uses the git ls-files output the walk already reads. That output lists a submodule as one path, so submodules stay in the set.
  • walk_enters also keeps any directory inside or above the literal start of a team file glob, such as config/teams for config/teams/**/*.yml. Team files are read before they're tracked (Consider new, untracked team config files during validate/generate #113). A glob that starts with a wildcard has an empty literal start, so the walk enters every directory, as it does today.
  • Outside a git repository, git ls-files fails, so there's nothing to skip by and the walk is unchanged.

Behavior change

A vendored gem directory with no tracked files no longer gets a CODEOWNERS line. Today that line appears only in checkouts that have the directory, so validate can pass on one machine and fail on another. Staged files count as tracked, so a gem that's being added gets its line once it's git added.

Measured

This uses the checkout from #152: 2.0M entries on disk and 145k tracked files, on macOS with 12 cores. The generated CODEOWNERS is byte-identical to 0.7.0's, and to the file committed in that checkout.

The table comes from a prototype of this change. It shows medians of 5 runs, ordered 0.7.0, branch, branch, 0.7.0. The machine was busy, so each case shows both medians.

Case 0.7.0 this branch
--no-cache validate 16.6s, 16.6s 6.5s, 5.6s
validate, warm cache 12.0s, 17.6s 4.2s, 5.1s
validate <one file>, warm cache 16.6s, 9.4s 2.5s, 2.6s

A single run of the final build agrees:

  • --no-cache generate: 14.9s down to 5.8s.
  • --no-cache validate: 16.4s down to 5.9s.
  • System CPU: about 77s down to 16s per run.

I didn't measure a clean checkout with this build. A clean checkout has little to skip, so CI times shouldn't change much.

Tests

  • Unit tests for tracked_directories, literal_glob_prefix and walk_enters.
  • tests/untracked_directory_test.rs, where each test fails when its half of the change is removed:
    • A vendored gem with no tracked files gets no CODEOWNERS line. This fails on main.
    • An untracked team file in a new directory under config/teams is still read. This fails without the team file exemption.
    • A vendored gem that's a submodule keeps its CODEOWNERS line. This fails if the set holds only the directories above tracked paths.
  • Existing tests: the fix: don't let git ignore rules override tracked-file inclusion #110 (global gitignore) and Consider new, untracked team config files during validate/generate #113 (untracked team file) tests pass unchanged.
  • Checks: cargo test (336 passed, 3 ignored), cargo clippy --all-targets --all-features -- -D warnings and cargo fmt -- --check.

Since #110 turned off git's ignore rules for the walk, it entered every
directory on disk outside the top-level ignore_dirs, and dropped the
untracked files in them one at a time. A large gitignored directory,
like build output or nested git worktrees, cost a full traversal even
though nothing in it could end up in the project.

The walk now skips a directory unless it holds a tracked file, could
hold a team file (those are read before they're tracked, #113), or is
the project root. A vendored gem directory with no tracked files no
longer gets a CODEOWNERS line, so that line no longer depends on the
checkout.

Fixes #152
git ls-files lists a submodule as one path, without its contents, so
the tracked directory set only held its parents and the walk skipped
it. A vendored gem that's a submodule then lost its CODEOWNERS line.
The set now holds each tracked path as well as the directories above
it.
@dduugg
dduugg requested a review from a team as a code owner November 13, 2026 21:04
@github-project-automation github-project-automation Bot moved this to Triage in Modularity Nov 13, 2026
@dduugg
dduugg enabled auto-merge (squash) November 14, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

The project walk enters directories that hold no tracked files

1 participant