Fix npm workspace members named build/dist skipped (#1627)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1627

Summary

The npm-family crawler skipped every directory named dist, build, coverage, tmp, temp, __pycache__ or vendor, even when the root package.json workspaces field declares it as a member. npm, yarn and Bun install a member's unhoistable dependencies into <member>/node_modules from the root lock. For a member at packages/build, packages/dist, vendor/lib or tools/build, that copy was never crawled. Agent apply left it unpatched, apply --check said "in sync", and vex attested not_affected for a vulnerable copy that runs in production.

The crawler now also crawls the node_modules of those declared members. Undeclared build-output and cache dirs stay skipped.

Root cause

NpmCrawler::workspace_children (crates/socket-patch-core/src/crawlers/npm_crawler.rs) filters children by name against SKIP_DIRS. Nothing consulted the workspaces patterns, so a declared member with one of those names was pruned along with real build output.

Fix

• Bugbot (fixed in a8326e5): literal-prefix components are now checked for CACHEDIR.TAG, and the workspace root itself is exempt, as in the main walk.
• New skipped_workspace_member_roots sits next to configured_install_roots and is merged into find_local_node_modules_dirs (and into its test oracle, like the configured roots). The workspace roots it reads are the scan root and the project of every node_modules the walk found, so nested workspaces count. For each root it reads package.json workspaces with the existing governing_root::workspace_patterns parser. It walks each pattern from its literal prefix, to the depth the pattern allows (** = any depth, within the existing 20k-dir budget per pattern), and keeps dirs that:
  • cross a SKIP_DIRS name,
  • are listed by the patterns in any package manager's reading (workspace_globs::workspaces_include_as over WorkspacesDialect::ALL, which covers braces, extglobs and negation), and
  • hold a package.json.
• The member walk keeps the main walk's rules: real dirs only (no symlinks), no hidden dirs, no node_modules, no CACHEDIR.TAG-tagged dirs. Names come from directory listings, so on a case-insensitive disk a pattern spelled packages/build for an on-disk packages/Build (which the walk doesn't prune) is not listed twice.
• docs/ecosystems.md ("npm: which node_modules trees are crawled") documents the exception.

The npm / PyPI / gem wrappers only dispatch to the binary, so they need no change.

Tests (red → green)

Issue	Test	On main	With fix
#1627 apply	in_process_alternate_installers::apply_patches_workspace_member_named_like_a_build_dir (members build, dist, vendor, tmp, temp, coverage)	FAILED: packages/build/node_modules/ms/index.js unpatched	ok
#1627 apply --check + vex	in_process_alternate_installers::apply_check_and_vex_see_unpatched_member_named_like_a_build_dir	FAILED: --check exit 0, expected 1	ok (exit 1; vex has no not_affected until the member copy is patched, then attests it)
#1627 crawl, every reported layout	crawler_npm_e2e::crawl_all_finds_declared_members_with_skip_dir_names (packages/*, vendor/*, literal tools/build, braces, {packages:[…]} + **, yarn berry !(docs)/*)	FAILED: build/dist members missing	ok
nested workspace root	crawler_npm_e2e::crawl_all_finds_skip_dir_members_of_a_nested_workspace	fails (member pruned)	ok
tagged scan root (Bugbot)	crawler_npm_e2e::crawl_all_finds_members_of_a_tagged_scan_root	FAILED on the previous head	ok
controls	crawl_all_still_skips_undeclared_skip_dirs, crawl_all_member_walk_keeps_the_walk_rules (glob and literal members under a tagged cache dir, case-mismatched pattern; its literal-prefix case failed on the previous head), the existing crawl_all_skips_hidden_and_skip_dirs	ok	ok

Local runs:

• cargo fmt --all -- --check: clean
• cargo clippy --workspace --all-features -- -D warnings: clean
• cargo test -p socket-patch-core --all-features: 7161 passed, 4 failed. The 4 (copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_…, pypi_requirements::wire_failure_rolls_back_…) are permission-denial tests that can't fail as root, and this container runs as uid 0. They're in modules this PR doesn't touch, and CI runs them as non-root.
• cargo test -p socket-patch-cli --all-features --test in_process_alternate_installers: 40/40. --test in_process_agent_reapply: 27/27.
• The full cargo test --workspace --all-features build ran out of this session's disk allowance (more than 19 GB of test binaries), so CI runs the rest of the CLI suites.

Review notes (/code-review high)

I fixed these findings: a budget shared across patterns, CACHEDIR-tagged dirs not pruned, nested workspace roots ignored, a case-insensitive duplicate, !(..) extglob patterns dropped, a duplicated workspaces parser, a no-op dedup loop, and a vacuous VEX assertion (apply --check never writes VEX, so the test now runs vex itself with a manifest that has a vulnerability).

Not changed, and why:

• Exempt SKIP_DIRS inside the main parallel walk instead of a second walk. Deciding there whether a pattern could match something below a pruned dir needs prefix matching over the whole glob grammar (braces across /, extglobs, picomatch). That grammar lives in workspace_globs and isn't built for prefix matching. The separate walk reuses the exact membership check, and it costs one package.json read per workspace root plus one listing per pattern prefix. A ** pattern can cost more, but only up to the per-pattern budget.
• pnpm pnpm-workspace.yaml members under node-linker=hoisted. Out of scope for this npm report. pnpm's default isolated layout keeps copies in the root .pnpm store, which is already crawled. Follow-up if a hoisted-pnpm repro turns up.
• Budget exhaustion is silent. Before this change none of these members were crawled, so a truncated ** walk is still strictly more coverage than main. Refusing (as the Bun helper does) would need a new error path through the agent crawl.

https://claude.ai/code/session_01QE3Va4u8ZT15HpxzAULNcG

---

Note

Medium Risk
Changes which npm node_modules trees are discovered, directly affecting patch apply, drift check, and VEX attestation in monorepos.

Overview
Fixes #1627: declared npm/yarn/Bun workspace members whose folders are named like build output (packages/build, vendor/lib, etc.) were skipped by the crawler, so nested node_modules there were never patched, --check could look clean, and vex could claim not_affected incorrectly.

The crawler now adds skipped_workspace_member_roots, which reads root workspaces patterns (via workspace_patterns, now pub(crate)), walks each pattern with the same pruning rules as the main tree (including CACHEDIR.TAG on literal prefixes), and includes <member>/node_modules when a listed member crosses a SKIP_DIRS segment and has a package.json. Those roots are merged into find_local_node_modules_dirs (and the test oracle). Undeclared dist/build/… trees stay pruned.

Tests cover agent apply, apply --check, vex, and crawl_all for globs, braces, nested workspaces, and tagged roots. docs/ecosystems.md documents the workspace exception.

^{Reviewed by Cursor Bugbot for commit a8326e5. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a8326e5. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Labeled Ready for review (burn-down agent).

• Head: a8326e5
• CI: 54/54 green (35 success, 19 skipped). No failures on main.
• Bugbot: reviewed a8326e5. Its one finding (npm_crawler.rs:181) was fixed earlier and the thread is resolved. 0 open threads.
• Mergeable with no conflicts. CHANGELOG.md is not touched.
• Reviewer focus: the npm workspace walk no longer skips declared workspace members named build/dist (npm_crawler.rs, governing_root.rs).

Slack announcement not sent this run (the Slack send tool was unavailable). The next run will retry.

---

Generated by Claude Code