rsconstruct 0.9.43

Rust based fast build system
# Architecture Review Findings

Full-codebase architecture audit (2026-07-18). Distinct from `CODE_REVIEW_FINDINGS.md` (line-level bugs, all fixed): these are structural/design problems. Checkboxes track future remediation work.

Recurring failure pattern across the codebase: **knowledge duplicated across hand-synchronized places with nothing enforcing agreement**, plus **advertised contracts the code doesn't implement**. The line-level code is careful; the seams between subsystems are where the debt lives.

## High severity

- [x] 1. **Schema has no single source of truth — drift has already happened.** *(Done 2026-08-01, by enforcement rather than by the `FieldSpec` refactor: five schema-consistency tests in `src/config/tests.rs` iterate every plugin and assert known-fields ⊆ type table, no dead type arms, checksum ⊆ known, descriptions ⊆ known. They found more drift than the audit listed — 19 dead arms (incl. `markdownlint.local_repo`/`npm_stamp`, not just the 8 named) and ~100 fields with no type validation at all. All fixed: `command`/`output_dir`/`formats` moved to the generic arm, ~40 processor-specific entries added (each verified against its struct declaration), `FieldType` gained `Table`/`Array`. The tests immediately caught a wrong type I introduced (`tags.sorted_tags` as StringArray), which is the point. **The `FieldSpec` consolidation itself is still worth doing** — the "no dead arms" check is probabilistic because `expected_field_type` is a match, not data; turning it into data makes that check total. Left as follow-up.)* One config field's schema lives in 5–8 hand-synced places (struct field, serde default fn, `Default` impl, `known_fields()`, `field_descriptions()`, `checksum_fields()`, `expected_field_type` arm in `src/config/mod.rs:1205`, docs page). Adding a processor also touches central tables (`scan_defaults_for` ~92 arms, `processor_defaults_for` ~71 arms, `src/config/mod.rs:511-692`). Proven drift: dead `expected_field_type` arms for renamed fields (`marp_bin`, `mmdc_bin`, `drawio_bin`, `chromium_bin`, `libreoffice_bin`, `markdown_bin`, `checker`, `linter`); marp `args` default encoded twice with different values (`MarpConfig::default()` vs `processor_defaults_for("marp")`); marp `timeout_secs`/`max_attempts` missing from type validation. Worst consequence: a forgotten `checksum_fields` entry on an output-affecting field = **silently stale build outputs**. Fix direction (no macros): one static `FieldSpec { name, ty, description, in_checksum, required }` table per config, carried on the `ProcessorPlugin` inventory entry, from which known-fields/types/descriptions/defaults all derive; one test iterating `all_plugins()` asserting coverage. Invert the checksum default (in unless opted out with a reason) so mistakes cause spurious rebuilds instead of stale outputs.

- [x] 2. **Three advertised contracts are not real.** *(Done 2026-08-01: dead fail-fast chunk-size arms deleted — `batch_chunk_size` now only models the real behavior, and architecture.md's three batching passages rewritten to state it (batching on by default, whole group per chunk, `--batch-size -1` for per-product recovery). Phase system reduced to what exists: `Phase` enum and the 5 never-fired phases deleted, `run_phase` → `run_post_config_hooks`, `rsconstruct phases` replaced by `rsconstruct hooks`. Stale docs re-synced: interrupt section (tokio watch channel reality), `ProductDiscovery` → the real `Processor` trait, cache.md layout (descriptors/ + the four redb files), descriptor-key formula (+processor_version, +variant), `output_depends_on_input_name` restated as a known limitation, `src/registry.rs` comment fixed, auto-detect table replaced with a pointer to `processors list`.)*
  - Fail-fast batching is dead code: both `None` arms of the chunk-size logic are unreachable (`src/executor/execution.rs:381-386,766` vs `src/builder/build.rs:272`); default fail-fast sends the whole level in one chunk and `execute_checker_batch` smears one exit code over every product — architecture.md promises chunk-size-1 incremental recovery that doesn't exist.
  - Phase hooks (`src/phases.rs`): 5 of 6 phases never fire; `run_phase` is called exactly once (PostConfig, `src/config/mod.rs`); hooks receive only `&mut Config` so pipeline phases couldn't observe the graph anyway. The pipeline is enumerated 4 uncoordinated times (`phases::Phase`, `cli::BuildPhase`, `GraphSnapshot`, string-keyed `phase_timings`).
  - Docs describe removed/nonexistent systems: architecture.md's interrupt section (global flag + 50 ms poll — code uses a tokio watch channel), a four-method `ProductDiscovery` trait (doesn't exist), cache.md's "no separate database / descriptors live in objects/" (there is `descriptors/` + `db.redb`), `output_depends_on_input_name` (zero hits in src/), and `processors/mod.rs:26` references a `src/registry.rs` that doesn't exist.

- [x] 3. **Cache-key composition is decentralized, with correctness holes behind it.** *(Done 2026-08-01: new `src/cache_key.rs` owns composition. The opaque `Product.config_hash: Option<String>` — spliced by four contributors using three different separator conventions — is replaced by `CacheKey`, an ordered list of named `(Component, value)` pairs (`Config`/`Analyzer`/`ToolVersion`/`Variant`) folded into the digest by one function. The component kind is mixed into the digest, so a config value and a tool value that happen to be equal no longer collide, and `variant` — previously spliced into `descriptor_key` separately — is now a normal component. Payoff: `rsconstruct product show` prints the components individually, so any descriptor-key change is attributable to exactly one contributor. **Tool-version keying is now on by default** (`[build] hash_tool_versions`, default true): `processor_tool_hashes` no longer returns an empty map without a lock file, and identifies tools by a content hash of the resolved binary — no `--version` subprocess, mtime-cached, and strictly stronger than a version string since it also catches a rebuilt binary reporting an unchanged version. A lock file, when present, still refines the identity. Integration tests pin both directions (upgrade invalidates; `hash_tool_versions = false` doesn't); the upgrade test was verified to fail against the old behavior, reproducing the exact "Skipping (unchanged)" symptom. B1/B2/B7 had already closed the other three holes.)* Key material assembled in ≥5 uncoordinated places (`Product::descriptor_key` `src/graph.rs:98`, `output_config_hash` allowlist, `extend_config_hash`, `apply_tool_version_hashes` — which uses a different combination style — and per-processor `checksum_fields`). Concrete holes:
  - Creator `output_dirs`/`output_files` are excluded from the config hash with a comment claiming outputs are "part of the product's identity" — true only of the legacy key. **Changing a creator's output_dirs = permanent silent SKIP.**
  - `cache = false` (`StandardConfig.cache`) is documented, validated, described — and read by no code. A documented no-op.
  - Tool-version keying is opt-in: without `tools lock`, upgrading a tool leaves all cached PASSes valid.
  - Legacy `CacheEntry`/`CACHE_TABLE` is never written but still consulted by status (`builder/build.rs` stale/new split) — every non-current product reports "new".
  Fix direction: a single `CacheKey` builder owning every component; delete legacy types; hash installed tool versions by default.

- [x] 4. **Testing is architecturally unenforced.** CI workflows contain no `cargo test` at all. ~150 of ~436 integration tests silently self-disable when a tool is missing (82 `tool_available()` early-return sites + the `test_checker!` macro's skip guards) — a skip is indistinguishable from a pass, in a project whose creed is "never silently skip." Near-zero unit tests in core modules: all of `src/executor/`, all of `src/builder/`, 8 of 9 `src/object_store/` files, `src/checksum.rs`, `src/watcher.rs`; everything is exercised through 569 spawns of the real binary. Fix direction: CI job with the full tool matrix + a `RSCONSTRUCT_TEST_REQUIRE_TOOLS=1` mode turning skips into failures; unit-test the pure cores (executor policy/retry, checksum, object-store validity/restore, run_checker chunking).
  *Done 2026-08-01:* skips are gone unconditionally — `tool_available()` replaced by a panicking `require_tool()` (no env-var mode; tests never skip, period), which immediately surfaced a broken venv black/a2x, a pdfunite probe that had always skipped, and a hadolint rule violation. CI test job added (`.github/workflows/test.yml`, registry-driven tool install + `cargo test` on every push). The named pure cores now have unit tests: `IncrementalPolicy::classify` decision table, `compute_parallel_levels`/`has_failed_dependency`, batch chunk-size + batching-gate decisions (extracted to free fns), `run_checker` chunk packing (extracted to `checker_chunk_ranges`), `execute_checker_batch*` fan-out semantics, checksum core (injection-proofing, buffer boundaries, MISSING/order semantics, per-context caching), object-store validity/restore (tiered blob verification, corrupted-output re-restore, tree order-insensitive change detection, read-only descriptor overwrite). Still integration-only: the retry loop body, `src/builder/`, `src/watcher.rs` — those unit tests are blocked on the finding 6/7 extractions.

- [x] 5. **No output layer — the `--json` contract is unenforceable.** *(Done 2026-08-01: new `src/output.rs` is the sink — `info`/`detail`/`warn`/`error`/`diagnostic`, each consulting the quiet/json predicate in one place, with the stdout-vs-stderr decision made by the function you call rather than at each site (`info`/`detail` → stdout; `warn`/`error`/`diagnostic` → stderr, so they survive `--json` without corrupting it). The entire build path — all of `src/executor/` and `src/object_store/` — is migrated and now contains **zero** bare `println!`. The key part is that this is now **enforced**: a unit test scans those modules and fails with the offending file:line if a bare `println!` reappears, which is what makes it a contract rather than the habit that had already failed (12 guards across ~445 sites). Verified the test catches a deliberately introduced violation. End-to-end, every stdout line parses as JSON across `--json`, `--json -v`, `--json --color=always`, `--json -v --color=always`. **Not done:** migrating the ~400 `println!` sites in command handlers (`tools list`, `sloc`, `processors files`, …). Those already branch on `is_json_mode()` and print their own JSON document, which *is* the machine output in that mode — they are not the leak risk, and rewriting them would be churn with no contract behind it.)* 437 `println!/eprintln!` sites across 32 modules; JSON gating checked in only ~17 files, `quiet()` at 8 sites. Verbose/colored lines leak into `--json` stdout today (`src/executor/execution.rs:408,523,735`; unconditional `println!` in `flush_words`), and the test harness tolerates it (`BuildResult::parse` discards non-JSON lines). `runtime_flags` itself is fine (write-once OnceLock); the missing piece is a single emit-sink. Fix direction: one `output` module (`emit_human`/`emit_event` consulting quiet/json/color), migrate call sites, and a test asserting every stdout line under `--json -v` parses as JSON.

## Medium severity

- [ ] 6. **God modules and inverted layering.** *(2026-08-02: the god-module half is done — `processors/mod.rs` 2,324 → 1,473, `graph.rs` 1,423 → 1,133, `main.rs` 971 → 840, and the `crate::cli` layering inversion is fixed. Two bullets were examined and rejected as findings (the "duplicated renderer set" is two different graphs; visibility is unenforceable in a binary crate). **Left open for one reason:** the `processors → config → registries` cycle is real and intrinsic, and breaking it properly means splitting the plugin entry into data and constructor halves — a deliberate restructuring, traced below rather than half-attempted.)*
  - [x] **Tool engine extracted.** `src/processors/mod.rs` 2,324 → **1,637** lines: the install engine, `InstallMethod`/`InstallCtx`, binary recipes and the `TOOLS` table moved to `src/processors/tools.rs` (711 lines incl. its 5 tests, which moved with it). Nothing there touches the `Processor` trait. Re-exported from `processors` so ~40 call sites keep reading `processors::TOOLS`.
  - [x] **Renderers extracted and the `cli` import inverted-layering fixed.** `src/graph.rs` 1,423 → **1,133**: the six renderers (dot/mermaid/json/text/svg/html) moved to `src/graph_render.rs`. Separately, the display types (`DisplayOptions`, `InputDisplay`, `OutputDisplay`, `PathFormat`) moved out of `cli.rs` into a neutral `src/display.rs` — they still derive `ValueEnum`, which never required living in the CLI module. `graph.rs` and `executor/mod.rs` now import `crate::display`, so **no core module imports `crate::cli`** any more. Verified all four text renderers still produce output. (`watcher.rs` still imports `cli::BuildOptions`; that is finding 7's shape, not this one's.)
  - [x] **`stats` extracted.** *(2026-08-02: `ProductTiming`, `BuildStats`, `ProcessStats` and `FailedProduct` moved to `src/processors/stats.rs`. **`processors/mod.rs` is now 1,473 lines, down from 2,324** across the two extractions. What remains in it is subprocess infra plus the processor contract — related enough that splitting `exec` out too would separate `run_command` from the trait that calls it, so the remaining mix is deliberate rather than pending.)*
  - **Module cycle: `processors → config → registries → processors` — not broken, and traced 2026-08-02 to say why.** Each edge is intrinsic, not accidental: `config` needs `registries` because per-processor validation (`known_fields`, defaults, must-fields) is defined by the plugin table; `registries` needs `processors` because a plugin's whole job is constructing `Box<dyn Processor>`; `processors` needs `config` because `Processor`'s one required method returns `&StandardConfig`. Moving the trait to a leaf module relocates the cycle rather than removing it — the trait *is* config-shaped. Genuinely breaking it means splitting the plugin entry into a data half (field tables, which `config` needs) and a constructor half (which needs `Processor`), so `config` depends only on the data half. That is a worthwhile change and a large one; it is the real content of this bullet, and it is not attempted here. Rust permits module cycles, so this costs comprehensibility, not compilation.
  - [x] **`main.rs`'s largest inline command body extracted.** *(2026-08-02: `Commands::Cache` was a 136-line inline match arm — by far the biggest and the specific example this bullet names. Moved to `src/builder/cache_cmd.rs`; `run()` keeps the dispatch line. `main.rs` 971 → 840 lines. All `cache` subcommands verified working end-to-end (size/stats/list/trim). The remaining arms are 47 lines and under, i.e. no longer outliers.)*
  - [x] **"Duplicated renderer set" examined and rejected as a finding.** *(2026-08-02: `builder/tools.rs`'s `tools_graph_dot`/`_mermaid`/`_text`/`_json`/`_svg` render a **tool → processor** graph; `graph_render.rs` renders the **input → product → output** build graph. Different data model, different nodes and edges — they share only DOT/Mermaid output syntax, which is the format, not logic. Unifying them would mean an abstraction over "things with nodes and edges" that buys nothing and obscures both. The one genuine duplicate — three copies of dot→SVG — was already collapsed into one under finding 13. `run_tools_command` is still long, but length alone is not the defect this bullet claimed.)*
  - **Visibility is convention-only — measured 2026-08-02 and left as is, deliberately.** 718 bare `pub` items against 93 `pub(crate)`/`pub(super)` in 38.5k lines. But this is a **binary** crate: nothing outside it can link against these symbols, so `pub` and `pub(crate)` are already equivalent to the compiler, and rewriting 718 declarations would be pure churn with no behavior change and no new enforcement. Real enforcement would need the layers split into separate crates in a workspace (`config`, `graph`, `processors`, `builder`), where `pub` starts meaning something — which is also the natural home for the cycle-breaking above. That is a restructuring decision about the project, not a defect to fix in place, and it should be taken deliberately rather than as audit cleanup.

- [ ] 7. **`Builder` is a god object; the pipeline is not a first-class thing.** *(2026-08-02: the pipeline split is done — `Builder::build` went from ~218 lines to 113, across three named phases, with a `BuildPlan` intermediate that did not previously exist. **Left open for one reason:** the reporter interface. Presentation is still inline `println!` in the execute half, so a daemon/LSP consumer can reuse `plan_build` but not `build`. Everything else this finding names is closed.)* One struct, impl blocks in 10 files, ~35 public entry points; `Builder::build` (~240 lines) mixes CLI semantics (alias expansion), config mutation, tool preflight, orchestration, presentation, and exit-code mapping. Nothing between "whole build" and "one product" is testable/reusable; a daemon/LSP mode (anticipated by `BuildContext`'s docs) would have to re-enter through the CLI-shaped surface. Fix direction: extract a `BuildPipeline` type (discover→analyze→resolve→classify→execute) with a reporter interface; `Builder` becomes the thin command dispatcher.
  - [x] **CLI-semantics prologue extracted.** The `-p`/`-x` resolution — alias expansion, unknown-name pooling, conflict rejection, and the `-x`-only case that synthesizes an include list from "everything minus excludes" — is now the free function `resolve_processor_filter`, cutting ~47 lines out of `Builder::build`. The payoff is the finding's own complaint: this logic had **no unit tests** because exercising it required standing up a `Builder`. It now has 5, covering each branch including both error paths. One of them caught that my first assertion assumed insertion order where `expand_aliases` sorts.
  - [x] **The pipeline is now a first-class thing.** *(2026-08-02: `Builder::build` split into three named phases — `apply_cli_overrides` (flags → config/context state), `plan_build` (discover → filter → preflight → graph → target filtering), and the classify/execute body that remains in `build`. **`build()` went from ~218 lines to 113.** `plan_build` returns a `BuildPlan { processors, graph, phase_timings }` — the intermediate this finding said did not exist, so nothing sat between "whole build" and "one product". Two things fell out that confirm the seam is real: `plan_build` needs only `&self`, not `&mut self` (all config mutation is confined to `apply_cli_overrides`), and the processor filter turned out to be consumed entirely within planning — the graph already reflects it — so it is no longer threaded through the execution half. **Not done:** the reporter interface. Presentation is still inline `println!`/`phases_debug` calls rather than an injected reporter, so a daemon/LSP consumer could reuse `plan_build` but would still inherit `build`'s stdout behavior. That is the remaining step, and it is now a much smaller one.)*

- [x] 8. **Processor trait + capability model accretion.** *(Done 2026-08-02: the two remaining sub-items are closed.*
  - *__`SimpleGenerator` is now generic__ over `C: AsRef<StandardConfig> + Serialize + KnownFields`, so "one extra config field" no longer costs a hand-rolled `Processor`. `execute_fn` receives `&C` rather than `&StandardConfig`, and a new `extra_tools_fn: Option<fn(&C) -> Vec<String>>` covers the other reason generators were hand-rolled: a required tool named by a config field. **pandoc is converted** and its whole `Processor` impl deleted. That conversion fixed a live bug rather than just moving code: pandoc's configured `pdf_engine` is a binary it shells out to, but the hand-rolled `required_tools` was the only thing declaring it, and nothing tied it to discovery — with `extra_tools_fn` it now appears in `tools list-configured` (verified: `xelatex [installed] (pandoc)`), so `tools install` can actually provision it. The 12 existing `SimpleGeneratorParams` sites carry an explicit `extra_tools_fn: None`. `SimpleChecker` is left concrete: **aspell was the cited motivation and does not fit** — it holds a `WordManager` and loads a custom dictionary, i.e. real state beyond config, so genericizing the config type would not have absorbed it.*
  - *__pdfunite/ipdfunite now honor the discovery contract.__ Both walked the filesystem via `find_dirs_with_ext` + `fs::read_dir`; both now group `FileIndex::query` results by parent directory. This was a correctness bug, not just a layering one: the walk saw neither `.rsconstructignore` nor virtual files from upstream processors, and re-ran the IO on every fixed-point pass. Verified end-to-end that ignoring a source directory now drops its product (3 products → 2). `find_dirs_with_ext`/`collect_dirs_with_ext` had no other callers and are deleted. **Not done:** `tags` still walks the filesystem — its `read_dir` sites are tag-database and output-tree management, not product discovery, so they are a different concern than the one this finding names.)*
  - [x] **The panicking `discover()` default and the duplicate accessors are gone.** `standard_config() -> Option<&StandardConfig>` is deleted: all 27 implementations returned the same `&self.config.standard` that the required `scan_config()` already returns, and the `Option` is what forced the `discover` default to `.expect()` — a required method wearing a default's clothes that panicked for any processor implementing one accessor but not the other. `discover()` now reads `scan_config()` and is a real default. `config_json()`'s default follows the same accessor.
  - [x] **`is_native` double-declaration is now pinned by a test.** A `SimpleGenerator` declares it in `SimpleGeneratorParams` (where it decides whether `config.command` is a required tool) and again in the plugin registration (where it drives `processors list`), with nothing linking them. `simple_generator_is_native_declarations_agree` compares the two for all 12 SimpleGenerators; verified it fails when either is flipped alone. (A broader "native ⇒ no required tools" invariant was tried first and rejected: it has real exceptions — tera renders in-process but its template functions shell out to `git`, and clippy is non-native yet runs `cargo` rather than its configured command.)
  - 15 methods, 2 required; ~~default `discover()` panics at runtime (`.expect(...)`) — a required method disguised as a default (38 impls override it); `scan_config()`/`standard_config()` are duplicate accessors~~; `discover_for_clean`/`tool_version_commands`/`config_has_fix` each have exactly one overrider.
  - Capability drift is live: `fix list` filters on static `can_fix` alone while `fix` also honors `config_has_fix` — a configured `script.fix_command` is fixable but invisible in the list (`builder/fix.rs:117` vs `:21-22`); `black` declares `can_fix: true` with no fix params (works by accident; `supports_fix_batch()` wrongly false, silently disabling batch fixing); `is_native` declared twice per SimpleGenerator.
  - ~~`SimpleChecker`/`SimpleGenerator` are hardwired to `StandardConfig`: one extra config field forces a ~100-line hand-rolled processor (pandoc, aspell).~~ *(Fixed for `SimpleGenerator`; `SimpleChecker` left concrete on purpose — see the first sub-item above.)*
  - ~~Discovery contract inconsistently honored: pdfunite/ipdfunite walk the filesystem directly (evading ignore rules and virtual files, re-running IO every fixed-point pass) despite architecture.md's "processors never walk the filesystem".~~ *(Fixed — both now go through `FileIndex`. `tags` still walks the filesystem, but for tag-database and output-tree management rather than product discovery, so it is a different concern than this bullet names.)*

- [x] 9. **Executor scheduling wastes parallelism structurally.** *(Closed 2026-08-02 on measurement — __the level-barrier critique does not reproduce__, and the real cost this finding was pointing at turned out to be elsewhere and is now fixed. Re-profiled after the `tool_version_hashes` fix (see "what is actually left" below): **9.39× effective parallelism on 16 cores**, and the task timeline shows the scheduler is already work-conserving — the second wave of products starts at 467 ms, precisely when the first task ends, with no idle gap at any slot. There is no barrier stalling ready work to remove. The longest task (761 ms) no longer dominates the 1,240 ms span either, so this is not the degenerate "one irreducible task" shape that made the first profile inconclusive. A ready-queue scheduler would buy ~0 ms on this graph while replacing the most correctness-critical scheduling code in the project. **Reopen only with a trace from a project whose graph is genuinely deep** (long producer→consumer chains, not a wide fan-out of independent files) showing measured idle slots — that is the shape the original critique describes and the one this profile cannot speak to. Related sub-claims left open on purpose: "needs rebuild" is still computed three times with intentional divergence, and estimate-vs-authoritative checksums are still the same type; both are clarity issues, not the parallelism defect this finding names.)*
  <br>*Original profile note (2026-08-02, before the tool-hash fix):* *(__not implemented, and the evidence argues against implementing it now.__ A 60-product/4-processor project (40 py + 20 md; ruff, black, mypy, markdownlint) traced with `--trace`: effective parallelism **8.43×** over the work span on a 16-core box, all four processors starting within 2 ms of each other and running concurrently, and the build ending exactly when its longest single task ends (mypy's 1.84 s batch). There is no level barrier to remove in this shape — the critical path __is__ one irreducible task, so a ready-queue scheduler would move the finish line by ~0 ms. What the trace did surface is unrelated to scheduling: **`tool_version_hashes` cost 1.573 s of a 3.489 s cold build (45%), serially**, dropping to 2 ms warm — so it is mtime-cached and correct, but a first build in CI (where the cache never survives) pays it in full. That is the real parallelism defect this project has today, and it is a caching/prefetch question, not a scheduler rewrite. **Recommendation: leave the level-barrier model alone; profile a project with a genuinely deep dependency chain (the shape this finding describes) before spending the expensive rewrite, and treat cold-start tool hashing as the separate, cheaper win.** Retained unchecked because the structural critique may still hold for graph shapes this profile does not cover.)* Level-barrier model: the longest product in a level stalls the entire next level even for products whose real dependencies finished; one thread per batch group; static contiguous chunk partitioning with no work stealing; a thread blocked on a `max_jobs` permit can't take other work. The adjacency lists/failure propagation/semaphores needed for a ready-queue scheduler already exist. This is the one *expensive* fix — gate it on a `--trace` profile of a real project. Related: "needs rebuild" is computed three times (classify, per-level, post-execution) with intentional divergence held together by comments; reify estimate-vs-authoritative checksums as distinct types.

- [x] 10. **Graph identity is maintained by convention.** *(Done — 2026-08-01 landed the canonical rebuild path and the transitive closure; 2026-08-02 closed the last two. **`retain_products` no longer corrupts the graph**: it was a bare `Vec::retain` whose only guard was a doc-comment saying the result was "only suitable for read-only iteration" — every product after the first removed one kept an `id` that no longer matched its index, so `get_product(p.id)` returned the wrong product and the lookup maps still referenced removed ids. It now goes through a new `rebuild_retaining`, extracted from `filter_by_targets` so the two share the one routine that knows how to drop products safely (previously each open-coded it and only one was correct). A test pins `id == index`, `get_product` round-tripping, and output ownership after a removal — verified to FAIL against the old `Vec::retain`. **The checker-identity asymmetry is gone**: a non-superset re-declaration was silently swallowed (`try_update_inputs`'s `false` was discarded, so the second declaration's inputs vanished and the checker ran on the first set) where the generator path hard-errors on the equivalent conflict; it now hard-errors too, naming the processor and both input sets, with a test. Same-primary-input checkers still collide by design — that is the dedup key doing its job, not a defect.)* ~~`Product.id == index` across three parallel Vecs, upheld manually~~ — now assigned inside `register_product` from the vector length, so the invariant holds by construction; `retain_products` deliberately leaves stale indexes (doc-comment guard only); ~~`filter_by_targets` rebuilds the world, duplicating `add_product`'s registration logic (two sites that must evolve in lockstep)~~ — both now call `register_product`, the single place that knows what "adding a product to the graph" means (a test asserts a filtered graph is indistinguishable from a directly-built one: every lookup index, the id invariant, and per-product state like `output_dirs`); ~~and takes **no transitive closure** — a `-t` build can keep a consumer while dropping its producer~~ *(fixed as B4)*. Checker identity `(processor, primary_input, variant)` is a second, weaker scheme: same-primary-input checkers collide by design, and a non-superset re-declaration is silently ignored where the generator path hard-errors. Fix direction: ~~one canonical rebuild path~~; explicit product identity keys; ~~close over upstream producers in target filtering~~.

- [x] 11. **Config pipeline fragility beyond the schema tables.** *(Done — 2026-08-01 closed two sub-items, 2026-08-02 closed the rest. Every bullet below is now resolved: four fixed, one closed by B3, and the toml_edit substitution rewrite closed as a deliberate decline rather than an omission — see its bullet for why reversing B9's shipped behavior would be a regression, not a fix.)*
  - [x] **`extract_var_names` no longer splits on `=`.** It parses the TOML and reads the `[vars]` table's keys. The line scanner split each line on the first `=`, so a multi-line array item like `"a=b"` recorded a bogus variable named `"a` — which made the undefined-variable check *too permissive*, since a genuinely undefined `${a}` would then validate. (The scanner's stated reason for being textual — "avoid parse errors on variable references" — didn't hold: an unsubstituted `${var}` is ordinary string content to a TOML parser.) Tests pin both the name extraction and the end-to-end consequence. B9 already closed the partial-reference half of this bullet.
  - [x] **`is_multi_instance` hard-errors on ambiguity.** `classify_section` returns a three-way `SectionShape` — `SingleInstance` / `MultiInstance` / `Ambiguous { colliding }` — instead of a bool that had to resolve every case. A section whose sub-table names collide with known config field names is now rejected at validation time, naming the exact keys and how to fix them, rather than being silently reinterpreted as single-instance. That silent path is what made adding a config field in a future release able to retroactively change how an existing user's config parsed. Unit tests cover all three shapes plus scan-field collisions; verified end-to-end.
  - [x] **Partial references — closed as "error", not "support".** *(2026-08-02: deliberately NOT implementing the toml_edit rewrite this bullet proposes. B9 already closed the actual hole — a partial reference is now a hard error naming the variable, and `docs/src/configuration.md` states a reference must be the entire quoted value. Making `"${base}/src"` interpolate would reverse a shipped, documented decision and silently change what existing configs mean, to satisfy an audit bullet written before that decision existed. The defect this bullet names — "flows through silently as a literal" — is gone either way; what remains is a feature request, and it should be raised as one rather than smuggled in as remediation.)*
  - ~~`is_multi_instance` heuristic: instance names colliding with any known field silently reinterpret the section; adding a field in a future release can retroactively change how an existing user's config parses~~; Lua plugins can never be multi-instance, no diagnostic.
  - ~~`--iset`/`--pset` are a second, weaker validation path: skips `must_fields`, the src_dirs rule, and the `max_jobs > 0` guard (`--iset x.max_jobs=0` bypasses the deadlock protection).~~ *(Closed by B3 — `apply_overrides` re-runs `validate_single_processor` over all instances. The "unreachable `CliOverride` provenance arm" sub-claim did not hold up and is left in place as defense-in-depth; see B3.)*
  - [x] **Redundant re-parsing reduced.** *(2026-08-02: `substitute_variables` parsed the same content twice — once via `extract_var_names` for the names, once for the values — and now parses once, with `var_names_of` reading the already-parsed document. The parse is kept non-fatal there so the textual undefined-variable scan still produces its specific message for a config that also has a syntax error. `build_span_map`/`build_global_span_map` were separate entry points that each re-parsed the same source, and `Config::load` called them in adjacent pairs: with a local overlay that was **four** `Document::parse` calls for spans alone. Replaced by a single `build_span_maps` returning both; the singles had no remaining callers and are deleted.)*
  - [x] **Provenance span invariant is now enforced, not just described.** *(2026-08-02: the invariant — substitution must never change the file's line count, or every provenance `rsconstruct.toml:<line>` below the change silently points at the wrong line — is stated once as a module doc on `config/variables.rs` and pinned by three tests: `line_preservation_invariant_holds` (whole pipeline, over the value shapes most likely to break it — multi-element arrays, tables, strings with embedded `\n`/`\t`), `remove_vars_section_preserves_line_count` (the blanking half, isolated), and `value_to_toml_inline_never_emits_newlines` (the function that would break it first, over every `toml::Value` variant). Both halves held in the current code; nothing had been stopping a future edit from breaking them. The span-walk multi-instance heuristic is unchanged — `classify_section`'s three-way `SectionShape` already made the parser's version authoritative and hard-erroring, so the remaining duplication is a read-only display path that cannot silently reinterpret a config.)*

- [x] 12. **Object store: two engineering standards side by side.** *(Done 2026-08-01. Descriptor atomicity, the explain-vs-rebuild divergence and the mtime-cache bypass in `needs_rebuild_descriptor` had been fixed earlier; this pass closed the rest. **Tree arms no longer ignore `output_paths`** — a declared output the descriptor never saw is now existence-checked, so adding an output to a product invalidates instead of silently skipping. **`restore_from_descriptor` was the last full-hash path**: it used `calculate_checksum` (whole-file read) where the sibling paths used the mtime-cached `verify_checksum` — that was the third redundant hash of every cached output byte per no-op build. Now routed through `verify_checksum`; `calculate_checksum` had no other caller and is deleted. **Remote pull is landed rather than removed**: `remote_pull` defaulted to true and was fully plumbed to the store while every fetch path sat behind `#[allow(dead_code)]`, so a configured remote was silently write-only and a populated bucket could never satisfy a restore. Descriptors are now pushed on write (previously only objects were, so a remote could never hold a complete entry) and pulled on a local miss via `get_descriptor_pulling`/`ensure_object`. Reporting paths (`--explain`, `product show`) use a separate non-downloading `object_available` so they agree with the build without the side effect of populating the cache; `can_restore_descriptor` is evaluated lazily in `classify` so skipped products don't drag objects down. Corrupt remote objects are rejected before admission (checksum) and malformed descriptors before storage (parse) — four unit tests over a `file://` backend cover cold-restore, pull-disabled, tree round-trip and corruption rejection. Vestigial `validity.rs` deleted. **Not done:** the Marker/Blob/Tree → one-`Vec<TreeEntry>` normalization, and the single-vs-multi-process decision; both are refactors rather than correctness holes now that the arms agree.)*
  - Blobs: atomic, format-tagged, read-only, verified (exemplary). Descriptors: plain `fs::write` over a read-only file with an acknowledged chmod-retry race; a torn descriptor permanently breaks `cache trim`. Apply the blob discipline (temp+rename) to descriptors.
  - Marker/Blob/Tree: ~20 match arms across 7 functions with real drift — `explain_descriptor` existence-checks where `needs_rebuild_descriptor` content-verifies (`--explain` can disagree with the build); Tree arms ignore `output_paths` entirely. Normalize to one `Vec<TreeEntry>` model (Marker = empty, Blob = single pathless entry).
  - Five persistence mechanisms (db.redb, mtime.redb, deps.redb, webcache.redb, JSON descriptors + blobs) with three concurrency models; the `db.redb` open-lock makes two concurrent rsconstruct processes impossible anyway, so the multi-process blob safety is unreachable. Decide single-process vs multi-process and align.
  - Remote cache is half a feature: blob push works (one exists-check + one upload subprocess per object, serial), descriptor push/fetch and object fetch are `#[allow(dead_code)]` — a populated bucket can never satisfy a restore. Land pull or error on the config.
  - Output verification bypasses the mtime cache: `needs_rebuild_descriptor` full-hashes every cached output byte via a function documented as "not hot-path", twice per no-op build (classify + per-level), third time on restore — catastrophic for cached trees like `.venv`. Route through `checksum_fast` or store `(mtime, size)` in tree entries.

- [x] 13. **Subprocess execution and exit codes.** *(Done 2026-08-01: ~26 bypassing spawn sites reduced to 2 documented exceptions. `run_command_with_stdin` added to the central path — it pumps the stdin write concurrently with draining stdout/stderr via `tokio::join!`, which is what the hand-rolled spawns needed a writer thread for; a unit test round-trips 1 MB through `cat` to pin that the deadlock is gone. Converted: aspell's stdin-feeding spawn, all **three** copies of dot→SVG (`processors::dot_to_svg`, `BuildGraph::to_svg`, `tools_graph_svg` — now one implementation), linux_module's `uname`, the tera analyzer's `git ls-files` (ctx threaded through `scan_hash_pieces`/`scan_template`), git clean + git ls-files in builder/clean, the dot probes in builder/graph, `tool_lock::query_tool_version`, and the seven pip/npm/gem/dpkg package probes in builder/tools + builder/doctor (collapsed into two helpers from seven copies of the same block). Routing `uname` through the checked path immediately surfaced the predicted drift: linux_module invoked it without declaring it, so it needed both a `required_tools()` entry and a TOOLS registry entry. The two remaining exceptions — tool installation (must inherit the terminal for `sudo`'s password prompt) and `open_file` (launches a detached viewer that must outlive rsconstruct, where `kill_on_drop` is exactly wrong) — are commented at the call site and documented in architecture.md, whose "all subprocess execution goes through run_command" claim is now accurate rather than aspirational. **Interrupt state consolidated**: the standalone `Arc<AtomicBool>` threaded through `build`/`watch`/`Executor::new` was set on the same line as `ctx.interrupt()` and only ever read OR'd with it — pure redundancy, and one call site constructed a throwaway always-false instance. Deleted; `BuildContext` is now the single source. Verified end-to-end that Ctrl+C still interrupts a running build and exits 130.)*
  - ~~`classify_error`'s fallback substring-matches the rendered error chain, which embeds subprocess stderr: tool output containing "unknown field" → exit 2; "interrupted" → 130. The typed `RsconstructError` path already covers the main producers and survives context wrapping — delete the string-matching fallback (keep the io::Error chain check), type the stragglers, default to BuildError.~~ *(Done 2026-08-01: the four load-bearing untyped producers — toml parse, serde deserialize, variable substitution, invalid-config aggregation, unknown shell — are now typed `ConfigError`s; the string fallback is deleted; unit tests pin that tool stderr can never influence the exit code.)*
  - Interrupt state is triplicated: standalone `Arc<AtomicBool>` threaded through signatures + `BuildContext.interrupted` + the tokio watch channel; the executor ORs the first two. Consolidate into BuildContext (its own docs say that was the plan).

## Low severity

- [x] 14. **Cross-platform support is aspirational.** *(Done 2026-08-02, resolved as **unix-only**. This wasn't a new decision: `docs/src/internal/rejected-problems.md:121` already recorded "RSConstruct is Linux-only. No platform compatibility code will be added" — the `#[cfg(not(unix))]` branches contradicted a decision the project had already made, and the release matrix has never contained a Windows target. All five non-unix branches in `platform.rs` deleted; the wrappers stay as named OS-call sites rather than platform switches. `get_mode` no longer returns `Option<u32>` — the `None` existed only to represent "not unix", so two object-store tests that guarded their mode assertions behind `if let Some(mode)` now assert unconditionally, which is what "tests never skip" requires. Descriptor `mode` stays `Option<u32>`: there `None` legitimately means unreadable metadata. Docs corrected: `binary-releases.md` advertised a `rsconstruct-windows-x86_64.exe` that no workflow has ever built; CLAUDE.md's philosophy bullet now forbids `#[cfg]` **including** in platform.rs, since a branch that cannot be compiled or tested is dead code; fast-stat.md's two "cross-platform" notes restated as the real Linux-vs-macOS split, which does still exist for io_uring. The unix assumptions the finding lists — `flock`, `/dev/null`, `$HOME`, apt — are now documented as deliberate rather than latent bugs.)*
- [ ] 15. **Watch mode discards `event.paths`** — full Builder + FileIndex + discovery + classify on every debounced event; latency scales with project size, not change size. Acceptable today; blocks future incrementality (same root as finding 7). *(Re-assessed 2026-08-02 after finding 7's split landed — **still not delivered, and the blocker turns out to be deeper than finding 7.** The split did help: `plan_build` is now a named phase that a caller could hand extra inputs to, which was not true before. But the actual obstacle is discovery's contract, not `Builder`'s shape: `build_graph_with_processors_impl` runs every processor's `discover` against a `FileIndex` rebuilt from disk, and each processor scans the whole index. Feeding it "only these paths changed" means teaching discovery to produce a *partial* graph and then reconciling that against the previous graph — deciding which products survive, which are stale, and which disappeared — which is a new design, not a parameter. The honest scope is: finding 7 was a prerequisite, not the whole blocker. `event.paths` is still used only to decide whether an event is ignorable (`watcher.rs:99`). Left open deliberately rather than half-built; the finding's own assessment — "acceptable today" — still holds.)*
- [x] 16. **Unbounded growth:** webcache stores full HTTP bodies keyed by URL forever (no TTL/eviction, reopens its DB per fetch, duplicates the object store's job); mtime.redb accumulates entries for deleted files. Fold webcache into the CAS; prune mtime entries during `cache trim`.
  *Done 2026-08-01:* webcache entries now carry a fetch timestamp and expire after `[cache] webcache_ttl_secs` (default 7 days; 0 disables the cache). This was a **correctness** bug as much as a growth one — with no expiry, a JSON schema that changed upstream was never picked up, and the only remedy was knowing `cache webcache clear` existed. The DB is opened once per process behind a `OnceLock` instead of `Database::create` per fetch (redb allows one handle, so the old code also serialized every webcache operation against itself). `rsconstruct cache trim` now reclaims three kinds of dead weight instead of one: unreferenced objects, mtime rows whose file no longer exists (`checksum::prune_mtime_cache`), and expired webcache entries — verified end-to-end reclaiming 4 mtime rows after deleting two inputs. `webcache list` gained Age/State columns. Integration tests pin both directions of expiry against a seeded cache. **Not done:** folding the webcache into the CAS. It is a different thing — keyed by URL with time-based invalidation, not by content hash — and putting it in the CAS would need a URL→content index that is itself the webcache. The TTL closes the actual defect.
- [x] 17. **Vestigial code:** `scan_root_valid` always returns true, called ~20 times; `ctx!` macro still marked "will be removed once migrated"; `main.rs` hand-resolves scan defaults for `terms` bypassing the defaults framework; fixed-point discovery loop hits its 10-pass cap silently without erroring or injecting the final pass's virtual files.
  *Done 2026-08-01:* `scan_root_valid` and its `checker_auto_detect_with_scan_root` wrapper deleted (22 references across 12 files). `ctx!` deleted — the comment was accurate, not stale: `errors::ctx()` is `#[track_caller]`, so the macro was never needed, and **the codebase now has zero `macro_rules!`**, making the project's stated goal real (CLAUDE.md's exception clause removed). The `terms`/`tags` hand-resolution is replaced by `instance_config_or_default()`, which also fixed a live panic — `rsconstruct tags check` crashed with "scan fields not resolved" in any project that hadn't declared `[processor.tags]`; the now-unused `resolve_scan` went with it. Discovery cap was fixed earlier as B10.

## Concrete bugs surfaced by this review (fixable independently of refactors)

- [x] B1. Creator `output_dirs`/`output_files` change never invalidates cache → permanent stale SKIP (finding 3) *(Done 2026-08-01: both fields added to `CreatorConfig::checksum_fields()`, matching `ExplicitConfig`; the "outputs are part of the product's identity" comment was true only of the legacy key.)*
- [x] B2. `cache = false` config field is a no-op (finding 3) *(Done 2026-08-01: field deleted — wiring it into the executor's hot path risked new bugs for a feature nobody could ever have used; reintroduce deliberately with tests if wanted.)*
- [x] B3. `--iset <x>.max_jobs=0` bypasses the load-time deadlock guard; overrides skip semantic validation generally (finding 11) *(Done 2026-08-01: `apply_overrides` re-runs `validate_single_processor` over all instances after applying; integration test pins the max_jobs=0 rejection. The "unreachable CliOverride provenance arm" sub-claim didn't hold up: `apply_output_dir_defaults` runs before overrides, so overridden values are never remapped — the arm is defense-in-depth, left in place.)*
- [x] B4. `filter_by_targets` takes no transitive closure — `-t` can drop a kept consumer's producer (finding 10) *(Done 2026-08-01: worklist closure over `dependencies` before the graph rebuild; unit tests cover single-hop, chains, and that consumers are not pulled in.)*
- [x] B5. `fix list` and `fix` disagree on what's fixable (`can_fix` vs `can_fix || config_has_fix`) (finding 8) *(Done 2026-08-01: shared `is_fixable` predicate used by both.)*
- [x] B6. `black` declares `can_fix: true` with no fix params — violates the documented invariant; batch fixing silently disabled for it (finding 8) *(Done 2026-08-01: `fix_prepend_args: ["--quiet"]` — same reformat-in-place behavior, now explicit, and batch fixing engages.)*
- [x] B7. Status stale/new distinction reads a table nothing writes — always reports "new" (finding 3) *(Done 2026-08-01: status now classifies via `explain_descriptor` on both paths — which also fixed the finding-12 asymmetry by making the Blob arm content-verify — and the legacy `CACHE_TABLE`/`CacheEntry` were deleted.)*
- [x] B8. `--json -v` leaks non-JSON lines onto stdout (finding 5) *(Done 2026-08-01: worse than reported — plain `--json` leaked build-phase lines with no `-v` at all, and `--color=always` put raw ANSI escapes on stdout. Fixed via a `human_output_enabled()` predicate replacing the quiet-only guards, forcing executor verbosity off in JSON mode, moving `--keep-going` error lines to stderr, and making JSON mode unconditionally disable color. Integration test asserts every stdout line parses as JSON across four flag combinations.)*
- [x] B9. `"${var}/suffix"` partial variable references pass through silently as literals (finding 11) *(Done 2026-08-01: post-substitution residual scan errors on any remaining `${...}` outside comments; docs now state a reference must be the entire quoted value.)*
- [x] B10. Fixed-point discovery non-convergence at the 10-pass cap is silent (finding 17) *(Done 2026-08-01: hard GraphError at the cap naming the still-adding processors; the cap is now `[build] max_discovery_passes` (default 10) per the config-knob rule.)*

## Suggested order of attack

*Original plan, annotated with where it stands as of 2026-08-02.*

1. ~~**Cheap + high value:** the `FieldSpec` consolidation (collapses findings 1, parts of 3 and 8); delete the fake contracts (dead batching arms, unfired phases, string-match exit fallback, legacy cache table) and re-sync the three stale docs; CI test job + strict-skip mode (finding 4).~~ **Done**, except the `FieldSpec` consolidation itself — finding 1 was closed by enforcement tests instead, and the data-table refactor is still worth doing (see finding 1's note).
2. ~~**The concrete bugs B1–B10** — each is small and independent.~~ **All ten done.**
3. **Structural:** ~~output sink (5)~~, ~~descriptor atomicity + Tree normalization (12)~~, Builder/pipeline split (7), ~~module extractions (6)~~ — **partly done**: the tool-engine and renderer extractions landed and the `crate::cli` layering inversion is fixed, but `exec`/`stats`, `main.rs`'s `run()`, and the `processors → config → registries` cycle remain, and the `BuildPipeline` split (7) has not been attempted.
4. ~~**Only with profiling evidence:** the ready-queue scheduler (9).~~ **Profiled 2026-08-02; the evidence argues against doing it** — see finding 9.

**What is actually left, in priority order:**

*(Updated 2026-08-02 after the sweep. Everything bounded is done; the three items below are all genuine design decisions rather than defects, which is why each is stated as a choice with its cost rather than a task.)*

1. **Split the plugin entry into data and constructor halves** — breaks the `processors → config → registries` cycle (finding 6) and is the prerequisite for a workspace split, which is the only thing that would make visibility mean anything (also finding 6). The largest remaining item and the one that unlocks the others.
2. **A reporter interface for the execute half** (finding 7) — `plan_build` is already reusable; `build` still prints inline. Small now that the phases are separated.
3. **Path-incremental watch mode** (finding 15) — needs discovery to produce a partial graph and reconcile it against the previous one. A new design, not a parameter.
2. ~~**Cold-start `tool_version_hashes` (surfaced by finding 9's profile)** — 45% of a cold build, serial, and CI never keeps the cache that makes it cheap.~~ **Done 2026-08-02.** `tool_identity` content-hashed every tool binary; with pandoc at ~200 MB and dozens of tools, that read hundreds of MB on every cold build. Now identified by `(path, size, mtime)` — the same identity assumption the mtime cache already makes for every project input, and still strictly stronger than a `--version` string (it catches a rebuilt binary reporting an unchanged version). Measured on a 60-product project: **`tool_version_hashes` ~1.49s → 0 ms; total cold build 2.83s → 1.16s (−59%)**; the integration suite went from ~72s to ~32s, since every test project paid the same cost. Both invalidation tests (`tool_upgrade_invalidates_cached_results`, `hash_tool_versions_false_ignores_tool_upgrade`) still pass. **Parallelizing was tried first and rejected on measurement** — it was *slower* on a cold page cache (~2.4s vs ~1.5s) because the threads contended on one large sequential read; the comment in `processor_tool_hashes` records why it stays serial.
4. **Finding 1's `FieldSpec` consolidation** — still worth doing as a refactor, but its *safety* argument is now covered. The gap was that `every_expected_type_arm_is_a_live_field` enumerates fields from the plugin table and so cannot see an arm keyed to a processor that no longer exists — that arm is simply never reached. `every_expected_type_arm_names_a_real_processor` (added 2026-08-02) closes it by recovering the arm names from the source text and checking each against the plugin registry; verified to fail, naming the offender, when a bogus processor name is spliced into an arm. Turning the match into data would still be cleaner, but it is now a readability change rather than a correctness one.

*Closed during the sweep and no longer listed above:* the `tool_version_hashes` cold-start cost (item 2 of the previous list — see finding 9), finding 10's checker-identity and `retain_products` items, finding 11's provenance span invariant, and finding 6's `stats` extraction and `main.rs` `run()` split.