fix(shared-infra): record skipped files in speckit.manifest.json

[eldar702]

Summary

In install_shared_infra (src/specify_cli/shared_infra.py), the skip branches in both the scripts loop and the templates loop now record each skipped file in speckit.manifest.json, so a fresh-manifest run against an already-populated .specify/ tree no longer writes an empty files field.

Fixes #2107

Problem

When install_shared_infra ran with force=False against a project that already had files under .specify/scripts/ and .specify/templates/ — but a fresh, empty manifest — every iteration hit the if dst.exists() and not force: skipped_files.append(...); continue branch. planned_copies and planned_templates stayed empty, the post-loop record loop had nothing to record, and manifest.save() serialised files: {}. The integration then believed nothing had been installed.

This bites users who delete or lose speckit.manifest.json, who extract .specify/ out-of-band, or who hit a code path where the manifest can't be loaded but the directory tree is intact.

Solution

Call manifest.record_existing(rel_skip) from inside both skip branches, but only when the path is not already tracked:

if dst_path.exists() and not force:
    rel_skip = dst_path.relative_to(project_path).as_posix()
    skipped_files.append(rel_skip)
    if rel_skip not in manifest.files:
        manifest.record_existing(rel_skip)
    continue

The guard matters: record_existing always re-hashes the on-disk content, so without it a customized template would have its manifest hash overwritten with the customized hash, defeating the user-modification detection that integration use relies on (test_use_preserves_modified_templates_unless_forced is the canonical regression test for that flow).

Symmetric change in both the scripts loop and the templates loop.

Test plan

• New regression test TestSpeckitManifestRecordsSkippedFiles::test_install_shared_infra_records_skipped_files — populates .specify/ via a normal first run, deletes the manifest, runs install_shared_infra again with force=False (every file goes down the skip branch), asserts the saved manifest is non-empty and is a superset of the first run's files. Fails on main, passes after the fix.
• Existing test_use_preserves_modified_templates_unless_forced continues to pass thanks to the if rel_skip not in manifest.files: guard — customized templates keep their original hash, so integration use still skips overwriting them.
• pytest full suite: 2787 passed, 34 skipped — zero regressions.

Notes

This change was AI-assisted. The fix was selected after a 3-agent solution-design debate where two designers independently identified the same root cause (skip branch in install_shared_infra failing to record), giving high convergent confidence. A third designer initially proposed a fix in claude/__init__.py, but that targets the integration manifest (claude.manifest.json), not the shared-infrastructure manifest (speckit.manifest.json) the issue describes — convergent analysis correctly localized the right file.

A first attempt at the fix omitted the if rel_skip not in manifest.files: guard and broke test_use_preserves_modified_templates_unless_forced (customized templates were silently re-hashed). Adding the guard fixed the regression while still recovering from a lost-manifest scenario.

[Copilot]

Pull request overview

This PR fixes a shared-infrastructure tracking bug where install_shared_infra(..., force=False) could skip all existing .specify/scripts/ and .specify/templates/ files without recording them, resulting in speckit.manifest.json being saved with an empty files mapping (issue #2107).

Changes:

• Record skipped (already-existing) shared scripts/templates into speckit.manifest.json during install_shared_infra when they are not already tracked.
• Add a regression test ensuring a “lost manifest” reinstall reconstructs a non-empty manifest that includes all previously tracked files.

Show a summary per file

File	Description
src/specify_cli/shared_infra.py	Records skipped existing shared infra files into the shared manifest during install.
tests/integrations/test_integration_claude.py	Adds a regression test covering the “lost manifest + skip branch” scenario.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/specify_cli/shared_infra.py:303

• Same issue as the scripts loop: calling manifest.record_existing(rel_skip) in the skip branch will raise if dst exists but is not a regular file (e.g., a directory). Add an dst.is_file() guard or raise an explicit error so a non-file collision doesn’t fail with an opaque hashing/open error.

                # ``.specify/`` tree does not silently drop it (#2107).
                # Skip if already tracked — preserving the original hash
                # keeps user-modification detection working downstream.
                if rel_skip not in manifest.files:
                    manifest.record_existing(rel_skip)

• Files reviewed: 2/2 changed files
• Comments generated: 2

[mnriem]

Please address Copilot feedback

[eldar702]

Thanks for the review! Pushed 6790654 to address all three Copilot findings.

What changed

1. Hardened IntegrationManifest.record_existing (the choke point)

The function's docstring already promised "an already-existing file", but didn't enforce it. It now raises ValueError if the path is a symlink or is not a regular file — mirroring the existing _ensure_safe_manifest_destination precedent in the same module. The is_symlink() check runs on the un-resolved path because _validate_rel_path calls resolve(), which would otherwise silently follow the symlink and record the target.

2. Guarded both call sites in install_shared_infra (the bug Copilot flagged at lines 277 + 303)

if dst_path.is_file() and rel_skip not in manifest.files:
    try:
        manifest.record_existing(rel_skip)
    except (OSError, ValueError) as exc:
        console.print(
            f"[yellow]⚠[/yellow]  could not record {rel_skip} in manifest: {exc}"
        )

Belt-and-suspenders: the is_file() guard short-circuits the common case (directory collision), and the try/except absorbs the rare TOCTOU race or permission error so a single weird path can't abort the whole install. The path still surfaces via the existing skipped_files warning at the bottom of the function.

3. Tightened the test helper (Copilot's low-confidence finding)

_read_manifest_files no longer falls back to data.get("_files"). It now asserts "files" in data and that the value is a dict, with descriptive failure messages. A future schema regression that renames the public key now fails red instead of being silently masked.

4. New regression tests

Two new methods in TestSpeckitManifestRecordsSkippedFiles plant a directory at the expected destination of a script and a template, then assert (a) install completes, (b) the path is NOT recorded in the manifest, (c) the path still appears in the user-visible warning.

5. Cosmetic

The user-facing warning string "shared infrastructure file(s) already exist" → "shared infrastructure path(s) already exist", since non-file entries can now legitimately land in skipped_files.

Verification

• All 1950 integrations tests + the new tests pass: pytest tests/integrations/ → passed
• Full repo: pytest tests/ → 2789 passed, 34 skipped
• Adversarial unit-checks confirm record_existing raises ValueError for directory, symlink, and out-of-root path; succeeds for a regular file.

Out of scope (intentionally)

• refresh_shared_templates (line 212 area) — its record_existing call runs after _write_shared_text, never on the skip branch, so it is not affected by this bug class.
• Pre-existing behavior when force=True and a directory occupies a destination — outside the scope of [Bug]: speckit install creates manifest with empty files — no skills available #2107.