Skip to content

Phase 4's rubric scores the diff only, so PR-description obligations like CLAUDE.md's retrofit note go unchecked #120

Description

@dmccoystephenson

Gap found during a Phase 9 self-audit

Filed here rather than on dmccoystephenson/create-dev-loop-dev-loop, which does
not resolve — the same note carried by #109, #116 and #118.

What was observed

The Phase 4 self-review rubric scores the diff. Every one of its items —
Scope, Tests-new, Tests-fix, Sibling structure, Sibling renames, Docs, Issue
resolution, the external-signal item — asks a question about files that changed.
Nothing in the rubric asks whether the PR description satisfies the
obligations a repository's CLAUDE.md places on it.

This repository places two such obligations, and both were reached during the
cycle that produced PR #119 only by adversarial reading, not by the rubric:

  • Research grounding. CLAUDE.md requires that a PR changing the template
    cite the relevant RESEARCH.md finding, or state explicitly that none applies.
    This one is covered, but only because the repo-specific rubric block happens
    to carry a RESEARCH.md citation item — an instance-level patch, not a
    template rule.
  • Retrofit notes. CLAUDE.md's "Promoting a rule into the template" section
    requires noting in the PR description which existing generated skills likely
    need a retrofit pass. Nothing checks it. The first version of PR Harden Phase 8's rebase fence and state the merge-authorization precedence rule #119's
    description omitted the note entirely for three template-wide rules, and the
    omission was caught during self-review only because that step was being read
    adversarially rather than scored. A rubric that had been followed literally
    would have returned all-PASS on a description that violated a stated project
    convention.

Note where the retrofit obligation reaches a generated skill from: it arrives via
{{CODE_PATTERNS}} in Phase 3, which is the implementation phase. The thing
it governs is produced in Phase 4. An instruction delivered one phase before
the artifact it constrains, with no checkpoint at the artifact, is a rule that
depends on the agent remembering it.

Why this is proposed as template-wide rather than repo-specific

Nothing in the mechanism is specific to create-dev-loop. Any repository whose
CLAUDE.md, CONTRIBUTING.md, or PR template imposes a requirement on the PR
description
— a required section, an issue-reference format, a changelog line, a
downstream-impact disclosure — reproduces it, because the rubric's unit of
analysis is the diff and a description is not part of the diff. The
gardener-interface disclosure this repository's own CLAUDE.md asks for under
"What belongs here vs. in gardener" is a second local instance of the same shape:
a breaking change to Steps 3, 5 or 6 must be flagged in the PR description, and
no rubric item would notice its absence.

Suggested direction

Add a universal rubric item to Phase 4, alongside the existing ones, of roughly
this shape:

  • PR description obligations: every requirement the repository places on
    the PR description itself — required sections, downstream-impact or
    breaking-change disclosures, citations, retrofit notes — is present. Score
    this by re-reading CLAUDE.md/CONTRIBUTING.md/the PR template against the
    posted description, not from memory of what was intended.

Scoping it as "re-read the source documents" rather than enumerating specific
sections keeps it correct across repositories whose conventions differ, and
matches how the neighbouring Docs item is already phrased.

Relationship to the existing rubric

This is additive and does not disturb any existing item, so it carries no phase
renumbering risk. It would, however, be a template change and therefore itself
subject to the anchor gate described in #109.

This issue body was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

Metadata

Metadata

Assignees

No one assigned

    Labels

    template-ruleShould be promoted into create-dev-loop.md

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions