Add Copier migration support and update template docs#393
Conversation
steve-downey
requested review from
bretbrownjr,
camio,
dietmarkuehl,
ednolan and
wusatosi
as code owners
|
A few reviewer notes to make the diff easier to scan:
Validation run locally:
|
|
I need to review this more thoroughly but on a first skim-through this looks good |
|
coverage: 100.0%. remained the same — steve-downey:copier into bemanproject:main |
|
Moved to Draft: Not high risk that anyone would merge, but there are a few round-trip issues I'm cleaning up, as well as docs and scripts on how to use the |
|
I do want to move from cookiecutter to copier, but I would really prefer to keep around the .exemplar_version file/mechanism, since I need it for some of my own internal tooling that keeps repositories up to date, which copier doesn't entirely replace for me. |
|
Reviewer Guide: Copier Automation and Testing These latest changes finalize the template maintenance pipeline to ensure the template never drifts from the reference implementation and doesn't break downstream users. What to look at: update_templates.py: A new automation script that synchronizes root codebase changes directly into the template Jinja files. You will notice an infra submodule bump and a CODEOWNERS tweak in this PR that aren't on main yet. I deliberately included these as the payload to test the Copier update machinery. To prove the update path, I executed a live copier update against the downstream transcode project. It safely delivered the infra changes without touching transcode's custom Catch2 logic. The project built flawlessly, passed its test suite, and I have successfully pushed the update. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Pull Request SummaryThis PR migrates the Exemplar templating from Cookiecutter to Copier. Over the course of stabilizing this migration, several underlying infrastructure and CI testing issues became hard blockers for the matrix pipeline. As a result, this PR includes some vital infrastructure fixes that are strictly outside the scope of Copier, but were integral to getting the 120+ CI checks consistently completely green. Here is the breakdown of the changes: 1. Primary Copier Migration (Core PR Scope)
2. Infrastructure & CI Matrix Fixes (Integral Fixes)To achieve a passing pipeline, the following underlying testing infrastructure bugs were patched out (which otherwise would have been separate PRs):
Merge Note: Once this merges and upstream |
ClausKlein
left a comment
ClausKlein
left a comment
There was a problem hiding this comment.
Is it really necessary that a copy of infra submodule is under templates?
It may be created with Beman-submodule, or not?
That's just capturing the commit sha into .exemplar_version using , yes?There's a post install hook for copier, too, so I think I can just move it over. |
I can change that. |
|
Sounds good |
Cookiecutter vs Copier cross-comparison as of ba2f9edGenerated a non-exemplar project with identical parameters using cookiecutter from Every remaining difference is either expected (Copier metadata) or an improvement. Full diff below. Summary
Exact diff (excluding
|
Replace the Cookiecutter-based template under cookiecutter/ with a Copier-based template under template/. The new layout uses copier.yml for declarative configuration and .jinja suffixed files for template rendering. Key changes: - Delete cookiecutter/ directory and its hooks/config - Add copier.yml with project questions, defaults, and post-copy tasks - Create template/ with Jinja2 versions of all project files - Add copier/check_copier.sh for round-trip validation - Add template/.copier-answers.yml.jinja so generated projects retain Copier metadata for future template updates - Update stamp.sh to use Copier instead of Cookiecutter - Document migration path in README and CONTRIBUTING The .copier-answers.yml persisted in generated projects enables copier update for template-driven maintenance going forward.
Rewrite the README to describe the Copier-based project lifecycle: quick-start generation with uvx, incubation workflow, transfer to the bemanproject org, and ongoing template updates. Add instructions for rebasing older exemplar clones onto a Copier-managed baseline and document the reconfiguration workflow using copier update --data. Update CONTRIBUTING.md with Copier template development guidelines and sync template/README.md.jinja with the new content.
Move cron schedule randomization into copier.yml questions with computed defaults so each generated project gets unique schedules. Remove the per-workflow randomization from the Jinja templates.
Add automation and CI infrastructure for maintaining and validating the Copier template: - copier/update_templates.py: synchronizes root codebase changes into the template/ Jinja files - copier/test_standard_project.sh: generates and tests template output across variants (GTest/Catch2, Modules ON/OFF) - copier/test_cmake_matrix.sh: validates template output against CMake versions 3.30 through 4.3.x Add two new CI jobs to ci_tests.yml: - copier-test: runs test_standard_project.sh on gcc-release - copier-cmake-matrix: runs test_cmake_matrix.sh across a version matrix, both guarded by generating_exemplar so they only run in exemplar itself Also fix EOF whitespace in vcpkg-release.yml.jinja and update portfile.cmake.in template for Copier variable syntax.
Document the template update loop, troubleshooting steps, and Copier maintenance workflow for project maintainers.
Update infra to bemanproject/infra@7b66b85 which simplifies the experimental import-std UUID rules into concise version-range blocks covering CMake 3.30 through 4.3.x. Sync the template copy of enable-experimental-import-std.cmake and .beman_submodule with the new upstream state.
Fix discrepancies that check_copier.sh catches between the Copier
template output and the actual exemplar repo:
- Bump template version from 2.4.0 to 2.4.1 in CMakeLists.txt.jinja
- Update infra-workflows refs from @1.7.2 to @1.7.3 in ci_tests and
pre-commit-update templates
- Fix {%- endraw %} to {% endraw %} in pre-commit-update and
vcpkg-release templates to preserve trailing newlines
- Add blank line between cron schedule and concurrency in ci_tests
- Update README badge format to multi-line clickable badges matching
main's style
- Exclude .claude/ from check_copier.sh diff
check_copier.sh now passes cleanly for both the exemplar self-test
and the non-exemplar project generation.
Restructure template/README.md.jinja so the heading, badges, description, license, and usage sections are shared between exemplar and non-exemplar output, matching what cookiecutter produced. Previously the non-exemplar branch rendered only "TODO" with no heading or badges. Guard exemplar-specific paths in .gitattributes.jinja (template/, copier/) and .pre-commit-config.yaml.jinja exclude pattern so generated non-exemplar projects don't reference directories they don't contain. Fix extra blank line in CONTRIBUTING.md.jinja rendering and remove the include-path NOTE that main deliberately removed.
beman-tidy requires SPDX-License-Identifier in all files within the first 25 lines. Add the header to the answers file template so generated projects pass beman-tidy out of the box.
4 participants
Summary
This switches exemplar's template workflow over to Copier in a way that is usable for both new stamped repositories and older forks that started life as a plain GitHub template copy.
Copier is a modern project templating tool for keeping a generated repository tied back to its source template. In practice, that gives us two things we want here:
copier.ymlcopier updatelaterWhy
The previous flow was effectively a one-shot stamp-out process. It could generate a project, but it did not leave enough metadata behind for template-driven updates afterward.
This change closes that gap by making stamped projects retain the information Copier needs to understand:
That lets older exemplar-based repositories be rebased onto a Copier-generated baseline and then participate in template updates going forward.
What changed
.copier-answers.ymltemplate so generated repositories keep Copier metadatastamp.shand local template validationParity
At this point the Copier-based workflow is functionally at parity with the prior cookiecutter-style stamping flow:
copier updatesupportSo this is not a reduction in template capability; it preserves the existing stamp-out behavior and adds an update path that the earlier flow did not provide.
Validation
./copier/check_copier.sh