feat: schema generation by gennaroprota · Pull Request #1178 · cppalliance/mrdocs

gennaroprota · 2026-04-16T15:07:10Z

This adds a --schemas[=<dir>] option that emits two schemas derived from the reflection metadata in the source tree: mrdocs-dom-schema.json, which describes the Handlebars DOM produced by the generator, and mrdocs.rng (RelaxNG), which describes the XML output. The hand-written mrdocs.rnc is removed and replaced by the generated .rng everywhere CI used it, eliminating drift between the schema and the actual output.

The schemas are produced by walking the reflection types and member descriptions in the source tree, so they stay in sync with the code automatically as new metadata types are added. The DOM reference documentation is switched from a hand-maintained page to one generated by an Antora extension that reads the JSON schema.

A generic XML emitter is factored out of the existing XMLTags / XMLWriter code into src/lib/Support/Xml.{cpp,hpp}, and several reflection types (AccessKind, ConstexprKind, OperatorKind, StorageClassKind, ParamDirection, TypeKind) are made describable through new MapReflectedType / MergeReflectedType / TypeTraits headers in include/mrdocs/Support/. Three small fixes ride along: OperatorKind is serialized as a string in the DOM (not the underlying enum), the XML emitter no longer drops template args / noexcept / explicit specifiers, and asserts fire when a reflection type or member lacks a description.

Changes

Source: New src/lib/Schemas/ with DomDescriptions.hpp, DomSchemaWriter.hpp, RngSchemaWriter.{cpp,hpp}. Generic XML emitter extracted into src/lib/Support/Xml.{cpp,hpp} and consumed by Gen/xml/XMLTags and XMLWriter (which shrink correspondingly). --schemas wired through tool/ToolArgs.{cpp,hpp} and tool/ToolMain.cpp. Several specifier headers in include/mrdocs/Metadata/ refactored alongside new MapReflectedType, MergeReflectedType, and TypeTraits headers in include/mrdocs/Support/. Schema-writer headers live in src/lib/Schemas/ (deliberately kept out of the public API).
Tests: New src/test/Schemas/DomSchemaWriter.cpp covering the JSON schema writer end-to-end.
Golden tests: ~286 XML fixture diffs reflecting the newly emitted attributes (template args, noexcept, explicit) that the old XML writer was silently dropping. These are intentional output corrections.
Docs: New docs/modules/ROOT/pages/schemas.adoc documents --schemas. docs/extensions/dom-reference.js renders the DOM reference page from the JSON schema; the corresponding hand-written section was removed from generators.adoc. mrdocs-dom-schema.json and mrdocs.rng are also linked as downloadable attachments.
Build: mrdocs.rnc removed; CMakeLists.txt updated so mrdocs.rng and mrdocs-dom-schema.json are emitted and verified during the build. The Java prerequisite (previously needed to process mrdocs.rnc) is removed.
Breaking changes: The XML schema artifact distributed with mrdocs is now a .rng instead of .rnc: downstream tooling that consumes the schema file by name will need to update. The generated XML output now includes template args, noexcept, and explicit attributes that were silently omitted before; consumers parsing the XML and not expecting those attributes may need to adjust.

Testing

src/test/Schemas/DomSchemaWriter.cpp exercises the JSON schema writer against a fixed set of reflection types.
The committed mrdocs.rng and mrdocs-dom-schema.json files are paired with two CI verification steps that regenerate them from the current source and fail if the on-disk version drifts from the regenerated one. This keeps the schemas in sync with the code on every build rather than relying on a contributor remembering to regenerate.
The ~286 golden-test XML diffs serve as regression coverage for the newly emitted attributes: any future omission of template args / noexcept / explicit will cause the golden tests to fail.
An assert fires when a reflection type or member lacks a description, so any future addition to the reflection metadata that ships without schema metadata trips immediately in CI rather than silently producing an under-specified schema.

Documentation

Two pieces:

docs/modules/ROOT/pages/schemas.adoc documents the new --schemas option.
docs/extensions/dom-reference.js is an Antora extension that renders the DOM reference from mrdocs-dom-schema.json; the hand-written DOM section in generators.adoc was removed in favor of this generated page, so DOM-reference docs now stay in sync with the code automatically.

github-actions · 2026-04-16T15:07:34Z

✨ Highlights

🧪 Existing golden tests changed (behavior likely shifted)

🧾 Changes by Scope

Scope	Lines Δ%	Lines Δ	Lines +	Lines -	Files Δ	Files +	Files ~	Files ↔	Files -
🥇 Golden Tests	54%	16141	9945	6196	287	-	287	-	-
📄 Docs	29%	8567	7737	830	10	7	3	-	-
🛠️ Source	11%	3273	2925	348	32	6	26	-	-
🏗️ Build	3%	988	26	962	2	-	1	-	1
🧪 Unit Tests	1%	320	320	-	1	1	-	-	-
🔧 Toolchain	1%	244	57	187	9	-	8	-	1
🔧 Toolchain Tests	1%	221	20	201	7	-	7	-	-
🧰 Tooling	<1%	36	2	34	3	-	1	-	2
⚙️ CI	<1%	2	2	-	1	-	1	-	-
Total	100%	29792	21034	8758	352	14	334	-	4

Legend: Files + (added), Files ~ (modified), Files ↔ (renamed), Files - (removed)

🔝 Top Files

docs/mrdocs-dom-schema.json (Docs): 4978 lines Δ (+4978 / -0)
test-files/golden-tests/symbols/record/class-template-specializations-1.xml (Golden Tests): 3903 lines Δ (+2718 / -1185)
docs/mrdocs.rng (Docs): 2532 lines Δ (+2532 / -0)

Generated by 🚫 dangerJS against 019d019

codecov · 2026-04-16T15:08:19Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.03%. Comparing base (ab5f412) to head (019d019).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1178      +/-   ##
===========================================
- Coverage    82.12%   82.03%   -0.09%     
===========================================
  Files           33       32       -1     
  Lines         3149     3090      -59     
  Branches       734      714      -20     
===========================================
- Hits          2586     2535      -51     
+ Misses         387      385       -2     
+ Partials       176      170       -6

Flag	Coverage Δ
bootstrap	`82.03% <ø> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cppalliance-bot · 2026-04-16T15:14:09Z

An automated preview of the documentation is available at https://1178.mrdocs.prtest2.cppalliance.org/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-05-14 07:44:08 UTC

alandefreitas · 2026-04-21T05:49:58Z

:)

gennaroprota · 2026-04-21T08:38:08Z

Path coverage is OK, isn't it? I added the PR description.

alandefreitas

What about documentation? How do I use the feature? What about the schema in the documentation? Do we have no schema files in the repository? How do we check if the files are up to date in CI? Some of the changes in the golden files are also kind of weird (like ids changing and things like that and some groups are just empty, which could probably be removed in the schema - I'm not sure).

The PR is great though. The only reason I left many comment is because the PR is huge. 😅

alandefreitas · 2026-05-02T00:01:35Z

+        // -------------------------------------------------------
+        // Header
+        // -------------------------------------------------------
+        line("#");


This main function is kind of hard for a human to read. Is this one of these cases where instead of using reflection to improve the schema, we make the function very complex to match the bad pattern we used to have before?

Indeed. The function is long because the schema is currently describing what XMLWriter happens to emit. That's bad.

So has this one been resolved?

RncSchemaWriter.hpp no longer exists, because we emit RNG directly now. RngSchemaWriter.cpp should be relatively easy to read.

`Symbol`'s `tag_invoke` overload added four convenience booleans -- `isRegular`, `isSeeBelow`, `isImplementationDefined`, `isDependency` -- outside the described struct. Because reflection couldn't see them, the JSON schema writer had to mirror the same hardcoded list. This removes those booleans and lets templates compare the described enum directly; e.g.: {{#if isRegular}} -> {{#if (eq extraction "regular")}} For the two `filter_by` / `any_of_by` sites that previously keyed on the booleans, the helper family gains variadic siblings `filter_by_eq` and `any_of_by_eq` -- signature `(container, key, value1, value2, ...)`. This addresses review feedback on PR cppalliance#1178.

Add a --schemas[=<dir>] option that writes a JSON Schema file (mrdocs-dom-schema.json) describing every object and field available to Handlebars templates. The schema is derived from the same compile-time reflection metadata used by MapReflectedType.hpp, so it stays in sync with the code automatically. The option requires no config file or source files — it writes the schema and exits immediately.

OperatorKind was the only enum serialized as a raw integer. All other enums serialize as human-readable strings. Change tag_invoke to use getOperatorName, consistent with the rest.

--schemas now writes both mrdocs-dom-schema.json (Handlebars DOM) and mrdocs.rnc (XML output). The XML schema mirrors XMLWriter.cpp's serialization.

…ClassKind Replace manual toString and tag_invoke overloads with MRDOCS_DESCRIBE_ENUM for four enums whose kebab-case names match the existing string representations. The XML writer now emits these fields (e.g. <access>public</access>, <constexpr>constexpr</constexpr>) where they were previously silently skipped. None/none sentinel values are suppressed via a generic has_none_enumerator check. TypeKind stays manual because toKebabCase("LValueReference") produces "l-value-reference", not the established "lvalue-reference".

… --schemas This guarantees the RELAX NG schema stays in sync with the C++ type definitions. Every CI run now validates all golden test XML files against a schema derived from the same reflection metadata that produces the XML.

This adds a small lookup table keyed by (typeName, memberName) carrying hand-written descriptions for the DOM seen by Handlebars templates which become "description" fields in the JSON schema.

The newly-added JsonEmitter.hpp duplicated functionality already provided by `dom::JSON::stringify`. This drops the new header and uses the existing `stringify`.

The new function name is more descriptive. Contextually, this expands the doc-comment to spell out what the function returns.

The schema headers are only used by ToolMain and the unit tests. They don't need to live under include/mrdocs/.

output XMLWriter silently dropped three kinds of data: - `SpecializationName::TemplateArgs`: `Polymorphic<Name>` fell into `writePolymorphic`'s generic branch with `T` deduced to the base Name (`Polymorphic::operator*` returns a base reference), so only `Name`'s own described members were emitted. Template specializations rendered as `<name>SmallVector</name>` rather than `<specialization-name>SmallVector<...></specialization-name>`. - `NoexceptInfo`: no `MRDOCS_DESCRIBE_STRUCT` - it serializes to a string via `tag_invoke`, which `writeElement` had no path for. Functions with noexcept-specifications produced no `<noexcept>` element. - `ExplicitInfo`: same pattern. explicit constructors and conversion operators produced no `<explicit>` element. This adds a `NameKind` branch in `writePolymorphic` (using a "-name" suffix on the kind tag to disambiguate from the `Name::Identifier` field), and adds a `NoexceptInfo`/`ExplicitInfo` branch in `writeElement` that emits the `toString()` value as text, skipping the empty case. Also update `RncSchemaWriter` to match: `Polymorphic<Name>` -> `AnyName`, drop `NoexceptInfo`/`ExplicitInfo` from the omit list. Most XML golden fixtures regenerate to include the now-emitted elements.

The XMLTags helper in src/lib/Gen/Xml/ contained two pieces of machinery that aren't specific to XML doc generation: an XML escaper (xmlEscape) and a tag/indent stream emitter. A RELAX NG schema writer, which will be introduced with the next commit, also needs them. So, we factored them out.

The --schemas option now writes a RELAX NG XML document directly. This gets rid of the trang RNC->RNG conversion step. Which, in turn, means we no longer need Java. The bootstrap script dependency on Java will be removed with the next commit.

The bootstrap script checked for Java because the build needed it to run trang.jar, which converted the RNC schema to RNG. But trang.jar is no longer used (the --schemas option directly emits a .rng), so Java is no longer needed.

The schema writer emits a `description` field for every type and every described member it touches, looking it up from DomDescriptions.hpp. When the lookup failed, it silently returned an empty string, so forgetting a description caused an undocumented `$defs` entry to be silently emitted. This adds an assert that fires when the lookup of a type or a member finds no entry. This allowed finding many missing entries, which have been added. Any future described type added to the schema trips the assert at build time until descriptions for it and its members are provided.

…ation targets Both schema files are now committed under docs/, parallel to the existing docs/mrdocs.schema.json (the YAML config schema) and are exposed to the Antora docs site as downloadable attachments. Two new CTest targets, `rng-schema-check` and `dom-schema-check`, run `cmake -E compare_files` between the freshly-generated schemas in the build tree and the checked-in copies; drift fails the test. The schemas custom_command is lifted out of the LibXml2 conditional so the freshness checks run independently of whether libxml2 is available. .gitattributes pins the two schema files to LF line endings, because --schemas emits LF line endings and we do a byte-for-byte comparison.

This replaces the hand-written DOM reference with one generated from mrdocs-dom-schema.json. A new Antora extension walks the file's `$defs` in source order and emits one section per type.

`Symbol`'s `tag_invoke` overload added four convenience booleans -- `isRegular`, `isSeeBelow`, `isImplementationDefined`, `isDependency` -- outside the described struct. Because reflection couldn't see them, the JSON schema writer had to mirror the same hardcoded list. This removes those booleans and lets templates compare the described enum directly; e.g.: {{#if isRegular}} -> {{#if (eq extraction "regular")}} For the two `filter_by` / `any_of_by` sites that previously keyed on the booleans, the helper family gains variadic siblings `filter_by_eq` and `any_of_by_eq` -- signature `(container, key, value1, value2, ...)`. This addresses review feedback on PR cppalliance#1178.

alandefreitas · 2026-06-04T16:16:14Z

Thanks, removing the isXxx booleans is a good change.

Is there a way to let reflection drive the missing parts that are manually written right now? If that's not possible right now, could we open an issue to track it? This is exactly what reflection was meant to solve.

Separately, as I mentioned in the previous comment, a few of the golden XML changes still looked odd to me.

gennaroprota changed the title ~~Feat/schema generation~~ feat: schema generation Apr 17, 2026

gennaroprota force-pushed the feat/schema_generation branch 6 times, most recently from cdda298 to b9fac7c Compare April 17, 2026 16:31

alandefreitas reviewed May 2, 2026

View reviewed changes

gennaroprota force-pushed the feat/schema_generation branch 7 times, most recently from 73561af to f9d5f63 Compare May 6, 2026 10:14

gennaroprota force-pushed the feat/schema_generation branch from f9d5f63 to fb9c795 Compare May 13, 2026 14:41

gennaroprota force-pushed the feat/schema_generation branch from fb9c795 to 0ae30ec Compare May 13, 2026 16:08

gennaroprota force-pushed the feat/schema_generation branch from 0ae30ec to 903033b Compare May 14, 2026 06:19

gennaroprota added 5 commits May 14, 2026 09:18

fix: serialize OperatorKind as a string in the DOM

b715b79

OperatorKind was the only enum serialized as a raw integer. All other enums serialize as human-readable strings. Change tag_invoke to use getOperatorName, consistent with the rest.

feat: have --schemas also generate an XML schema

9f8174c

--schemas now writes both mrdocs-dom-schema.json (Handlebars DOM) and mrdocs.rnc (XML output). The XML schema mirrors XMLWriter.cpp's serialization.

gennaroprota added 13 commits May 14, 2026 09:18

feat(schemas): describe DOM types and members in the JSON schema

112f9e1

This adds a small lookup table keyed by (typeName, memberName) carrying hand-written descriptions for the DOM seen by Handlebars templates which become "description" fields in the JSON schema.

refactor: use dom::JSON::stringify instead of a duplicate emitter

35b5d2f

The newly-added JsonEmitter.hpp duplicated functionality already provided by `dom::JSON::stringify`. This drops the new header and uses the existing `stringify`.

refactor: rename buildDomSchema to buildDomJsonSchema

1cf59f8

The new function name is more descriptive. Contextually, this expands the doc-comment to spell out what the function returns.

refactor: move the schema headers out of the public API

d4f7446

The schema headers are only used by ToolMain and the unit tests. They don't need to live under include/mrdocs/.

build: drop the Java prerequisite

a6a12c7

The bootstrap script checked for Java because the build needed it to run trang.jar, which converted the RNC schema to RNG. But trang.jar is no longer used (the --schemas option directly emits a .rng), so Java is no longer needed.

docs: add documentation for the --schemas option

7fa789f

docs: replace the manual DOM reference with a generated one

6ce122b

This replaces the hand-written DOM reference with one generated from mrdocs-dom-schema.json. A new Antora extension walks the file's `$defs` in source order and emits one section per type.

gennaroprota force-pushed the feat/schema_generation branch from 903033b to 019d019 Compare May 14, 2026 07:37

Conversation

gennaroprota commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Testing

Documentation

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✨ Highlights

🧾 Changes by Scope

🔝 Top Files

Uh oh!

codecov Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cppalliance-bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alandefreitas commented Apr 21, 2026

Uh oh!

gennaroprota commented Apr 21, 2026

Uh oh!

alandefreitas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alandefreitas May 2, 2026

Choose a reason for hiding this comment

Uh oh!

gennaroprota May 4, 2026

Choose a reason for hiding this comment

Uh oh!

alandefreitas May 13, 2026

Choose a reason for hiding this comment

Uh oh!

gennaroprota May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alandefreitas commented Jun 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gennaroprota commented Apr 16, 2026 •

edited

Loading

github-actions Bot commented Apr 16, 2026 •

edited

Loading

codecov Bot commented Apr 16, 2026 •

edited

Loading

cppalliance-bot commented Apr 16, 2026 •

edited

Loading