Read a function's Doxygen from its MEOS twin, in one order on every machine

doxygen_sources lists each tree it reads sorted, with every <name>_meos.c file before the other files of its tree. A function defined both in the file the PostgreSQL extension compiles and in the MEOS twin that replaces it in the MEOS build, as route_exists and route_geom are in meos/src/npoint/ways.c and ways_meos.c, is then documented by the twin MEOS compiles, and the brief, the parameter descriptions, the nullability and the output parameters every reader keeps from the first block it meets come from that block on every machine.

Why. The list was in the order glob returns, which is the order the filesystem lists a directory in, so the brief of route_exists read "the ways cache" on one machine and "the edge table" on another from the same MobilityDB commit, and a consumer that generates from the catalog on one machine and checks it on another reads a difference neither its code nor MobilityDB causes: MobilityDB/MobilityAPI-Python#19 is the case.

Measured. Over MobilityDB 93e1d279be the catalog this derives on a machine whose filesystem lists ways_meos.c first is the same as before, all 6625 functions; the GitHub runner, which listed ways.c first, read the edge-table briefs.

Witness. tests/test_nullable.py::test_the_meos_twin_documents_a_function_defined_twice lists ways.c and ways_meos.c in both orders and reads the twin's brief for route_exists each time; with the previous order the listing that puts ways.c first reads the edge-table brief and the test fails. The suite passes with the MobilityDB sources present: 578 tests, 98 skipped.

Author: estebanzimanyi

parser/outparam.py

@@ -28,11 +28,22 @@ def doxygen_sources(meos_root: str | Path) -> list[str]:
     ``meos/`` and documents them on its ``.c`` definitions (mirrors doxygroup/run.py
     pgtypes). Every ``@param`` reader, here and in #extract_nullable, scans this one
     list, so a function documented in ``pgtypes/`` states its parameters to all of them
-    alike."""
+    alike.
+
+    The list is in one order on every machine: each tree sorted, with a ``<name>_meos.c``
+    file before every other file of its tree. A function defined both in the file the
+    PostgreSQL extension compiles (``ways.c``) and in its MEOS twin (``ways_meos.c``) is
+    then documented by the twin MEOS compiles, and every reader keeping the first block
+    it meets (#extract_briefs among them) reads that block."""
     root = Path(meos_root)
-    files = glob.glob(str(root / "src/**/*.c"), recursive=True)
-    files += glob.glob(str(root / "include/**/*.h"), recursive=True)
-    files += glob.glob(str(root.parent / "pgtypes/**/*.c"), recursive=True)
+
+    def tree(pattern: str) -> list[str]:
+        found = glob.glob(pattern, recursive=True)
+        return sorted(found, key=lambda f: (not f.endswith("_meos.c"), f))
+
+    files = tree(str(root / "src/**/*.c"))
+    files += tree(str(root / "include/**/*.h"))
+    files += tree(str(root.parent / "pgtypes/**/*.c"))
     return files
 
 

tests/test_nullable.py

@@ -7,9 +7,11 @@
 
 Plain unittest, no pytest dependency; writes a tiny synthetic source tree.
 """
+import glob
 import tempfile
 import unittest
 from pathlib import Path
+from unittest import mock
 
 from parser.nullable import (attach_function_docs, extract_briefs, extract_nullable,
                              extract_param_docs, merge_nullable)
@@ -106,6 +108,24 @@ def test_reads_the_tag_inside_the_param_entry(self):
         self.assertNotIn("create", nul["jsonb_set_lax"])
         self.assertNotIn("jb", nul["jsonb_set_lax"])
 
+    def test_the_meos_twin_documents_a_function_defined_twice(self):
+        """A function defined in the file the extension compiles and in its MEOS
+        twin keeps the twin's brief, whichever file the filesystem lists first, as
+        #test_briefs_read_the_brief_of_each_block reads the brief of one block."""
+        src = Path(self.tmp.name) / "src"
+        block = "/**\n * @brief {}\n */\nbool\nroute_exists(int64 rid)\n{{\n}}\n"
+        (src / "ways.c").write_text(
+            block.format("Return true if the edge table contains a route"))
+        (src / "ways_meos.c").write_text(
+            block.format("Return true if the ways cache contains a route"))
+        real_glob = glob.glob
+        for listed in (sorted, lambda files: sorted(files, reverse=True)):
+            with self.subTest(listed=listed), mock.patch(
+                    "parser.outparam.glob.glob",
+                    lambda pattern, **kw: listed(real_glob(pattern, **kw))):
+                self.assertEqual(extract_briefs(self.tmp.name)["route_exists"],
+                                 "Return true if the ways cache contains a route")
+
     def test_param_docs_leave_out_the_tag(self):
         """The description #test_extracts_only_tagged_params reads, without the
         tag."""