Merge pull request #19 from estebanzimanyi/fix/vendor-parse-reads-the-mobilitydb-sources

Generate the dispatcher's table from the catalog CI derives, as MobilityAPI does

Author: estebanzimanyi

.github/workflows/python.yml

@@ -20,6 +20,43 @@ jobs:
         run: ruff check . || echo "::warning::ruff found issues — please fix before merging"
         continue-on-error: true
 
+  generated-table:
+    name: The generated table is what the catalog declares
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      # The MEOS-API catalog, derived here from MobilityDB master by the action
+      # every binding in the ecosystem uses. It is never committed: it is a
+      # derived artifact of one MobilityDB commit, and a tracked copy is drift
+      # waiting to happen.
+      - name: Derive the MEOS-API catalog
+        id: provision
+        uses: MobilityDB/MEOS-API/.github/actions/provision-meos@master
+        with:
+          mobilitydb-ref: master
+
+      # catalog_gen.py is the catalog's exposable functions projected into
+      # Python, so it is generated rather than kept by hand, and this is what
+      # makes the copy in the tree the current one. A function MEOS gains,
+      # loses or changes arrives here as a difference to regenerate, which is
+      # the whole reason the table is generated: a hand-kept copy goes stale
+      # in silence instead.
+      - name: The generated table is what the catalog declares
+        run: |
+          python3 tools/codegen.py -catalog "${{ steps.provision.outputs.catalog-path }}" \
+            -o mobilityapi/catalog_gen.py
+          if ! git diff --quiet -- mobilityapi/catalog_gen.py; then
+            echo "::error::the generated table is not what the catalog declares. Regenerate it: python3 tools/codegen.py -catalog <meos-idl.json>"
+            git diff --stat -- mobilityapi/catalog_gen.py
+            git diff -- mobilityapi/catalog_gen.py | head -n 100
+            exit 1
+          fi
+          echo "the committed table is what the catalog of MobilityDB master declares"
+
   python-import:
     name: Import smoke test
     runs-on: ubuntu-latest

.github/workflows/vendor-drift.yml

@@ -0,0 +1,50 @@
+# Generation
+
+MobilityAPI-Python is a tier over MEOS, so what it knows about MEOS is
+generated from the MEOS-API catalog rather than kept by hand. The generator
+lives in `tools/codegen.py`, its output is committed, and CI derives the catalog
+from MobilityDB master, regenerates against it, and refuses a difference. This
+is the arrangement of MobilityAPI's Go tier, whose generator is
+`tools/codegen/main.go`.
+
+## What is generated
+
+`mobilityapi/catalog_gen.py` holds `FUNCTIONS`, one entry per MEOS function the
+catalog states as exposable, ordered by name. Each entry is the catalog's own
+entry restricted to the keys the dispatcher reads: `name`, `category`,
+`params`, `returnType`, `network`, `wire` and `doc`.
+`FunctionSignature.from_catalog_entry` reads an entry of the table exactly as it
+reads one of the catalog, and the `Dispatcher` serves every function of the
+table: `GET /catalog` lists them and `POST /functions/{name}` invokes one.
+
+Run it against a catalog:
+
+```bash
+python3 tools/codegen.py -catalog <meos-idl.json> -o mobilityapi/catalog_gen.py
+```
+
+A catalog entry without one of the keys the dispatcher reads, or a catalog
+stating no function as exposable, stops the generator.
+
+## Where the facts come from
+
+`meos-idl.json` is the machine-readable description of the MEOS C library that
+[MEOS-API](https://github.com/MobilityDB/MEOS-API) derives from the MEOS headers
+and sources, and that every binding in the ecosystem generates from. Whether a
+function is exposable, and how each parameter and the result travel over the
+wire, is the catalog's `network` and `wire` projection, which MEOS-API derives
+from each function's types, SQL name and group.
+
+The catalog is a derived artifact of one MobilityDB commit, so it is never
+committed. CI derives it with
+`MobilityDB/MEOS-API/.github/actions/provision-meos@master`, the action every
+binding uses, and the `generated-table` job of `.github/workflows/python.yml`
+regenerates the table against it. A function MEOS gains, loses or changes on
+master arrives there as a difference to regenerate.
+
+To derive the catalog by hand, run MEOS-API's `tools/provision-meos.sh` over a
+MobilityDB checkout:
+
+```bash
+<MEOS-API>/tools/provision-meos.sh --mdb-src <MobilityDB> --catalog-out meos-idl.json
+```

GENERATION.md

@@ -1,8 +1,8 @@
 """MobilityAPI catalog-driven dispatcher package.
 
-The MobilityAPI ingestion plan (docs/MEOS_API_INGESTION_PLAN.md) calls for
-replacing the hand-written MEOS-dispatching endpoint modules with thin
-dispatchers driven by the vendored MEOS-API catalog. This package provides
+The MEOS-dispatching endpoints are thin dispatchers driven by the MEOS-API
+catalog, through the table mobilityapi/catalog_gen.py that tools/codegen.py
+generates from it. This package provides
 the three foundation pieces the migrating endpoints share:
 
 - ``Dispatcher``     — catalog-driven function lookup + invocation.