Skip to content

docs(openapi): document GeoDatasets append params + add/enrich other core plugin endpoints#1015

Merged
tariqksoliman merged 1418 commits into
NASA-AMMOS:developmentfrom
JPL-Devin:development
Jul 9, 2026
Merged

docs(openapi): document GeoDatasets append params + add/enrich other core plugin endpoints#1015
tariqksoliman merged 1418 commits into
NASA-AMMOS:developmentfrom
JPL-Devin:development

Conversation

@tariqksoliman

Copy link
Copy Markdown
Member

With Devin: JPL-Devin#106

Purpose

  • A user reported that programmatically appending features to a GeoDataset (via the Configure API + long-term token) did not obey the group_id, start-time, and end-time fields that were set when the GeoDataset was created.
  • Root cause is a documentation gap, not a code bug: the append endpoints only populate start_time/end_time/group_id/feature_id when the corresponding prop names are re-supplied on the append request. The *_field config stored at creation is read only to decide which columns exist, and is never used to compute per-feature values on append. Without the props, appended features get NULL for those columns and won't match temporal/group_id queries.
  • This PR updates the Swagger/OpenAPI spec (docs/mmgis-openapi.json, served at /api-docs) and the GeoDatasets API markdown doc to document those params, then extends the audit to the rest of the core backend plugins — adding the remaining undocumented functional endpoints and enriching thin/undescribed field docs. No runtime code changed — docs only.

Proposed Changes

GeoDatasets (original scope)

  • [ADD] POST /append/{name}: group_id_prop and feature_id_prop query params; documented body-level startProp/endProp/groupIdProp/featureIdProp; prominent note that append does not auto-reuse the fields configured at creation.
  • [FIX] POST /append/{name} & /append/{name}/{start_end_prop}: request body IS the GeoJSON FeatureCollection, not a { geojson: ... } wrapper.
  • [ADD] GET /get & /get/{layer}: group_id, id, spatialFilter, format. GET /aggregations: startProp, endProp, format. POST /search: operator, type, last. POST /recreate: groupIdProp, featureIdProp, filename + action description. Documented POST /intersect and GET /bulk_aggregations.
  • Added E2E test tests/e2e/api/geodatasets-append-fields.spec.js covering create-with-fields → append-with-props → temporal + group_id queries, plus the append-without-props NULL gotcha.

Other core backend plugins (follow-up scope)

  • [ADD] Previously-undocumented functional endpoints:
    • GET /api/stac/collections, GET /api/stac/collections/{collection}/export (new STAC tag)
    • POST /api/draw/aggregations
    • POST /api/utils/getminmax, POST /api/utils/ll2aerll_bulk
    • POST /api/configure/reference-mission/save-to-base
  • [CHANGE] Enriched thin/undescribed field docs on existing endpoints: users login/signup/first_signup/resetPassword, utils getprofile/getbands, configure add/updateGeneralOptions, shortener shorten/expand, longtermtoken generate/clear.
    • Notably documented that longtermtoken period is milliseconds from creation (or the string never) — a unit string like "1d" parses to 1ms and the token is immediately expired.
  • Excluded on request: Sightline endpoints, the internal testwebhooks stubs, and POST /api/configure/rename (empty no-op handler in code).

github-actions Bot and others added 30 commits June 18, 2026 23:43
Save the current toolPanelWidth before opening Review and set it
to 390px (only if currently narrower). On close, restore the
previous width via the Zustand uiStore.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The Zustand store update triggers a React re-render, but the
jQuery-managed #toolPanel content may not reflect the width change
immediately. Set the width directly on the #toolPanel element and
update the drag handle position for instant visual feedback.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…y-plugins

fix: copy plugins/ directory into Docker runtime stage
… is open

Extract cleanupReview() in DrawTool_Publish.js to handle width
restoration and review panel removal. Call it from both the close
button handler and separateFromMMGIS() so the panel width is
always restored when leaving DrawTool.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…sage

Replace inline style assignments with a CSS class (.truncated) so
the base styles from the stylesheet apply without needing !important.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…eview-panel

fix: DrawTool Review panel hidden by tool panel backdrop-filter containment
…ll control chars in logs

Fix 2: Add permissions: contents: read to secrets-detection.yaml and pin
actions/checkout to SHA (matching bump-version.yml).

Fix 3: Validate _source parameter entries in geodatasets routes to only
allow alphanumeric characters, underscores, and dots before query
construction.

Fix 4: Simplify sanitizeForLog regex to strip ALL control characters
(0x00-0x1F and 0x7F), closing the newline/carriage-return log injection
gap.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Apply forceAlphaNumUnder per-segment (split by dot) to preserve nested
JSON paths like 'properties.name' while sanitizing each path component.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Update inline copy and test expectations to reflect the new regex that
strips ALL control characters (0x00-0x1F, 0x7F) including newlines,
tabs, and carriage returns.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…feedback

Fix 1A: Rewrite newTag HTML injection in DrawTool_Files.js using jQuery
DOM construction (.attr(), .text(), .append()) instead of raw string
interpolation in $.append().

Fix 1B: Remove inline value attributes from F_.sanitize() in
single-quoted attribute contexts (DrawTool_Templater.js:671,
DrawTool_Editing.js:684). Set values safely via jQuery .val() after DOM
insertion.

Add allowList parameter to Utils.forceAlphaNumUnder() to optionally
preserve specific characters (e.g. dots for nested JSON paths).
Simplify _source validation to use forceAlphaNumUnder(s, ['.']) and
fall back to null if all entries are filtered out.

Revise sanitizeForLog to preserve newlines and tabs for stack trace
readability while now stripping carriage returns (0x0D) for log
injection prevention. Update tests to match.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The initial Fix 1A only covered the 'add new tag' path. Three additional
rendering paths (existing efolders, folders, tags at lines 910-958) also
used raw string interpolation for tag names. Extract buildTagItemHtml()
helper that uses jQuery DOM construction (.attr(), .text()) to safely
render all tag items.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Add '-' to forceAlphaNumUnder allowList since JSONB property names
commonly contain hyphens (e.g. start-time, image-url).

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…-2-3-4

security: XSS prevention, CI hardening, geodataset validation, log sanitization
…, backend pagination

- Rewrite jQuery Search.js as React Search.jsx with design-system components
- Add GlobalSearchPanel.jsx: advanced filter builder in floating right panel
  with layer multi-select, filter builder (same data model as Filtering.js),
  paginated results list grouped by layer
- Add 'All Layers (Fields)' cross-layer search with field:value syntax
- Backend: add offset/limit query params to geodatasets_get endpoint
- Update OpenAPI docs with offset/limit parameter specs
- Make right panel float over map (not push splitscreens)
- Update Reference-Mission blueprint with search variables for geodataset layers
- Add unit tests (20 tests) for filter encoding and field parsing
- Add E2E tests for search bar and advanced panel UI
- Remove jQuery Search.js, Dropy/autocomplete dependencies from Search

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ng tracking

- Fix URL parameter search: pass L_.searchFile as forceSTS to doWithSearch
  so lname resolves correctly before async state update
- Fix setSearching(false) timing: use pendingRef counter so 'Searching...'
  state persists until all async fetch callbacks complete
- Clean up unused imports (useMemo, createPortal, layerType, keyOptions)

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- Replace single layer dropdown with sectioned dropdown:
  - 'Search by Field' section with filterable field list (name, type, source layers)
  - 'Search Specific Layer' section (collapsed by default)
- Default mode: cross-layer search by display name (no dropdown interaction needed)
- Field selection shows chip, input becomes value search for that field
- Layer selection scopes search to specific layer (preserves existing behavior)
- Add GET /api/geodatasets/schema bulk endpoint: returns field names, types, and
  source layers for multiple geodataset tables in one call
- Add geodatasets_schema API call definition in calls.js
- Document /api/geodatasets/schema in OpenAPI spec
- Update E2E tests for new dropdown structure
- Remove unused Select import from Search.jsx (dropdown is now custom)

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The bulk schema endpoint needs to be accessible without admin auth,
just like /get, /aggregations, /intersect, and /search.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…sing

- searchGeodatasets and searchByField: geodatasets_search returns GeoJSON
  Feature objects with coordinates under .geometry.coordinates, not
  .coordinates directly
- GlobalSearchPanel aggregation parsing: response format is
  { key: { type, aggs: { val: count } } }, not an array — extract
  Object.keys(vals.aggs) for autocomplete values

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
devin-ai-integration Bot and others added 25 commits July 7, 2026 18:02
When toggling geodataset layers on, addGeoJSONData / updateVectorLayer
re-selects L_.activeFeature from a previous search — causing stale
highlights. Clearing activeFeature at the start of select mode
prevents layer reloads from re-selecting the old feature.

Also filters vector layers by sourceLayers in group select mode so
doWithSearch only runs on layers that actually have the clicked value.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…e, multi-field search, test corrections, variable naming

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
… query

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…-search

feat: Global Feature Search — unified panel, field search, operator selection, layer management
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…lowercase

fix: lowercase COG colormap_name before passing to TiTiler
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…nit-crash

fix: prevent Coordinates.init crash when coordll disabled
… params

- append: add group_id_prop/feature_id_prop query params, document body-level startProp/endProp/groupIdProp/featureIdProp, and note that append does not auto-reuse the fields configured at creation
- get: add group_id, id, spatialFilter, format query params
- aggregations: add startProp, endProp, format
- search: add operator, type, last body params
- recreate: add groupIdProp, featureIdProp, filename
- document previously-undocumented POST /intersect and GET /bulk_aggregations

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…riptions

Add previously-undocumented functional endpoints: STAC collections list
and collection export, POST /api/draw/aggregations, POST /api/utils/getminmax,
POST /api/utils/ll2aerll_bulk, and POST /api/configure/reference-mission/save-to-base.
Enrich per-field descriptions on several thin endpoints (users login/signup/
first_signup/resetPassword, utils getprofile/getbands, configure add/
updateGeneralOptions, shortener shorten/expand, longtermtoken generate/clear),
notably documenting that longtermtoken 'period' is milliseconds or 'never'.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…penapi-docs

docs(openapi): document GeoDatasets append params + add/enrich other core plugin endpoints
@tariqksoliman tariqksoliman self-assigned this Jul 9, 2026
@tariqksoliman tariqksoliman added the documentation Changes to in-app help, code comments, swaggers or documentation site label Jul 9, 2026
@tariqksoliman tariqksoliman merged commit 1231229 into NASA-AMMOS:development Jul 9, 2026
4 of 7 checks passed
@github-project-automation github-project-automation Bot moved this to Done in MMGIS Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Changes to in-app help, code comments, swaggers or documentation site

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant