Skip to content

refactor: pass the CKAN settings to safe-http.ts per call - #594

Merged
aborruso merged 1 commit into
ondata:mainfrom
bjornhagstrom:refactor/inject-safe-http-settings
Nov 12, 2026
Merged

aborruso merged 1 commit into
ondata:mainfrom
bjornhagstrom:refactor/inject-safe-http-settings

Conversation

@bjornhagstrom

Copy link
Copy Markdown
Contributor

Step 2 as agreed in #590, and the last one: safe-http.ts no longer reads any configuration or names CKAN.

What changes.

  • http.ts reads the CKAN_* settings exactly as before (the two size caps once at load, the others on every call) and passes them explicitly. There is no module-level configuration in safe-http.ts.
    • validateServerUrl and assertRedirectKeepsPrivateExemption take a UrlGuardSettings object: the allowlist and the private hosts as the raw comma-separated strings, plus the two setting names that the messages show.
    • safeFetch takes a function that returns that object. It is called at each redirect hop, which is when the settings are read today.
    • isPrivateHostAllowed takes the list, and decodePossiblyCompressed takes the decompression cap.
    • createSsrfSafeLookup keeps the caller's settings object, as today, and takes a privateHosts getter. The getter is called at each lookup, because the agents are kept for the process and the list is read at lookup time today.
  • http.ts exports thin wrappers under the same names. Every export of utils/http.js is unchanged in name and type (20), so no importer and no test changes.
  • Two functions move verbatim from safe-http.ts to http.ts: warnIfPrivateHostsOverHttp and assertHttpAllowlistConfigured. They are startup checks that only read CKAN_* settings and print CKAN text.
  • The process-wide cache of the safe agents moves to http.ts. safe-http.ts keeps the builder, createSafeAgents.
  • Comments in safe-http.ts that named CKAN or sparql_query now say the same thing generically, or the text moved to the wrapper in http.ts.

One gap I noticed and left alone, since this step changes no tests: no test checks that CKAN_PRIVATE_HOSTS reaches the axios agents' lookup for a listed host name. The existing agent tests start on 127.0.0.1, where Node skips the lookup. The gap is the same on main, and a regression there would fail closed. A small test for it can come later if you want one.

How to check.

  • git diff main -- src/utils/safe-http.ts | grep -i ckan shows only removed lines.
  • npm test: 726 passed, 6 skipped, as on main; no test file touched.
  • Timing: changing CKAN_ALLOWED_DOMAINS while a safeFetch redirect is pending, or the settings object of a lookup after creating it, gives the same result as on main. I checked this with a throwaway script against both commits; it is not added as a test.
  • npm run smoke: 18/18 against the live portals (2026-11-12); tests/smoke/cases.json untouched.
  • npm run build and npm run build:worker succeed. The Worker bundle grows by 0.4 KB, the wrappers.
  • The exports of src/utils/http.ts are identical before and after, in names and types, compared with the TypeScript checker. On src/utils/ and src/transport/http.ts, the checker gives the same four errors as on main.

AI assistance: prepared with Claude (Anthropic) under my direction and review, with automated reviews before opening.

🤖 Generated with Claude Code

safe-http.ts no longer reads process.env. http.ts reads the CKAN_*
settings at the same moments as before and passes them explicitly:
validateServerUrl and assertRedirectKeepsPrivateExemption take a
UrlGuardSettings object (lists plus the setting names shown in
messages); safeFetch takes a function returning it, called at each
redirect hop as before; isPrivateHostAllowed takes the list;
decodePossiblyCompressed takes the decompression cap (read once at
load, as before); createSsrfSafeLookup keeps the caller's settings
object and takes a privateHosts getter, read at each lookup because
the agents are kept for the process. No module-level configuration in
safe-http.ts.

http.ts exports wrappers under the same 20 names and types, so no
importer and no test changes; messages are byte-identical. The two
startup checks (warnIfPrivateHostsOverHttp,
assertHttpAllowlistConfigured), which only read CKAN_* settings, move
verbatim to http.ts, and so does the cache of the safe agents
(createSafeAgents in safe-http.ts builds them). CKAN and sparql_query
mentions in safe-http.ts comments move to http.ts or become generic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

RetriggerView in Greptile

[Medium impact] This PR appears safe to merge; no actionable issues were found.

Summary

This PR moves CKAN configuration reads and the cached axios agents into http.ts. safe-http.ts now receives settings and size caps from its caller.

  • The HTTP layer passes settings to reusable safety helpers.

Reviews (1) · Last reviewed commit: "refactor: pass the CKAN settings to safe..." · Reviewed by Greptile

@aborruso aborruso left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Björn, this is what we agreed in #590, and it stays within that scope.

Checked against the base (205ba4a):

  • utils/http.ts exports the same 20 names before and after; nothing else imports safe-http.ts.
  • safe-http.ts has no process.env and no CKAN mention left. Settings are passed per call, with no module-level state. The size caps are read at load, the rest on every call, redirect hop and lookup, as before.
  • The defaults fail closed: createSsrfSafeLookup without a private-host getter gets an empty list, and safeFetch requires urlGuard.
  • Messages are unchanged, and the two startup checks moved verbatim.
  • It merges cleanly with main (#593 touched one hint line in http.ts).
  • npm run build, npm run build:worker and npm test (726 passed, 6 skipped) are green locally.

One nit, not blocking: const raw = privateHosts; in isPrivateHostAllowed is now redundant.

The missing test for CKAN_PRIVATE_HOSTS reaching the axios lookup can come separately if you feel like it; as you say, it fails closed.

@aborruso
aborruso merged commit 2667613 into ondata:main Nov 12, 2026
2 checks passed
aborruso added a commit that referenced this pull request Nov 12, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RCCNPJK12mLzKydJ8Cp5Lc
aborruso pushed a commit that referenced this pull request Nov 12, 2026
…596)

Also drops the redundant local in isPrivateHostAllowed (review of #594).

Co-authored-by: bjornhagstrom <983409+bjornhagstrom@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants