Add SPOG support for ?o= routing in httpPath by msrathore-db · Pull Request #1316 · databricks/databricks-jdbc

msrathore-db · 2026-03-24T19:13:13Z

Summary

SPOG (Single Panel of Glass) replaces workspace-specific hostnames with account-level vanity URLs. When httpPath contains ?o=<workspaceId>, the driver needs to:

Preserve the ?o= value through URL parsing
Extract the clean warehouse ID (without ?o=) for SEA JSON body
Inject x-databricks-org-id header on endpoints that don't carry ?o= in the URL path (telemetry, feature flags, DBFS volume)

Changes

Fix 1 — Property parser (DatabricksConnectionContext.java):

split("=") broke httpPath values containing = (e.g. httpPath=/sql/1.0/warehouses/xxx?o=yyy split into 3 parts, dropping the workspace ID)
Changed to indexOf("=") + substring to split on first = only

Fix 2 — Warehouse ID regex (DatabricksJdbcConstants.java):

(.+) captured abc123?o=999 as warehouse ID, corrupting SEA JSON body
Changed to ([^?]+).* to stop capture at query params
Trailing .* needed because Java .matches() requires full-string match

Fix 3 — SPOG header extraction (DatabricksConnectionContext.java):

Uses URIBuilder to parse ?o=<workspaceId> from httpPath
Injects x-databricks-org-id into customHeaders map
Respects explicitly set header (won't override)

Fix 4 — Header propagation (telemetry, feature flags, DBFS volume):

TelemetryPushClient.java: Added connectionContext.getCustomHeaders() to telemetry POST
DatabricksDriverFeatureFlagsContext.java: Added connectionContext.getCustomHeaders() to feature flag GET
DBFSVolumeClient.java: Added connectionContext.getCustomHeaders() to all 5 HTTP request paths (4 sync + 1 async), with null guard for test constructor

What does NOT get the header

OAuth endpoints (/oidc/v1/token, OIDC discovery) — they are workspace-agnostic and reject x-databricks-org-id with HTTP 400. These use their own HTTP client and never see customHeaders.

Context

Per the SPOG Peco Clients doc:

Thrift: ?o= in the POST URL handles routing naturally
SEA, telemetry, feature flags, DBFS volume: Need x-databricks-org-id header since they use separate endpoint paths

Jira: XTA-15079

Test plan

Unit tests added:

testBuildPropertiesMap_preservesQueryParamInHttpPath — httpPath with ?o= preserved
testBuildPropertiesMap_handlesValueWithMultipleEquals — values with multiple = preserved
testBuildPropertiesMap_handlesValueWithNoEquals — key-only params handled
testSpogContext_extractsOrgIdFromHttpPath — x-databricks-org-id extracted from ?o=
testSpogContext_extractsCleanWarehouseId — warehouse ID is abc123 not abc123?o=...
testSpogContext_noOrgIdWithoutQueryParam — no header injected for legacy URLs
testSpogContext_explicitHeaderTakesPrecedence — explicit http.header.x-databricks-org-id wins over ?o=
3 SPOG URL validation tests in ValidationUtilTest

E2E tested (not committed):

DBSQL + PAT: Thrift and SEA on SPOG host
GP Cluster + PAT: Thrift on SPOG host
OAuth M2M: Thrift and SEA on SPOG host
OAuth U2M: Thrift and SEA on SPOG host (with browser login)
Telemetry: Verified 200 on SPOG with org-id header
Feature flags: Verified 200 on SPOG with org-id header
DBFS Volume LIST: Verified on SPOG host
Legacy host: All auth modes unaffected

NO_CHANGELOG=true

This pull request was AI-assisted by Isaac.

SPOG replaces workspace-specific hostnames with account-level vanity URLs. This requires workspace routing info (?o=<workspaceId> or x-databricks-org-id header) on all HTTP requests to the SPOG host. Fixes: 1. Property parser: use indexOf to split on first '=' only, so values containing '=' (like httpPath with ?o=) are preserved correctly (DatabricksConnectionContext.java) 2. Warehouse ID regex: (.+) -> ([^?&]+) to stop at query params (DatabricksJdbcConstants.java) 3. Auto-inject x-databricks-org-id header from ?o= in httpPath into custom headers map, propagating to Thrift, SEA, telemetry, and feature flags (DatabricksConnectionContext.java) 4. Propagate custom headers to telemetry and feature flag clients (TelemetryPushClient.java, DatabricksDriverFeatureFlagsContext.java) Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

DBFSVolumeClient uses the SDK's ApiClient directly for /api/2.0/fs/* endpoints. These requests need the x-databricks-org-id header for SPOG routing, same as telemetry and feature flags. Add connectionContext.getCustomHeaders() to all 5 HTTP request paths in DBFSVolumeClient (4 sync via apiClient, 1 async via requestBuilder). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

- Fix regex: ([^?&]+) -> ([^?&]+).* so warehouse/endpoint patterns match full JDBC URLs in isValidJdbcUrl() when ?o= is present. The trailing .* consumes remaining URL content for .matches() while [^?&]+ still stops at query params for .find() in buildCompute(). - Add SPOG URL test cases to ValidationUtilTest (3 new URLs with ?o=) - Add unit tests for buildPropertiesMap (property parser preserves ?o=) - Add unit tests for parseCustomHeaders (org-id extraction from ?o=, clean warehouse ID extraction, explicit header precedence) - Remove leftover SpogIntegrationTests.java from earlier exploration Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

Existing tests create DBFSVolumeClient with a test constructor that sets connectionContext to null. Guard getCustomHeaders() calls with null check to avoid NPE in test paths while preserving SPOG header propagation in production paths. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

gopalldb · 2026-04-07T05:49:31Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksConnectionContext.java

+      if (queryStart >= 0) {
+        String queryString = httpPath.substring(queryStart + 1);
+        for (String param : queryString.split("&")) {
+          String[] kv = param.split("=", 2);


use some Url parser utility for this

Updated to use URI builder. Thanks

Replace manual string splitting of ?o= from httpPath with Apache URIBuilder.getQueryParams() as suggested in code review. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

gopalldb · 2026-04-07T10:49:22Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksConnectionContext.java

+      try {
+        for (NameValuePair param :
+            new URIBuilder("http://placeholder" + httpPath).getQueryParams()) {
+          if ("o".equals(param.getName())


declare o as constant

gopalldb · 2026-04-07T10:49:33Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksConnectionContext.java

+          }
+        }
+      } catch (URISyntaxException e) {
+        // Malformed httpPath — skip SPOG header extraction


add logging

gopalldb · 2026-04-07T10:50:11Z

src/main/java/com/databricks/jdbc/common/DatabricksJdbcConstants.java

-  public static final Pattern HTTP_WAREHOUSE_PATH_PATTERN = Pattern.compile(".*/warehouses/(.+)");
-  public static final Pattern HTTP_ENDPOINT_PATH_PATTERN = Pattern.compile(".*/endpoints/(.+)");
+  public static final Pattern HTTP_WAREHOUSE_PATH_PATTERN =
+      Pattern.compile(".*/warehouses/([^?&]+).*");


we also allow some other http paths, I guess /endpoints/ or something

gopalldb

Some minor comments

msrathore-db added 2 commits March 25, 2026 00:42

msrathore-db force-pushed the spog-support-v2 branch from ddcf35e to df5451e Compare March 25, 2026 06:03

msrathore-db added 3 commits April 6, 2026 11:34

Merge remote-tracking branch 'upstream/main' into spog-support-v2

6330ae5

msrathore-db requested a review from gopalldb April 6, 2026 07:21

gopalldb reviewed Apr 7, 2026

View reviewed changes

msrathore-db force-pushed the spog-support-v2 branch from a27cd13 to 0f16824 Compare April 7, 2026 06:09

gopalldb reviewed Apr 7, 2026

View reviewed changes

gopalldb approved these changes Apr 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SPOG support for ?o= routing in httpPath#1316

Add SPOG support for ?o= routing in httpPath#1316
msrathore-db wants to merge 6 commits intodatabricks:mainfrom
msrathore-db:spog-support-v2

msrathore-db commented Mar 24, 2026 •

edited by atlassian bot

Loading

Uh oh!

gopalldb Apr 7, 2026

Uh oh!

msrathore-db Apr 7, 2026

Uh oh!

gopalldb Apr 7, 2026

Uh oh!

gopalldb Apr 7, 2026

Uh oh!

gopalldb Apr 7, 2026

Uh oh!

gopalldb left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

msrathore-db commented Mar 24, 2026 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

What does NOT get the header

Context

Test plan

Uh oh!

gopalldb Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

msrathore-db Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

gopalldb Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

gopalldb Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

gopalldb Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

gopalldb left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

msrathore-db commented Mar 24, 2026 •

edited by atlassian bot

Loading