Add SPOG (Custom URL) routing support via x-databricks-org-id header#347
Merged
msrathore-db merged 8 commits intomainfrom Apr 21, 2026
Merged
Add SPOG (Custom URL) routing support via x-databricks-org-id header#347msrathore-db merged 8 commits intomainfrom
msrathore-db merged 8 commits intomainfrom
Conversation
The U2M flow uses PKCE (public app) and should not send a client secret. Previously, ClientSecret was always set to "" on the oauth2.Config, which caused Go's oauth2 library to send an empty client_secret via Basic auth. The OIDC server rejects this with "Public app should not use a client secret". Only set ClientSecret when it's non-empty, so public apps use the "none" token endpoint auth method as intended. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Feature flags:
- Fix endpoint path: /api/2.0/feature-flags -> /api/2.0/connector-service/feature-flags/GOLANG/{version}
- Fix response parsing: map format -> array of {name, value} entries
- Add extraHeaders for SPOG routing (x-databricks-org-id)
- Extract ?o=<workspaceId> from httpPath in connector
U2M OAuth:
- Don't set ClientSecret for public apps (PKCE)
- Force AuthStyleInParams to prevent Basic auth with empty password
- Server rejects "Public app should not use a client secret" otherwise
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Mirrors equivalent logging added to OSS JDBC and pysql. Emits at DEBUG level in three paths of extractSpogHeaders: 1. Malformed query string in httpPath — log and skip. 2. httpPath has "?" but no ?o= param — log and skip. 3. Injection happens — log the extracted workspace ID so customers diagnosing SPOG routing can confirm the header was added. Also adds a detailed docstring explaining the role this header plays: Thrift routing stays URL-driven via ?o= in httpPath; only the separate endpoints (telemetry, feature flags) need the header for account-level routing on SPOG hosts. Helps with customer support: when a customer reports "SPOG isn't routing correctly", they can enable DEBUG logging and immediately see whether the driver saw their ?o= value. Signed-off-by: Madhavendra Rathore Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
# Conflicts: # connector.go # telemetry/config.go # telemetry/config_test.go # telemetry/driver_integration.go # telemetry/featureflag.go # telemetry/featureflag_test.go
… API churn
Replace the per-function extraHeaders parameter threading (added during
the previous merge with main) with a minimal http.Client wrapper in the
top-level dbsql package.
Before (5 files, including 3 in telemetry/*):
connector.go extracted ?o=<workspaceId>, then passed it as an
ExtraHeaders field through telemetry.TelemetryInitOptions →
isTelemetryEnabled → featureFlagCache.isTelemetryEnabled →
fetchFeatureFlag, where it was applied to the outbound request.
The telemetry push path (telemetry/exporter.go) was NOT covered.
After (2 files — connector.go and auth/oauth/u2m/u2m.go):
connector.go extracts ?o= as before, but now wraps the driver's
*http.Client with a headerInjectingTransport that sets the SPOG
header on every outbound request through that client. Passes the
wrapped client (not c.client) into TelemetryInitOptions.HTTPClient.
Advantages:
- telemetry/*.go files revert to identical-to-main. No API churn.
- Both feature-flag and telemetry-push paths automatically get the
SPOG header (previously only feature-flag did).
- Future HTTP paths that reuse the telemetry http.Client inherit
SPOG routing for free.
Thrift is unaffected: it uses c.client directly (not the wrapper) and
routes via ?o= in the URL path. The transport wrapper is only applied
to the HTTP client handed to telemetry.
Signed-off-by: Madhavendra Rathore
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Earlier commits in this branch (23697e5, 0ec7e06) modified u2m.go to avoid sending an empty client_secret on the PKCE public-app flow, citing server rejection with "Public app should not use a client secret". Empirical verification (2026-04-21): - Prod Legacy (adb-6436897454825492.12.azuredatabricks.net): PASS with unpatched u2m.go — server accepts the request. - Stg Legacy (adb-7064161269814046.2.staging.azuredatabricks.net): FAIL with 400 Bad Request on unpatched u2m.go. Since the production server tolerates the current behavior, the patch isn't strictly required for customers. Reverting to keep the PR minimal and matching upstream main exactly for this file. If staging server strictness later rolls out to prod, we can re-add this fix then. Signed-off-by: Madhavendra Rathore Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
msrathore-db
changed the title
Covers extractSpogHeaders (8 cases: missing/empty query, valid o=, missing o=, empty value, multi-param, duplicate o=, bare `?`) and headerInjectingTransport (injection, caller-set not overridden, other headers preserved, original client untouched). Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
samikshya-db
approved these changes
Apr 21, 2026
Collaborator
samikshya-db
left a comment
samikshya-db
left a comment
There was a problem hiding this comment.
Thanks for this, please add in nodejs too 🙏
2 tasks
vikrantpuppala
added a commit
that referenced
this pull request
Apr 21, 2026
## Summary Bump `DriverVersion` to `1.11.0` and add the v1.11.0 section to `CHANGELOG.md`. ### Changes since v1.10.0 - Enable telemetry by default with DSN-controlled priority (#320, #321, #322, #349) - Add SPOG (Custom URL) routing support via `x-databricks-org-id` header (#347) - Add statement-level query tag support (#341) - Add AI coding agent detection to User-Agent header (#326) - Fix CloudFetch returning stale column names from cached results (#351) - Fix resource leak: close staging Rows in execStagingOperation (#325) Internal/infra-only changes are omitted from the user-facing notes (CI hardening, dependabot bumps, CODEOWNERS). ## Test plan - [x] `go build ./...` clean - [x] `go test ./... -count=1 -short` passes locally ## Next steps after merge 1. Tag the merge commit as `v1.11.0` and push the tag 2. Trigger `peco-databricks-sql-go` in secure-public-registry-releases-eng with `ref=v1.11.0`, `dry-run=true` to verify 3. Re-run with `dry-run=false` for the actual release NO_CHANGELOG=true This pull request was AI-assisted by Isaac. Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On SPOG (Custom URL / account-level) workspaces,
httpPathhas the form/sql/1.0/warehouses/<id>?o=<workspaceId>. The?o=parameter routes Thrift calls correctly via the URL, but other endpoints (telemetry push, feature-flag check) run on separate hosts and needx-databricks-org-idas an HTTP header to route to the right workspace. Without it, those requests 404 or get misrouted on SPOG hosts.Change
All contained in
connector.go:extractSpogHeaders(httpPath string) map[string]string— parses the?o=query param usingurl.ParseQuery(stdlib, not regex). Returns{"x-databricks-org-id": "<workspaceId>"}ornil. Three DEBUG log paths cover: malformed query, missing?o=, and successful extraction.headerInjectingTransport— a lightweighthttp.RoundTripperwrapper that clones the request per the contract and sets the provided headers if not already set by the caller.withSpogHeaders(base *http.Client, headers map[string]string) *http.Client— returns a new client with the same settings but a wrapped transport.In
Connect(), whenextractSpogHeadersreturns non-nil, the driver passes a wrapped client intoTelemetryInitOptions.HTTPClient. The wrapped client is used for both the feature-flag check and the telemetry push. The driver's ownc.clientis left alone, so Thrift routing (which uses?o=in the URL) is unaffected.Why a transport wrapper instead of threading a parameter
An earlier revision of this PR threaded an
extraHeadersparameter throughtelemetry.TelemetryInitOptions→isTelemetryEnabled→featureFlagCache.isTelemetryEnabled→fetchFeatureFlag. That approach:config.go,featureflag.go,driver_integration.go).telemetry/exporter.go(telemetry push) still sentContent-Typeas the only header — SPOG routing would 404 at push time.The RoundTripper wrapper:
telemetry/*identical toorigin/main. Zero API churn.x-databricks-org-idfor some reason, the wrapper does not override).Endpoints covered
x-databricks-org-id?/api/2.0/connector-service/feature-flags/GOLANG/{v}c.clientdirectly; already routes via URL)login.microsoftonline.com)Verification
go build ./...clean.Note on the earlier
auth/oauth/u2m/u2m.gochangeThe two earliest commits on this branch (
23697e5,0ec7e06) modifiedauth/oauth/u2m/u2m.goto avoid sending an emptyclient_secreton the PKCE public-app flow, documented as a fix for the server's"Public app should not use a client secret"rejection. That change was empirically verified to not be needed (see commit3576c92which reverts it):adb-6436897454825492.12.azuredatabricks.net): Unpatchedu2m.goPASSES end-to-end. Server accepts the emptyclient_secret.adb-7064161269814046.2.staging.azuredatabricks.net): Unpatchedu2m.goFAILS withunexpected HTTP status 400 Bad Requestduring token exchange.Since production tolerates the current behavior, customers aren't impacted. Keeping this PR minimal; if staging-level strictness later rolls out to prod, we can re-add the u2m fix then.
Test plan
go test ./...)x-databricks-org-idon feature-flag check and telemetry push against a SPOG workspace (not gated by this PR, but prerequisite for SPOG telemetry to work at all)extractSpogHeadersreturns nil)