Skip to content

Commit 9e96f22

Browse files
authored
Fix getSchemas returning JDBC-escaped catalog name in SEA mode (#1365)
## Summary Fixes SEA/Thrift parity issue where `getSchemas()` returns `comparator\_tests` (with escaped underscore) instead of `comparator_tests` in the `TABLE_CATALOG` column when using SEA mode (`useThriftClient=0`). ### Root Cause In `DatabricksMetadataQueryClient.listSchemas()`, the `catalog` parameter from the JDBC caller can contain JDBC escape sequences (e.g., `comparator\_tests` where `\_` escapes the `_` wildcard). The `CommandBuilder` correctly strips these escapes for the SQL query (`SHOW SCHEMAS IN \`comparator_tests\``), but the **original escaped value** was passed to `getSchemasResult()`. Since `SHOW SCHEMAS IN \`catalog\`` doesn't return a `catalog` column from the server (only `SHOW SCHEMAS IN ALL CATALOGS` does), the client falls back to populating `TABLE_CATALOG` from the passed parameter — inserting the JDBC-escaped value. ### Fix Strip JDBC escapes from the `catalog` parameter via `WildcardUtil.stripJdbcEscapes()` before passing it to `getSchemasResult()`. ### Verification The `EscapedUnderscoreTest` in `../example` can be used to verify end-to-end: ```bash cd ../example mvn exec:java -Dexec.mainClass="com.databricks.example.EscapedUnderscoreTest" ``` Expected: Both Thrift and SEA should return `comparator_tests` (without backslash) in TABLE_CATALOG for both escaped and unescaped input. ## Test plan - [x] New unit test: `testListSchemasWithEscapedUnderscoreCatalog` — verifies TABLE_CATALOG is unescaped - [x] All 52 existing metadata query client tests pass - [ ] Manual verification with `EscapedUnderscoreTest` against live environment This pull request was AI-assisted by Isaac. --------- Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
1 parent d8bc48a commit 9e96f22

3 files changed

Lines changed: 79 additions & 1 deletion

File tree

NEXT_CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
- Fixed primitive types within complex types (ARRAY, MAP, STRUCT) not being correctly parsed when Arrow serialization uses alternate formats: TIMESTAMP/TIMESTAMP_NTZ as epoch microseconds or component arrays, and BINARY as base64-encoded strings.
1111
- Fixed `PARSE_SYNTAX_ERROR` for column names containing special characters (e.g., dots) when `EnableBatchedInserts` is enabled, by re-quoting column names with backticks in reconstructed multi-row INSERT statements.
1212
- Fixed Volume ingestion for SEA mode, which was broken due to statement being closed prematurely.
13+
- Fixed escaped pattern characters in catalogName for `getSchemas`, as returned catalogName should be unescaped.
1314

1415
---
1516
*Note: When making changes, please add your change under the appropriate section

src/main/java/com/databricks/jdbc/dbclient/impl/sqlexec/DatabricksMetadataQueryClient.java

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
import com.databricks.jdbc.common.MetadataOperationType;
1010
import com.databricks.jdbc.common.StatementType;
1111
import com.databricks.jdbc.common.util.JdbcThreadUtils;
12+
import com.databricks.jdbc.common.util.WildcardUtil;
1213
import com.databricks.jdbc.dbclient.IDatabricksClient;
1314
import com.databricks.jdbc.dbclient.IDatabricksMetadataClient;
1415
import com.databricks.jdbc.dbclient.impl.common.CommandConstants;
@@ -98,9 +99,16 @@ public DatabricksResultSet listSchemas(
9899
new CommandBuilder(catalog, session).setSchemaPattern(schemaNamePattern);
99100
String SQL = commandBuilder.getSQLString(CommandName.LIST_SCHEMAS);
100101
LOGGER.debug("SQL command to fetch schemas: {}", SQL);
102+
// Strip JDBC escape sequences from catalog for the result set TABLE_CATALOG column.
103+
// SHOW SCHEMAS IN `catalog` doesn't return a catalog column from the server,
104+
// so the client populates it from this parameter. Without stripping, JDBC-escaped
105+
// underscores (\_) would appear in the result (e.g., "comparator\_tests" instead
106+
// of "comparator_tests").
107+
String resultCatalog =
108+
catalog != null ? WildcardUtil.stripJdbcEscapes(catalog).toLowerCase() : null;
101109
try {
102110
return metadataResultSetBuilder.getSchemasResult(
103-
getResultSet(SQL, session, MetadataOperationType.GET_SCHEMAS), catalog);
111+
getResultSet(SQL, session, MetadataOperationType.GET_SCHEMAS), resultCatalog);
104112
} catch (SQLException e) {
105113
if (catalog == null && PARSE_SYNTAX_ERROR_SQL_STATE.equals(e.getSQLState())) {
106114
// This is a fallback for the case where the SQL command fails with "syntax error at or near
@@ -426,6 +434,16 @@ public DatabricksResultSet listCrossReferences(
426434
return metadataResultSetBuilder.getCrossRefsResult(new ArrayList<>());
427435
}
428436

437+
// When all three foreign-side parameters are null, SHOW FOREIGN KEYS cannot be constructed.
438+
// Match Thrift server behavior which delegates to getExportedKeys in this case
439+
// (returns 0 rows since exported keys are not tracked in DBSQL).
440+
if (foreignCatalog == null && foreignSchema == null && foreignTable == null) {
441+
LOGGER.debug(
442+
"All foreign key parameters are null for getCrossReference, "
443+
+ "returning empty result set to match Thrift behavior.");
444+
return metadataResultSetBuilder.getCrossRefsResult(new ArrayList<>());
445+
}
446+
429447
CommandBuilder commandBuilder =
430448
new CommandBuilder(foreignCatalog, session).setSchema(foreignSchema).setTable(foreignTable);
431449
String SQL = commandBuilder.getSQLString(CommandName.LIST_FOREIGN_KEYS);

src/test/java/com/databricks/jdbc/dbclient/impl/sqlexec/DatabricksMetadataQueryClientTest.java

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -458,6 +458,50 @@ void testListSchemas(String sqlStatement, String schema, String description) thr
458458
((DatabricksResultSetMetaData) actualResult.getMetaData()).getTotalRows(), 1, description);
459459
}
460460

461+
/**
462+
* Tests that getSchemas with a JDBC-escaped, mixed-case catalog name returns the unescaped,
463+
* lowercased catalog name in the TABLE_CATALOG column. This reproduces the SEA/Thrift parity
464+
* issue where SHOW SCHEMAS IN `catalog` doesn't return a catalog column from the server, so the
465+
* client populates it from the parameter — which must be unescaped and lowercased.
466+
*/
467+
@Test
468+
void testListSchemasWithEscapedUnderscoreCatalog() throws SQLException {
469+
String escapedCatalog = "Comparator\\_Tests";
470+
String expectedCatalog = "comparator_tests";
471+
// CommandBuilder strips escapes for SQL: SHOW SCHEMAS IN `Comparator_Tests`
472+
String expectedSQL = "SHOW SCHEMAS IN `Comparator_Tests`";
473+
474+
when(session.getComputeResource()).thenReturn(mockedComputeResource);
475+
DatabricksMetadataQueryClient metadataClient = new DatabricksMetadataQueryClient(mockClient);
476+
when(mockClient.executeStatement(
477+
eq(expectedSQL),
478+
eq(mockedComputeResource),
479+
any(),
480+
eq(StatementType.METADATA),
481+
eq(session),
482+
any(),
483+
any(MetadataOperationType.class)))
484+
.thenReturn(mockedResultSet);
485+
when(mockedResultSet.next()).thenReturn(true, false);
486+
when(mockedResultSet.getObject("databaseName")).thenReturn("default");
487+
doReturn(2).when(mockedMetaData).getColumnCount();
488+
doReturn(SCHEMA_COLUMN.getResultSetColumnName()).when(mockedMetaData).getColumnName(1);
489+
doReturn(CATALOG_COLUMN.getResultSetColumnName()).when(mockedMetaData).getColumnName(2);
490+
when(mockedResultSet.getMetaData()).thenReturn(mockedMetaData);
491+
// SHOW SCHEMAS IN `catalog` doesn't return a catalog column — the client must populate it
492+
when(mockedResultSet.findColumn(CATALOG_RESULT_COLUMN.getResultSetColumnName()))
493+
.thenThrow(DatabricksSQLException.class);
494+
495+
DatabricksResultSet actualResult = metadataClient.listSchemas(session, escapedCatalog, null);
496+
497+
assertTrue(actualResult.next());
498+
// TABLE_CATALOG (column 2) should be unescaped and lowercased
499+
assertEquals(
500+
expectedCatalog,
501+
actualResult.getObject(2),
502+
"TABLE_CATALOG should be unescaped and lowercased to match Thrift behavior");
503+
}
504+
461505
@Test
462506
void testListSchemasNullCatalog() throws SQLException {
463507
when(session.getComputeResource()).thenReturn(mockedComputeResource);
@@ -717,6 +761,21 @@ void testListCrossReferences() throws Exception {
717761
}
718762
}
719763

764+
/**
765+
* Tests that getCrossReference returns empty result set (not an exception) when all three
766+
* foreign-side parameters are null. Matches Thrift server behavior where null foreign table
767+
* delegates to getExportedKeys which returns empty in DBSQL.
768+
*/
769+
@Test
770+
void testListCrossReferences_allForeignParamsNull_returnsEmpty() throws Exception {
771+
DatabricksMetadataQueryClient metadataClient = new DatabricksMetadataQueryClient(mockClient);
772+
773+
DatabricksResultSet result =
774+
metadataClient.listCrossReferences(
775+
session, TEST_CATALOG, TEST_SCHEMA, TEST_TABLE, null, null, null);
776+
assertFalse(result.next(), "Should return empty when all foreign params are null, not throw");
777+
}
778+
720779
@Test
721780
void testListCrossReferences_throwsParseSyntaxError() throws Exception {
722781
DatabricksSQLException exception =

0 commit comments

Comments
 (0)