You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ivan-leventsov (via GitHub)" <gi...@apache.org> on 2026/06/30 09:38:58 UTC
[PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
ivan-leventsov opened a new pull request, #56898:
URL: https://github.com/apache/spark/pull/56898
### What changes were proposed in this pull request?
When using the built-in JDBC `TableCatalog` over Oracle, the driver's table listing returns objects that cannot be read as tables, for example a synonym that resolves to a PL/SQL procedure/function/package, or an invalid view. Probing such an object (`tableExists` via `SELECT 1 FROM <obj> WHERE 1=0`, or schema resolution via `SELECT * FROM <obj> WHERE 1=0`) raises `ORA-04044` ("procedure, function, package, or type is not allowed here") or `ORA-04063` ("... has errors"). These are currently unclassified, so `tableExists` throws instead of returning `false`, and table resolution surfaces a raw `FAILED_JDBC` error.
This PR adds a new `JdbcDialect.isNotSelectableObjectException` predicate (default `false`; `OracleDialect` recognizes `ORA-04044`/`ORA-04063`), kept separate from `isObjectNotFoundException` ("object does not exist"). With it:
- `JdbcUtils.tableExists` returns `false` for such objects instead of throwing;
- `JDBCRDD.resolveTable` throws a dedicated, clear error condition `JDBC_OBJECT_NOT_SELECTABLE` instead of a generic failure.
### Why are the changes needed?
A schema legitimately contains synonyms/views that are not selectable as tables. Probing them should not surface a raw external-engine error or make existence checks throw; such objects should be reported as not a readable table.
### Does this PR introduce _any_ user-facing change?
Yes. For a JDBC catalog over Oracle:
- reading an object that is not selectable as a table now fails with the dedicated error condition `JDBC_OBJECT_NOT_SELECTABLE` (SQLSTATE `42000`) instead of a raw `FAILED_JDBC` error;
- `tableExists` returns `false` for such an object instead of throwing.
### How was this patch tested?
New cases in `org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite` (docker-integration-tests), covering a synonym to a procedure, a synonym to a broken function (both `ORA-04044`), and an invalid view (`ORA-04063`). Each asserts that `tableExists` returns `false` and that reading the object fails with `JDBC_OBJECT_NOT_SELECTABLE`.
### Was this patch authored or co-authored using generative AI tooling?
Yes, co-authored using Cursor (AI assistant).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "ivan-leventsov (via GitHub)" <gi...@apache.org>.
ivan-leventsov commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3499881401
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.3.0")
+ def isNotSelectableObjectException(e: SQLException): Boolean = false
Review Comment:
Done, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3498720168
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.3.0")
Review Comment:
nvm, this is resolved already
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "ivan-leventsov (via GitHub)" <gi...@apache.org>.
ivan-leventsov commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3499875210
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala:
##########
@@ -54,6 +54,13 @@ private case class OracleDialect() extends JdbcDialect with SQLConfHelper with N
e.getMessage.contains("ORA-39165")
}
+ override def isNotSelectableObjectException(e: SQLException): Boolean = {
+ // ORA-04044: object is not a table (e.g. a synonym to a procedure/function/package).
+ e.getMessage.contains("ORA-04044") ||
Review Comment:
Yeah, makes sense, but I'd leave it as is in this PR and keep it consistent with the existing implementation to minimize risk and avoid potential side effects from this small change (for example, the behavior might differ across driver versions etc..)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #56898: [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables
URL: https://github.com/apache/spark/pull/56898
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3498713808
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala:
##########
@@ -54,6 +54,13 @@ private case class OracleDialect() extends JdbcDialect with SQLConfHelper with N
e.getMessage.contains("ORA-39165")
}
+ override def isNotSelectableObjectException(e: SQLException): Boolean = {
+ // ORA-04044: object is not a table (e.g. a synonym to a procedure/function/package).
+ e.getMessage.contains("ORA-04044") ||
Review Comment:
Optional, non-blocking: matching on `e.getMessage.contains(...)` is fragile to message format/locale; `e.getErrorCode == 4044` (and `4063`) would be more robust. This mirrors the existing `isObjectNotFoundException` here, so it's consistent either way — only worth changing if you want to harden both together.
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.3.0")
Review Comment:
`4.2.0` is already released (branch-4.2 is on `4.2.1-SNAPSHOT`). A master PR that backports normally ships first in `branch-4.x` (`4.3.0-SNAPSHOT`), so this new API should be `@Since("4.3.0")`. (The other `@Since("4.2.0")` annotations in this file predate the 4.2 cut.)
```suggestion
@Since("4.3.0")
```
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.3.0")
+ def isNotSelectableObjectException(e: SQLException): Boolean = false
Review Comment:
This is a general opt-in hook (base default `false`, like `isObjectNotFoundException`), but only Oracle overrides it today. Consider a short Scaladoc stating the contract, so the Oracle-only impl reads as the first adopter and the next dialect author knows when to override. Placed above the `@Since` line, e.g.:
```scala
/**
* Returns true if the given exception indicates the object exists but cannot be read as a
* table or view (e.g. a synonym that resolves to a procedure, or an invalid view), as opposed
* to not existing at all (see `isObjectNotFoundException`). Dialects override this to recognize
* their own error codes; the default is false.
*/
```
(Left as an illustrative block rather than a one-click suggestion so it doesn't collide with the `@Since` fix above.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3497914709
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.2.0")
Review Comment:
`dev/next_version_candidates.py` reports the current `branch-4.x` at `4.3.0` (master `5.0.0`), so a method added now first ships in `4.3.0` — `4.2.0` looks like it predates the recent version bump (the existing `@Since("4.2.0")` entries in this file are from the prior cycle).
```suggestion
@Since("4.3.0")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "ivan-leventsov (via GitHub)" <gi...@apache.org>.
ivan-leventsov commented on code in PR #56898:
URL: https://github.com/apache/spark/pull/56898#discussion_r3497970382
##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##########
@@ -808,6 +808,9 @@ abstract class JdbcDialect extends Serializable with Logging {
Option(e.getSQLState).exists(_.startsWith("42"))
}
+ @Since("4.2.0")
Review Comment:
Thanks, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57778][SQL] Handle Oracle JDBC objects that are not selectable as tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #56898:
URL: https://github.com/apache/spark/pull/56898#issuecomment-4848949328
thanks, merging to master/4.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org