You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/12/24 05:09:00 UTC
[jira] [Commented] (SPARK-26383) NPE when use DataFrameReader.jdbc
with wrong URL
[ https://issues.apache.org/jira/browse/SPARK-26383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728146#comment-16728146 ]
Hyukjin Kwon commented on SPARK-26383:
--------------------------------------
Yea, can you open a PR to fix the error message?
> NPE when use DataFrameReader.jdbc with wrong URL
> ------------------------------------------------
>
> Key: SPARK-26383
> URL: https://issues.apache.org/jira/browse/SPARK-26383
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: clouds
> Priority: Minor
>
> When passing wrong url to jdbc:
> {code:java}
> val opts = Map(
> "url" -> "jdbc:mysql://localhost/db",
> "dbtable" -> "table",
> "driver" -> "org.postgresql.Driver"
> )
> var df = spark.read.format("jdbc").options(opts).load
> {code}
> It would throw an NPE instead of complaining about connection failed. (Note url and driver not matched here)
> {code:java}
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:71)
> at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
> at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
> at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
> at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
> {code}
> as [postgresql jdbc driver document|https://jdbc.postgresql.org/development/privateapi/org/postgresql/Driver.html#connect-java.lang.String-java.util.Properties-] saying, The driver should return "null" if it realizes it is the wrong kind of driver to connect to the given URL.
> while [ConnectionFactory|https://github.com/apache/spark/blob/e743e848484bf7d97e1b4f33ea83f8520ae7da04/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L56] would not check if conn is null.
> {code:java}
> val conn: Connection = JdbcUtils.createConnectionFactory(options)()
> {code}
> and trying to close the conn anyway
> {code:java}
> try {
> ...
> } finally {
> conn.close()
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org