You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jacek Laskowski (Jira)" <ji...@apache.org> on 2021/09/30 12:13:00 UTC
[jira] [Commented] (SPARK-36904) The specified datastore driver
("org.postgresql.Driver") was not found in the CLASSPATH
[ https://issues.apache.org/jira/browse/SPARK-36904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422727#comment-17422727 ]
Jacek Laskowski commented on SPARK-36904:
-----------------------------------------
The solution from [this answer on SO|https://stackoverflow.com/a/62588338/1305344] didn't work for me and am still facing the exception.
{code}
./build/mvn \
-Pyarn,kubernetes,hadoop-cloud,hive,hive-thriftserver \
-DskipTests \
dependency:purge-local-repository \
clean install
{code}
{code}
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
... 190 more
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
... 192 more
{code}
> The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-36904
> URL: https://issues.apache.org/jira/browse/SPARK-36904
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.0
> Environment: Spark 3.2.0 (RC6)
> {code:java}
> $ ./bin/spark-shell --version
> Welcome to
> ____ __
> / __/__ ___ _____/ /__
> _\ \/ _ \/ _ `/ __/ '_/
> /___/ .__/\_,_/_/ /_/\_\ version 3.2.0
> /_/
> Using Scala version 2.12.15, OpenJDK 64-Bit Server VM, 11.0.12
> Branch heads/v3.2.0-rc6
> Compiled by user jacek on 2021-09-30T10:44:35Z
> Revision dde73e2e1c7e55c8e740cb159872e081ddfa7ed6
> Url https://github.com/apache/spark.git
> Type --help for more information.
> {code}
> Built from [https://github.com/apache/spark/commits/v3.2.0-rc6] using the following command:
> {code:java}
> $ ./build/mvn \
> -Pyarn,kubernetes,hadoop-cloud,hive,hive-thriftserver \
> -DskipTests \
> clean install
> {code}
> {code:java}
> $ java -version
> openjdk version "11.0.12" 2021-07-20
> OpenJDK Runtime Environment Temurin-11.0.12+7 (build 11.0.12+7)
> OpenJDK 64-Bit Server VM Temurin-11.0.12+7 (build 11.0.12+7, mixed mode) {code}
> Reporter: Jacek Laskowski
> Priority: Critical
> Attachments: exception.txt
>
>
> It looks similar to [hivethriftserver built into spark3.0.0. is throwing error "org.postgresql.Driver" was not found in the CLASSPATH|https://stackoverflow.com/q/62534653/1305344], but reporting here for future reference.
> After I built the 3.2.0 (RC6) I ran `spark-shell` to execute `sql("describe table covid_19")`. That gave me the exception (a full version is attached):
> {code}
> Caused by: java.lang.reflect.InvocationTargetException: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
> at jdk.internal.reflect.GeneratedConstructorAccessor64.newInstance(Unknown Source)
> at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
> at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:330)
> at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:203)
> at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:162)
> at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:285)
> at jdk.internal.reflect.GeneratedConstructorAccessor63.newInstance(Unknown Source)
> at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
> at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
> at org.datanucleus.NucleusContextHelper.createStoreManagerForProperties(NucleusContextHelper.java:133)
> at org.datanucleus.PersistenceNucleusContextImpl.initialise(PersistenceNucleusContextImpl.java:422)
> at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:817)
> ... 171 more
> Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
> at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
> at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
> at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
> ... 187 more
> Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("org.postgresql.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
> at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
> at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
> at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
> ... 189 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org