Spark with DF throws No suitable driver found for jdbc:oracle: after first call

[Mich Talebzadeh <mi...@peridale.co.uk>]

First I put the Oracle JAR file in spark-shell start up and also in
CLASSPATH

 

spark-shell --master yarn --deploy-mode client --driver-class-path
/home/hduser/jars/ojdbc6.jar

 

Now it shows clearly that load call is successful as shown in bold so it can
use the driver. However, the next method call channels.collect throws error!

 

scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc)

sqlContext: org.apache.spark.sql.SQLContext =
org.apache.spark.sql.SQLContext@33fb6c08

 

scala> val channels = sqlContext.read.format("jdbc").options(

     |     Map("url" -> "jdbc:oracle:thin:@rhes564:1521:mydb",

     |     "dbtable" -> "(select * from sh.channels where channel_id = 14)",

     |     "user" -> "sh",

     |     "password" -> "sh")).load

channels: org.apache.spark.sql.DataFrame = [CHANNEL_ID: decimal(0,-127),
CHANNEL_DESC: string, CHANNEL_CLASS: string, CHANNEL_CLASS_ID:
decimal(0,-127), CHANNEL_TOTAL: string, CHANNEL_TOTAL_ID: decimal(0,-127)]

 

scala> channels.collect

16/02/12 22:48:19 ERROR TaskSetManager: Task 0 in stage 0.0 failed 4 times;
aborting job

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in
stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0
(TID 3, rhes564): java.sql.SQLException: No suitable driver found for
jdbc:oracle:thin:@rhes564:1521:mydb

 

Dr Mich Talebzadeh

 

LinkedIn
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU
rV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr
V8Pw

 

 <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com

 

NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Technology Ltd, its subsidiaries nor their
employees accept any responsibility.