You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by durga <du...@gmail.com> on 2014/12/20 00:47:28 UTC
java.sql.SQLException: No suitable driver found
Hi I am facing an issue with mysql jars with spark-submit.
I am not running in yarn mode.
spark-submit --jars $(echo mysql-connector-java-5.1.34-bin.jar | tr ' ' ',')
--class com.abc.bcd.GetDBSomething myjar.jar "abc" "bcd"
Any help is really appreciated.
Thanks,
-D
14/12/19 23:42:10 INFO SparkUI: Started SparkUI at
http://ec2-54-90-85-197.compute-1.amazonaws.com:4040
14/12/19 23:42:10 INFO SparkContext: Added JAR
file:/root/abc/mysql-connector-java-5.1.34-bin.jar at
http://10.61.187.176:57956/jars/mysql-connector-java-5.1.34-bin.jar with
timestamp 1419032530456
14/12/19 23:42:10 INFO SparkContext: Added JAR file:/root/abc/myjar.jar at
http://10.61.187.176:57956/jars/filesplitter_2.10-1.0.jar with timestamp
1419032530459
14/12/19 23:42:10 INFO AppClient$ClientActor: Connecting to master
spark://ec2-54-90-85-197.compute-1.amazonaws.com:7077...
Exception in thread "main" java.sql.SQLException: No suitable driver found
for "jdbc:mysql://192.168.20.45:3306/abcdb"?user="root"&password="admin"
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: java.sql.SQLException: No suitable driver found
Posted by Michael Armbrust <mi...@databricks.com>.
With JDBC you often need to load the class so it can register the driver at
the beginning of your program. Usually this is something like:
Class.forName("com.mysql.jdbc.Driver");
On Fri, Dec 19, 2014 at 3:47 PM, durga <du...@gmail.com> wrote:
> Hi I am facing an issue with mysql jars with spark-submit.
>
> I am not running in yarn mode.
>
> spark-submit --jars $(echo mysql-connector-java-5.1.34-bin.jar | tr ' '
> ',')
> --class com.abc.bcd.GetDBSomething myjar.jar "abc" "bcd"
>
> Any help is really appreciated.
>
> Thanks,
> -D
>
> 14/12/19 23:42:10 INFO SparkUI: Started SparkUI at
> http://ec2-54-90-85-197.compute-1.amazonaws.com:4040
> 14/12/19 23:42:10 INFO SparkContext: Added JAR
> file:/root/abc/mysql-connector-java-5.1.34-bin.jar at
> http://10.61.187.176:57956/jars/mysql-connector-java-5.1.34-bin.jar with
> timestamp 1419032530456
> 14/12/19 23:42:10 INFO SparkContext: Added JAR file:/root/abc/myjar.jar at
> http://10.61.187.176:57956/jars/filesplitter_2.10-1.0.jar with timestamp
> 1419032530459
> 14/12/19 23:42:10 INFO AppClient$ClientActor: Connecting to master
> spark://ec2-54-90-85-197.compute-1.amazonaws.com:7077...
> Exception in thread "main" java.sql.SQLException: No suitable driver found
> for "jdbc:mysql://192.168.20.45:3306/abcdb"?user="root"&password="admin"
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: java.sql.SQLException: No suitable driver found
Posted by Michael Orr <mi...@gmail.com>.
Here is a script I use to submit a directory of jar files. It assumes jar files are in target/dependency or lib/
DRIVER_PATH=
DEPEND_PATH=
if [ -d "lib" ]; then
DRIVER_PATH="lib"
DEPEND_PATH="lib"
else
DRIVER_PATH="target"
DEPEND_PATH="target/dependency"
fi
DEPEND_JARS=log4j.properties
for f in `ls $DEPEND_PATH`; do DEPEND_JARS=$DEPEND_JARS,$DEPEND_PATH/$f; done
$SPARK_HOME/bin/spark-submit \
--class $1 \
--master yarn-client \
--num-executors 1 \
--driver-memory 4g \
--executor-memory 16g \
--executor-cores 4 \
--jars $DEPEND_JARS \
$DRIVER_PATH/core-ingest-*.jar ${*:2}
You would run it with a command like:
./run.sh class.to.submit arg1 arg2 …
> On Dec 22, 2014, at 1:11 AM, durga <du...@gmail.com> wrote:
>
> One more question.
>
> How would I submit additional jars to the spark-submit job. I used --jars
> option, it seems it is not working as explained earlier.
>
> Thanks for the help,
>
> -D
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20805.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: java.sql.SQLException: No suitable driver found
Posted by durga <du...@gmail.com>.
One more question.
How would I submit additional jars to the spark-submit job. I used --jars
option, it seems it is not working as explained earlier.
Thanks for the help,
-D
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20805.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: java.sql.SQLException: No suitable driver found
Posted by durga <du...@gmail.com>.
Hi All,
I tried to make combined.jar in shell script . it is working when I am using
spark-shell. But for the spark-submit it is same issue.
Help is highly appreciated.
Thanks
-D
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20804.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org