You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by gavin zhang <ga...@gmail.com> on 2015/03/22 15:32:21 UTC
How to use DataFrame with MySQL
OK, I have known that I could use jdbc connector to create DataFrame with
this command:
val jdbcDF = sqlContext.load("jdbc", Map("url" ->
"jdbc:mysql://localhost:3306/video_rcmd?user=root&password=123456",
"dbtable" -> "video"))
But I got this error:
java.sql.SQLException: No suitable driver found for ...
And I have tried to add jdbc jar to spark_path with both commands but
failed:
- spark-shell --jars mysql-connector-java-5.0.8-bin.jar
- SPARK_CLASSPATH=mysql-connector-java-5.0.8-bin.jar spark-shell
My Spark version is 1.3.0 while
`Class.forName("com.mysql.jdbc.Driver").newInstance` is worked.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-tp22178.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to use DataFrame with MySQL
Posted by Michael Armbrust <mi...@databricks.com>.
Can you try adding "driver" -> "com.mysql.jdbc.Driver"? This causes the
class to get loaded both locally and the workers so that it can register
with JDBC.
On Sun, Mar 22, 2015 at 7:32 AM, gavin zhang <ga...@gmail.com> wrote:
> OK, I have known that I could use jdbc connector to create DataFrame with
> this command:
>
> val jdbcDF = sqlContext.load("jdbc", Map("url" ->
> "jdbc:mysql://localhost:3306/video_rcmd?user=root&password=123456",
> "dbtable" -> "video"))
>
> But I got this error:
>
> java.sql.SQLException: No suitable driver found for ...
>
> And I have tried to add jdbc jar to spark_path with both commands but
> failed:
>
> - spark-shell --jars mysql-connector-java-5.0.8-bin.jar
> - SPARK_CLASSPATH=mysql-connector-java-5.0.8-bin.jar spark-shell
>
> My Spark version is 1.3.0 while
> `Class.forName("com.mysql.jdbc.Driver").newInstance` is worked.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-tp22178.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: How to use DataFrame with MySQL
Posted by matthewrj <ml...@gmail.com>.
I just ran into this too. Thanks for the tip!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-tp22178p23351.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to use DataFrame with MySQL
Posted by Rishi Yadav <ri...@infoobjects.com>.
for me, it's only working if I set --driver-class-path to mysql library.
On Sun, Mar 22, 2015 at 11:29 PM, gavin zhang <ga...@gmail.com> wrote:
> OK，I found what the problem is: It couldn't work with
> mysql-connector-5.0.8.
> I updated the connector version to 5.1.34 and it worked.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-tp22178p22182.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: How to use DataFrame with MySQL
Posted by gavin zhang <ga...@gmail.com>.
OK，I found what the problem is: It couldn't work with mysql-connector-5.0.8.
I updated the connector version to 5.1.34 and it worked.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-tp22178p22182.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org