You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Paul Wu (JIRA)" <ji...@apache.org> on 2017/07/28 18:09:02 UTC
[jira] [Comment Edited] (SPARK-17614) sparkSession.read()
.jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105402#comment-16105402 ]
Paul Wu edited comment on SPARK-17614 at 7/28/17 6:08 PM:
----------------------------------------------------------
The fix does not support the syntax on the syntax like this:
{{.jdbc(JDBC_URL, "(select * from emp)", connectionProperties);}}
Here is the stack trace:
{{Exception in thread "main" java.sql.SQLTransientException: com.datastax.driver.core.exceptions.SyntaxError: line 1:14 no viable alternative at input 'select' (SELECT * from [select]...)
at com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.<init>(CassandraPreparedStatement.java:108)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:60)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.<init>(JDBCRelation.scala:113)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:45)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:330)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125)
at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:166)
at com.att.cass.proto.CassJDBCWithSpark.main(CassJDBCWithSpark.java:44)}}
was (Author: zwu.net@gmail.com):
The fix does not support the syntax on the syntax like this:
{{.jdbc(JDBC_URL, "(select * from emp)", connectionProperties);}}
Here is the stack trace:
{{Exception in thread "main" java.sql.SQLTransientException: com.datastax.driver.core.exceptions.SyntaxError: line 1:14 no viable alternative at input 'select' (SELECT * from [select]...)
at com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.<init>(CassandraPreparedStatement.java:108)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:60)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.<init>(JDBCRelation.scala:113)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:45)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:330)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125)
at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:166)
at com.att.cass.proto.CassJDBCWithSpark.main(CassJDBCWithSpark.java:44)}}
> sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-17614
> URL: https://issues.apache.org/jira/browse/SPARK-17614
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.0.0
> Environment: Any Spark Runtime
> Reporter: Paul Wu
> Assignee: Sean Owen
> Priority: Minor
> Labels: cassandra-jdbc, sql
> Fix For: 2.1.0
>
>
> I have the code like the following with Cassandra JDBC (https://github.com/adejanovski/cassandra-jdbc-wrapper):
> final String dbTable= "sql_demo";
> Dataset<Row> jdbcDF
> = sparkSession.read()
> .jdbc(CASSANDRA_CONNECTION_URL, dbTable, connectionProperties);
> List<Row> rows = jdbcDF.collectAsList();
> It threw the error:
> Exception in thread "main" java.sql.SQLTransientException: com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...)
> at com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.<init>(CassandraPreparedStatement.java:108)
> at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
> at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
> at com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
> The reason is that the Spark jdbc code uses the sql syntax "where 1=0" somewhere (to get the schema?), but Cassandra does not support this syntax. Not sure how this issue can be resolved...this is because CQL is not standard sql.
> The following log shows more information:
> 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; Rack: %s
> 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM sql_demo WHERE 1=0
> 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] com.datastax.driver.core.Statement$1@41ccb3b9
> 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org