You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:43:12 UTC
[jira] [Resolved] (SPARK-23689) Spark 2.3.0/2.2.1 Some changes
cause org.apache.spark.sql.catalyst.errors.package$TreeNodeException:
[ https://issues.apache.org/jira/browse/SPARK-23689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-23689.
----------------------------------
Resolution: Incomplete
> Spark 2.3.0/2.2.1 Some changes cause org.apache.spark.sql.catalyst.errors.package$TreeNodeException:
> ----------------------------------------------------------------------------------------------------
>
> Key: SPARK-23689
> URL: https://issues.apache.org/jira/browse/SPARK-23689
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.1, 2.3.0
> Environment: My spark standalone cluster has two workers (MEM 250G) and one master.
> Mysql's version is 5.7.
> Reporter: Yuandong Song
> Priority: Major
> Labels: bulk-closed
>
> {code:java}
> String[] str = {"supplier", "nation", "partsupp", "part","lineitem"};
> for (int i = 0; i < str.length; i++) {
> System.out.println(i);
> Dataset<Row> t1 = ss.read().format("jdbc")
> .option("useSSL", false)
> .option("driver", "com.mysql.jdbc.Driver")
> .option("url", "jdbc:mysql://172.16.50.104:19100/test_tpch_1g")
> .option("user", "dbscale")
> .option("password", "abc123")
> .option("dbtable", str[i])
> .load();
> t1.createOrReplaceTempView(str[i]);
> t1.show();
> }
> Properties connProp = new Properties();
> connProp.put("driver", "com.mysql.jdbc.Driver");
> connProp.put("useSSL", "false");
> connProp.put("user", "dbscale");
> connProp.put("password", "abc123");
> String sqlstr = "SELECT ps_suppkey FROM partsupp WHERE ps_partkey IN " +
> "( SELECT p_partkey FROM part WHERE p_name LIKE 'dark%' ) AND ps_availqty > " +
> "( SELECT 0.5 * SUM(l_quantity) FROM lineitem WHERE l_partkey = partsupp.ps_partkey AND l_suppkey = partsupp.ps_suppkey AND l_shipdate >= '1993-01-01' AND l_shipdate < '1994-01-01' ) ";
> ss.sql(sqlstr).show();{code}
> I am using mysql as datasourse to make some tpch tests on spark.
> These codes runs successfully on Spark 2.2.0.
> But on Spark 2.2.1 and Spark 2.3.0,
> [there is a exception|https://i.stack.imgur.com/zoRoo.png]
> I guess SPARK-22472 causes it.
> [http://spark.apache.org/releases/spark-release-2-2-1.html]
> How can I resolve it in Spark 2.3.0 ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org