[jira] [Created] (SPARK-23689) Spark 2.3.0/2.2.1 Some changes cause org.apache.spark.sql.catalyst.errors.package$TreeNodeException:

["Yuandong Song (JIRA)" <ji...@apache.org>]

Yuandong Song created SPARK-23689:
-------------------------------------

             Summary: Spark 2.3.0/2.2.1 Some changes cause org.apache.spark.sql.catalyst.errors.package$TreeNodeException:
                 Key: SPARK-23689
                 URL: https://issues.apache.org/jira/browse/SPARK-23689
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.0, 2.2.1
         Environment: My spark standalone cluster has two workers (MEM 250G) and one master.

Mysql's version is 5.7.
            Reporter: Yuandong Song


{code:java}
    String[] str = {"supplier", "nation", "partsupp", "part","lineitem"};


    for (int i = 0; i < str.length; i++) {
        System.out.println(i);
        Dataset<Row> t1 = ss.read().format("jdbc")
                .option("useSSL", false)
                .option("driver", "com.mysql.jdbc.Driver")
                .option("url", "jdbc:mysql://172.16.50.104:19100/test_tpch_1g")
                .option("user", "dbscale")
                .option("password", "abc123")
                .option("dbtable", str[i])
                .load();
        t1.createOrReplaceTempView(str[i]);
        t1.show();
    }

    Properties connProp = new Properties();
    connProp.put("driver", "com.mysql.jdbc.Driver");
    connProp.put("useSSL", "false");
    connProp.put("user", "dbscale");
    connProp.put("password", "abc123");

    String sqlstr = "SELECT ps_suppkey FROM partsupp WHERE ps_partkey IN " +
            "( SELECT p_partkey FROM part WHERE p_name LIKE 'dark%' ) AND ps_availqty > " +
            "( SELECT 0.5 * SUM(l_quantity) FROM lineitem WHERE l_partkey = partsupp.ps_partkey AND l_suppkey = partsupp.ps_suppkey AND l_shipdate >= '1993-01-01' AND l_shipdate < '1994-01-01' ) ";
    ss.sql(sqlstr).show();{code}
I am using mysql as datasourse to make some tpch tests on spark.

These codes runs successfully on Spark 2.2.0.

But on Spark 2.2.1 and Spark 2.3.0, 
 [there is a exception|https://i.stack.imgur.com/zoRoo.png]

I guess SPARK-22472 causes it.

[http://spark.apache.org/releases/spark-release-2-2-1.html]

How can I resolve it in Spark 2.3.0 ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org