You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2019/07/28 22:52:50 UTC
[spark] branch master updated:
[SPARK-28277][SQL][PYTHON][TESTS][FOLLOW-UP] Re-enable commented out test
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3c52787 [SPARK-28277][SQL][PYTHON][TESTS][FOLLOW-UP] Re-enable commented out test
3c52787 is described below
commit 3c5278748d6c2890a9867a920b060d2b47d04c1f
Author: Huaxin Gao <hu...@us.ibm.com>
AuthorDate: Sun Jul 28 15:52:31 2019 -0700
[SPARK-28277][SQL][PYTHON][TESTS][FOLLOW-UP] Re-enable commented out test
## What changes were proposed in this pull request?
Fix for ```SPARK-28441 (PythonUDF used in correlated scalar subquery causes UnsupportedOperationException)``` is in. Re-enable the commented out test for ```udf(max(udf(column))) ```
## How was this patch tested?
use existing test ```udf-except.sql```
Closes #25278 from huaxingao/spark-28277n.
Authored-by: Huaxin Gao <hu...@us.ibm.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../resources/sql-tests/inputs/udf/udf-except.sql | 23 +++++++++++-----------
.../sql-tests/results/udf/udf-except.sql.out | 20 ++++++++++++++++++-
2 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/sql/core/src/test/resources/sql-tests/inputs/udf/udf-except.sql b/sql/core/src/test/resources/sql-tests/inputs/udf/udf-except.sql
index 17e2488..378b5bb 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/udf/udf-except.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/udf/udf-except.sql
@@ -45,15 +45,14 @@ SELECT (SELECT udf(min(k)) FROM t2) abs_min_t2 FROM t1 WHERE t1.k = udf('one');
-- Except operation that will be replaced by left anti join
---- [SPARK-28441] udf(max(udf(column))) throws java.lang.UnsupportedOperationException: Cannot evaluate expression: udf(null)
---- SELECT t1.k
---- FROM t1
---- WHERE t1.v <= (SELECT udf(max(udf(t2.v)))
---- FROM t2
---- WHERE udf(t2.k) = udf(t1.k))
---- MINUS
---- SELECT t1.k
---- FROM t1
---- WHERE udf(t1.v) >= (SELECT min(udf(t2.v))
---- FROM t2
---- WHERE t2.k = t1.k);
+SELECT t1.k
+FROM t1
+WHERE t1.v <= (SELECT udf(max(udf(t2.v)))
+ FROM t2
+ WHERE udf(t2.k) = udf(t1.k))
+MINUS
+SELECT t1.k
+FROM t1
+WHERE udf(t1.v) >= (SELECT min(udf(t2.v))
+ FROM t2
+ WHERE t2.k = t1.k);
diff --git a/sql/core/src/test/resources/sql-tests/results/udf/udf-except.sql.out b/sql/core/src/test/resources/sql-tests/results/udf/udf-except.sql.out
index cb8a4e8..0badaf0 100644
--- a/sql/core/src/test/resources/sql-tests/results/udf/udf-except.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/udf/udf-except.sql.out
@@ -1,5 +1,5 @@
-- Automatically generated by SQLQueryTestSuite
--- Number of queries: 8
+-- Number of queries: 9
-- !query 0
@@ -85,3 +85,21 @@ struct<min_t2:string>
-- !query 7 output
NULL
two
+
+
+-- !query 8
+SELECT t1.k
+FROM t1
+WHERE t1.v <= (SELECT udf(max(udf(t2.v)))
+ FROM t2
+ WHERE udf(t2.k) = udf(t1.k))
+MINUS
+SELECT t1.k
+FROM t1
+WHERE udf(t1.v) >= (SELECT min(udf(t2.v))
+ FROM t2
+ WHERE t2.k = t1.k)
+-- !query 8 schema
+struct<k:string>
+-- !query 8 output
+two
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org