You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/19 15:01:11 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
HyukjinKwon opened a new pull request #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960
## What changes were proposed in this pull request?
This PR backports https://github.com/apache/spark/pull/23248 which seems mistakenly not backported.
This is a regression introduced by https://github.com/apache/spark/pull/22104 at Spark 2.4.0.
When we have Python UDF in subquery, we will hit an exception
```
Caused by: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.AttributeReference cannot be cast to org.apache.spark.sql.catalyst.expressions.PythonUDF
at scala.collection.immutable.Stream.map(Stream.scala:414)
at org.apache.spark.sql.execution.python.EvalPythonExec.$anonfun$doExecute$2(EvalPythonExec.scala:98)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:815)
...
```
https://github.com/apache/spark/pull/22104 turned `ExtractPythonUDFs` from a physical rule to optimizer rule. However, there is a difference between a physical rule and optimizer rule. A physical rule always runs once, an optimizer rule may be applied twice on a query tree even the rule is located in a batch that only runs once.
For a subquery, the `OptimizeSubqueries` rule will execute the entire optimizer on the query plan inside subquery. Later on subquery will be turned to joins, and the optimizer rules will be applied to it again.
Unfortunately, the `ExtractPythonUDFs` rule is not idempotent. When it's applied twice on a query plan inside subquery, it will produce a malformed plan. It extracts Python UDF from Python exec plans.
This PR proposes 2 changes to be double safe:
1. `ExtractPythonUDFs` should skip python exec plans, to make the rule idempotent
2. `ExtractPythonUDFs` should skip subquery
## How was this patch tested?
a new test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601231141
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507520
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507898
Thank you, @HyukjinKwon and @cloud-fan .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601230565
cc @cloud-fan and @tgravescs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601471681
**[Test build #120075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120075/testReport)** for PR 27960 at commit [`423644e`](https://github.com/apache/spark/commit/423644eb63109d3e39c0c0c33c996e6cfda542a4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tgravescs commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
tgravescs commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601256846
thanks @HyukjinKwon looks like clean merge other then test change. LGTM pending jenkins
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507523
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120075/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507520
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4]
Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601328960
**[Test build #120060 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120060/testReport)** for PR 27960 at commit [`7a916ac`](https://github.com/apache/spark/commit/7a916aca9b7b434649862881d4754f845a3528d0).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class ArrowEvalPython(`
* `case class BatchEvalPython(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601472057
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24792/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601231159
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24777/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601329547
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601472047
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601231159
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24777/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4]
Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601230402
**[Test build #120060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120060/testReport)** for PR 27960 at commit [`7a916ac`](https://github.com/apache/spark/commit/7a916aca9b7b434649862881d4754f845a3528d0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601329561
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120060/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601232421
I do remember I backported it as I manually fixed some conflicts, but ...
Maybe some network problems happened but I didn't notice. Anyway thanks for doing it!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4]
Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507358
**[Test build #120075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120075/testReport)** for PR 27960 at commit [`423644e`](https://github.com/apache/spark/commit/423644eb63109d3e39c0c0c33c996e6cfda542a4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507597
Merged to branch-2.4.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601472047
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601231141
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601230402
**[Test build #120060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120060/testReport)** for PR 27960 at commit [`7a916ac`](https://github.com/apache/spark/commit/7a916aca9b7b434649862881d4754f845a3528d0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4]
Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601471681
**[Test build #120075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120075/testReport)** for PR 27960 at commit [`423644e`](https://github.com/apache/spark/commit/423644eb63109d3e39c0c0c33c996e6cfda542a4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601472057
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24792/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601329561
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120060/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601469622
Sure, will do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601507523
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120075/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601337022
The failure is relevant one, `test_udf_in_subquery`. Could you take a look, @HyukjinKwon ?
```
======================================================================
ERROR: test_udf_in_subquery (pyspark.sql.tests.SQLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py", line 3581, in test_udf_in_subquery
with self.tempView("v"):
AttributeError: 'SQLTests' object has no attribute 'tempView'
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27960:
[SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27960: [SPARK-26293][SQL][2.4] Cast exception when having python udf in subquery
URL: https://github.com/apache/spark/pull/27960#issuecomment-601329547
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org