You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/10 00:50:30 UTC
[GitHub] [spark] gemelen opened a new pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
gemelen opened a new pull request #29995:
URL: https://github.com/apache/spark/pull/29995
### What changes were proposed in this pull request?
Current solution in build file to enable build failure on compilation warnings with exclusion of deprecation ones is not portable after SBT version 1.3.13 (build import fails with compilation error with SBT 1.4) and could be replaced with more robust and maintainable, especially since Scala 2.13.2 with similar built-in functionality.
### Why are the changes needed?
Unblocks upgrade to SBT after 1.3.13.
Enhances build file maintainability.
Allows fine tune of warnings configuration in scope of Scala 2.13 compilation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
`build/sbt`'s `compile` and `Test/compile` for both Scala 2.12 and 2.13 profiles.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709378639
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34445/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709448324
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #29995:
URL: https://github.com/apache/spark/pull/29995
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708857495
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708946598
**[Test build #129787 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129787/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708711871
**[Test build #129774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129774/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707386217
**[Test build #129703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129703/testReport)** for PR 29995 at commit [`c8d2594`](https://github.com/apache/spark/commit/c8d25948b38115e3545b31f4b908e9c43e1ef840).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708848159
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34380/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707387038
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709904229
@srowen thanks a lot for your efforts to pass this changeset
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708711871
**[Test build #129774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129774/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707246872
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709452009
**[Test build #129846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129846/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706754266
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502935843
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
OK I'm inclined to believe this assertion is just an error. It was probably trying to test the element type, but it isn't. I think it could be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706633672
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709442647
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706739538
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707255972
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706459866
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708841739
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34380/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709489375
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706464499
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34211/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708855119
Jenkins test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708700375
**[Test build #129758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129758/testReport)** for PR 29995 at commit [`d4415a3`](https://github.com/apache/spark/commit/d4415a36d685d78b8a67ffef2e7bb2335c9d0e3a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708601721
**[Test build #129758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129758/testReport)** for PR 29995 at commit [`d4415a3`](https://github.com/apache/spark/commit/d4415a36d685d78b8a67ffef2e7bb2335c9d0e3a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709335405
**[Test build #129840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129840/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701538
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709548689
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708890548
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34394/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706637875
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129633/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701164
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34364/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706731545
**[Test build #129642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129642/testReport)** for PR 29995 at commit [`2b42a30`](https://github.com/apache/spark/commit/2b42a3083670cd835988abb209b2c1b5bb784c55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707323256
**[Test build #129703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129703/testReport)** for PR 29995 at commit [`c8d2594`](https://github.com/apache/spark/commit/c8d25948b38115e3545b31f4b908e9c43e1ef840).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708692945
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34364/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708895659
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502934176
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
```
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
```
is in fact
```
val serializer: ExpressionEncoder.Serializer[Array[T]] = ExpressionEncoder[Array[T]].createSerializer()
val applied: InternalRow = serializer.apply(array)
val converted: ArrayData = applied.getArray(0)
```
where `getArray(0)` is a call to https://github.com/apache/spark/blob/5264164a67df498b73facae207eda12ee133be7d/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/SpecializedGetters.java#L56 which returns ArrayData, not an element of array from it.
I do not see that we are able to get an element of this "array" from ArrayData without specifying tables of methods to specific underlying array we want from it (like for Boolean we want ArrayData.toBooleanArray and so on).
https://github.com/apache/spark/blob/v3.0.1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayData.scala does not allow to inquiry type of a data it stores (cause it's a blob in memory at the end, I suppose?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709442647
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709367556
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34445/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502934176
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
```
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
```
is in fact
```
val serializer: ExpressionEncoder.Serializer[Array[T]] = ExpressionEncoder[Array[T]].createSerializer()
val applied: InternalRow = serializer.apply(array)
val converted: ArrayData = applied.getArray(0)
```
where `getArray(0)` is a call to https://github.com/apache/spark/blob/5264164a67df498b73facae207eda12ee133be7d/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/SpecializedGetters.java#L56 which returns ArrayData, not an element of array from it.
I do not see that we are able to get element of this "array" from ArrayData without specifying tables of methods to specific underlying array we want from it (like for Boolean we want ArrayData.toBooleanArray and so on).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707349852
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34309/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708895650
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34394/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707323256
**[Test build #129703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129703/testReport)** for PR 29995 at commit [`c8d2594`](https://github.com/apache/spark/commit/c8d25948b38115e3545b31f4b908e9c43e1ef840).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709553932
Merged to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701179
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708412698
Jenkins test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708856557
**[Test build #129774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129774/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706754002
**[Test build #129642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129642/testReport)** for PR 29995 at commit [`2b42a30`](https://github.com/apache/spark/commit/2b42a3083670cd835988abb209b2c1b5bb784c55).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r504024074
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##########
@@ -115,18 +117,24 @@ class InMemoryTable(
case (micros: Long, TimestampType) =>
val localDate = DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
ChronoUnit.MONTHS.between(EPOCH_LOCAL_DATE, localDate)
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
}
case DaysTransform(ref) =>
extractor(ref.fieldNames, schema, row) match {
case (days, DateType) =>
days
case (micros: Long, TimestampType) =>
ChronoUnit.DAYS.between(Instant.EPOCH, DateTimeUtils.microsToInstant(micros))
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
}
case HoursTransform(ref) =>
extractor(ref.fieldNames, schema, row) match {
case (micros: Long, TimestampType) =>
ChronoUnit.HOURS.between(Instant.EPOCH, DateTimeUtils.microsToInstant(micros))
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
Review comment:
Yes, pretty dumb error from my side, completely missed it (cause there was no test case for that code to be executed).
Thanks for noticing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706627871
**[Test build #129633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129633/testReport)** for PR 29995 at commit [`0bf0faa`](https://github.com/apache/spark/commit/0bf0faaa56c42dd4a788ba168c577f1f4fd6eb74).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708848170
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708848170
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706739528
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34246/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502934176
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
```
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
```
is in fact
```
val serializer: ExpressionEncoder.Serializer[Array[T]] = ExpressionEncoder[Array[T]].createSerializer()
val applied: InternalRow = serializer.apply(array)
val converted: ArrayData = applied.getArray(0)
```
where `getArray(0)` is a call to https://github.com/apache/spark/blob/5264164a67df498b73facae207eda12ee133be7d/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/SpecializedGetters.java#L56 which returns ArrayData, not an element of array from it.
I do not see that we are able to get element of this "array" from ArrayData without specifying tables of methods to specific underlying array we want from it (like for Boolean we want ArrayData.toBooleanArray and so on).
https://github.com/apache/spark/blob/v3.0.1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayData.scala do not allow to inquiry type of a data it stores (cause it's a blob in memory at the end, I suppose?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709548689
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706631597
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34237/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r503319660
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##########
@@ -114,18 +115,21 @@ class InMemoryTable(
case (micros: Long, TimestampType) =>
val localDate = DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
ChronoUnit.MONTHS.between(EPOCH_LOCAL_DATE, localDate)
+ case _ => throw new IllegalArgumentException("Match: unsupported argument(s) type")
Review comment:
In these cases, why not put the value of the thing that wasn't matched in the error?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502900514
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
I've looked more at this assert and it seems to me that it should never pass, cause `ArrayData` does not have "is-a" relation with `T` type (`ArrayData` even is not a wrapper to `Array[T]` which also has no "is-a" relation with `T` according to Scala types).
Previously, it caused warnings
```
[warn] /tmp/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala:77:34: abstract type T is unchecked since it is eliminated by erasure
[warn] /tmp/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala:77:11: abstract type T is unchecked since it is eliminated by erasure
```
but they were not enforced by custom warning processor.
@srowen @hvanhovell @dongjoon-hyun is my understanding correct here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709332908
Jenkins retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709335405
**[Test build #129840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129840/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707256000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708867347
**[Test build #129787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129787/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502934176
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
```
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
```
is in fact
```
val serializer: ExpressionEncoder.Serializer[Array[T]] = ExpressionEncoder[Array[T]].createSerializer()
val applied: InternalRow = serializer.apply(array)
val converted: ArrayData = applied.getArray(0)
```
there `getArray(0)` is a call to https://github.com/apache/spark/blob/5264164a67df498b73facae207eda12ee133be7d/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/SpecializedGetters.java#L56 which returns ArrayData, not an element of array from it.
I do not see that we are able to get element of this "array" from ArrayData without specifying tables of methods to specific underlying array we want from it (like for Boolean we want ArrayData.toBooleanArray and so on).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706458519
**[Test build #129608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129608/testReport)** for PR 29995 at commit [`ee5f641`](https://github.com/apache/spark/commit/ee5f641a4237de8c4c969d715d4680fdee82271f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706468220
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34211/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706754266
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708867347
**[Test build #129787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129787/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708895659
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502930141
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
Is it not getting the first element of the array and asserting that it is a T? or is it getting the array at position 0?
I agree, T can't be an Array[T].
Is it possible to, in any event, change this to assert the type of array[0]?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502936611
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
I believe it slipped into code due to similarity and wasn't caught because of erasure (`converted.isInstanceOf[T]` was always true cause it checked for `Object` as `T`).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706458519
**[Test build #129608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129608/testReport)** for PR 29995 at commit [`ee5f641`](https://github.com/apache/spark/commit/ee5f641a4237de8c4c969d715d4680fdee82271f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707349880
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707342798
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34309/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709447762
Seems to be unlucky week to run tests that were green previously :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706637871
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r503410206
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##########
@@ -114,18 +115,21 @@ class InMemoryTable(
case (micros: Long, TimestampType) =>
val localDate = DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
ChronoUnit.MONTHS.between(EPOCH_LOCAL_DATE, localDate)
+ case _ => throw new IllegalArgumentException("Match: unsupported argument(s) type")
Review comment:
Yeah, that's right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701545
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129758/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709489351
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34452/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709489375
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706731545
**[Test build #129642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129642/testReport)** for PR 29995 at commit [`2b42a30`](https://github.com/apache/spark/commit/2b42a3083670cd835988abb209b2c1b5bb784c55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706459870
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129608/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706633669
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34237/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707228914
**[Test build #129700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)** for PR 29995 at commit [`20d4c3e`](https://github.com/apache/spark/commit/20d4c3e899edd46b7d25bcfdc2bc444a395f9b89).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709547703
**[Test build #129846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129846/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297786
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297786
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502900514
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
I've looked more at this assert and it seems to me that it should never pass, cause `ArrayData` does not have "is-a" relation with `T` type (`ArrayData` even is not a wrapper to `Array[T]` which also has no "is-a" relation with `T` according to Scala types).
Previously, it caused warnings
```
[warn] /tmp/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala:77:34: abstract type T is unchecked since it is eliminated by erasure
[warn] /tmp/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala:77:11: abstract type T is unchecked since it is eliminated by erasure
```
but they were not enforced by custom warning processor.
@srowen @hvanhovell @dongjoon-hyun is my understanding is correct here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708857495
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706739538
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709442658
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129840/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708948813
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706637871
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708601721
**[Test build #129758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129758/testReport)** for PR 29995 at commit [`d4415a3`](https://github.com/apache/spark/commit/d4415a36d685d78b8a67ffef2e7bb2335c9d0e3a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701179
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707228914
**[Test build #129700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)** for PR 29995 at commit [`20d4c3e`](https://github.com/apache/spark/commit/20d4c3e899edd46b7d25bcfdc2bc444a395f9b89).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502727803
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##########
@@ -106,6 +106,18 @@ class InMemoryTable(
case (micros: Long, TimestampType) =>
val localDate = DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
ChronoUnit.YEARS.between(EPOCH_LOCAL_DATE, localDate)
+ case _ => throw new Exception("""match may not be exhaustive.
Review comment:
I'd make this more meaningful, like an IllegalArgumentException referring to the thing that does not match. Something simple.
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
##########
@@ -705,7 +705,7 @@ class StreamingQuerySuite extends StreamTest with BeforeAndAfter with Logging wi
val q2 = startQuery(input(1).toDS.map { i =>
// Emulate that `StreamingQuery` get captured with normal usage unintentionally.
// It should not fail the query.
- q1
+ val q = q1
Review comment:
This warning has always bugged me. I'm not sure if this change subverts the test itself, but, not sure this is any better or worse at accidentally capturing q1, so I'm OK with it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707256000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709378666
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709441737
**[Test build #129840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129840/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297095
**[Test build #129700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)** for PR 29995 at commit [`20d4c3e`](https://github.com/apache/spark/commit/20d4c3e899edd46b7d25bcfdc2bc444a395f9b89).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709475863
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34452/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706468229
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706459866
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706637763
**[Test build #129633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129633/testReport)** for PR 29995 at commit [`0bf0faa`](https://github.com/apache/spark/commit/0bf0faaa56c42dd4a788ba168c577f1f4fd6eb74).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on a change in pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r502934176
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
##########
@@ -72,9 +72,9 @@ class UnsafeArraySuite extends SparkFunSuite {
arrayData
}
- private def toUnsafeArray[T : TypeTag](array: Array[T]): ArrayData = {
+ private def toUnsafeArray[T: Manifest](array: Array[T]): ArrayData = {
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
- assert(converted.isInstanceOf[T])
+ assert(manifest[T].runtimeClass.isInstance(converted))
Review comment:
```
val converted = ExpressionEncoder[Array[T]].createSerializer().apply(array).getArray(0)
```
is in fact
```
val serializer: ExpressionEncoder.Serializer[Array[T]] = ExpressionEncoder[Array[T]].createSerializer()
val applied: InternalRow = serializer.apply(array)
val converted: ArrayData = applied.getArray(0)
```
where `getArray(0)` is a call to https://github.com/apache/spark/blob/5264164a67df498b73facae207eda12ee133be7d/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/SpecializedGetters.java#L56 which returns ArrayData, not an element of array from it.
I do not see that we are able to get element of this "array" from ArrayData without specifying tables of methods to specific underlying array we want from it (like for Boolean we want ArrayData.toBooleanArray and so on).
https://github.com/apache/spark/blob/fa0d4bf69929c5acd676d602e758a969713d19d8/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayData.scala do not allow to inquiry type of a data it stores (cause it's a blob in memory at the end, I suppose?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r503993200
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##########
@@ -115,18 +117,24 @@ class InMemoryTable(
case (micros: Long, TimestampType) =>
val localDate = DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
ChronoUnit.MONTHS.between(EPOCH_LOCAL_DATE, localDate)
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
}
case DaysTransform(ref) =>
extractor(ref.fieldNames, schema, row) match {
case (days, DateType) =>
days
case (micros: Long, TimestampType) =>
ChronoUnit.DAYS.between(Instant.EPOCH, DateTimeUtils.microsToInstant(micros))
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
}
case HoursTransform(ref) =>
extractor(ref.fieldNames, schema, row) match {
case (micros: Long, TimestampType) =>
ChronoUnit.HOURS.between(Instant.EPOCH, DateTimeUtils.microsToInstant(micros))
+ case (v, t) =>
+ throw new IllegalArgumentException("Match: unsupported argument(s) type - ($v, $t)")
Review comment:
Heh, sorry one more thing - you need string interpolation to make this work
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706737241
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34246/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706459861
**[Test build #129608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129608/testReport)** for PR 29995 at commit [`ee5f641`](https://github.com/apache/spark/commit/ee5f641a4237de8c4c969d715d4680fdee82271f).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708948824
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129787/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gemelen commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
gemelen commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706946825
cc @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706468229
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707349880
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708948813
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709452009
**[Test build #129846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129846/testReport)** for PR 29995 at commit [`a9582e1`](https://github.com/apache/spark/commit/a9582e11f3ac5fba1dd02a55b3bdd7aa5ef25e34).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-709378666
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706627871
**[Test build #129633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129633/testReport)** for PR 29995 at commit [`0bf0faa`](https://github.com/apache/spark/commit/0bf0faaa56c42dd4a788ba168c577f1f4fd6eb74).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708848175
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34380/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708701538
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-708895664
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34394/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707387038
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29995: [WIP][SPARK-33080][BUILD] Replace fatal warnings snippet
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-706633672
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org