You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/02/15 21:07:01 UTC
[GitHub] [spark] sarutak opened a new pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
sarutak opened a new pull request #31291:
URL: https://github.com/apache/spark/pull/31291
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR fixes the test code build error for `docker-integration-tests` in `branch-3.0`.
Currently, building test code for docker-integration-tests in `branch-3.0` seems to fail due to `ojdbc6:11.2.0.1.0` and `jdb2jcc4:10.5.0.5` are not available.
The solution backports SPARK-32049 and partially SPARK-31272.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
To fix the build error.
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
No.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
This change affects `OracleIntegrationSuite` so I confirmed it passed.
This change also affects `DB2IntegrationSuite` but it's ignored for now so I didn't care for it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak closed pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak closed pull request #31291:
URL: https://github.com/apache/spark/pull/31291
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765328418
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38944/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766515934
Thank you for closing this PR, @sarutak . Sorry for asking that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766513597
O.K, I leave it as it is for now. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766512802
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779549039
@dongjoon-hyun No problem. Thank you for revisiting this PR!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779439499
Do you know why this PR has only 32 succeeded? In Maven, I'm seeing 33 succeeded.
```
[info] Run completed in 50 seconds, 23 milliseconds.
[info] Total number of tests run: 32
[info] Suites: completed 5, aborted 0
[info] Tests: succeeded 32, failed 0, canceled 0, ignored 6, pending 0
[info] All tests passed.
[info] Passed: Total 32, Failed 0, Errors 0, Passed 32, Ignored 6
[success] Total time: 313 s, completed Feb 15, 2021 12:42:02 PM
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766512802
I guess this can be one of SBT dependency management issue. Since Maven is our default one, I believe we can keep `branch-3.0` as-is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765256993
**[Test build #134358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134358/testReport)** for PR 31291 at commit [`58c8d6f`](https://github.com/apache/spark/commit/58c8d6fcb55b5490970ec16c3d6523088cb5106f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765355230
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134358/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766512271
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-766512271
@dongjoon-hyun It's strange. As you tried, I can't see this issue with Maven.
But if I build with sbt, I still get the following error.
```
build/sbt -Phive -Phive-thriftserver -Pdocker-integration-tests test:compile
...
[error] (docker-integration-tests/*:update) sbt.ResolveException: unresolved dependency: com.oracle#ojdbc6;11.2.0.1.0: not found
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765305892
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38944/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779659509
Thanks, @sarutak !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765256993
**[Test build #134358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134358/testReport)** for PR 31291 at commit [`58c8d6f`](https://github.com/apache/spark/commit/58c8d6fcb55b5490970ec16c3d6523088cb5106f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak closed pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
sarutak closed pull request #31291:
URL: https://github.com/apache/spark/pull/31291
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765355230
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134358/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765293554
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38944/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765340740
**[Test build #134358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134358/testReport)** for PR 31291 at commit [`58c8d6f`](https://github.com/apache/spark/commit/58c8d6fcb55b5490970ec16c3d6523088cb5106f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779439499
Do you know why this PR has only `32 succeeded`? In Maven, I'm seeing `33 succeeded`.
```
[info] Run completed in 50 seconds, 23 milliseconds.
[info] Total number of tests run: 32
[info] Suites: completed 5, aborted 0
[info] Tests: succeeded 32, failed 0, canceled 0, ignored 6, pending 0
[info] All tests passed.
[info] Passed: Total 32, Failed 0, Errors 0, Passed 32, Ignored 6
[success] Total time: 313 s, completed Feb 15, 2021 12:42:02 PM
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #31291:
URL: https://github.com/apache/spark/pull/31291
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779447452
Well. It turns out that this PR was based on the old branch status. When I use this PR exactly, the result is same for sbt/maven.
```
Run completed in 47 seconds, 417 milliseconds.
Total number of tests run: 32
Suites: completed 6, aborted 0
Tests: succeeded 32, failed 0, canceled 0, ignored 6, pending 0
All tests passed.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31291: [SPARK-34201][SQL][TESTS][BUILD][3.0] Fix the test code build error for docker-integration-tests in branch-3.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-765328418
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38944/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31291: [SPARK-34201][SQL][TESTS][3.0] Fix the test code build error for docker-integration-tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31291:
URL: https://github.com/apache/spark/pull/31291#issuecomment-779435659
Hi, @sarutak .
Since yesterday, I've been reconsidering your PR as a release manager of Apache Spark 3.0.2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org