You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/20 05:58:29 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
dongjoon-hyun opened a new pull request #34048:
URL: https://github.com/apache/spark/pull/34048
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922683531
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47957/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922686013
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143450/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923337971
Merged to 3.2/3.1 too. This is not a blocker.
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922732723
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923145944
For a record,
- SparkR image building is broken in Jenkins and all other environments.
- After we recover SparkR, Jenkins machine will still fail at SparkR test case because it has only R 3.x currently.
On the system with R 4.0+, `SparkR` test case is recovered correctly like the following.
```
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with no resources & statefulset allocation
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local hostpath storage on statefulsets
- PVs with local hostpath and storageClass on statefulsets
- PVs with local storage
- Launcher client dependencies
- SPARK-33615: Launcher client archives
- SPARK-33748: Launcher python client respecting PYSPARK_PYTHON
- SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python
- Launcher python client dependencies using a zip file
- Test basic decommissioning
- Test basic decommissioning with shuffle cleanup
- Test decommissioning with dynamic allocation & shuffle cleanups *** FAILED ***
The code passed to eventually never returned normally. Attempted 212 times over 4.005780755083333 minutes. Last failure
- Test decommissioning timeouts
- Run SparkR on simple dataframe.R example
Run completed in 18 minutes, 23 seconds.
Total number of tests run: 30
Suites: completed 2, aborted 0
Tests: succeeded 29, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
```
Please note that `Test decommissioning` failure is irrelevant one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922676789
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922660473
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143447/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922681809
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923337971
Merged to branch-3.2 too. This is not a blocker.
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923337971
Merged to branch-3.2 too
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922666058
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143449/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922675397
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47955/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923146526
Thank you, @HyukjinKwon . Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922678155
**[Test build #143450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143450/testReport)** for PR 34048 at commit [`f151234`](https://github.com/apache/spark/commit/f15123479bed6c17e3f80c4e38c3d3afe4a7ff74).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922698713
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922683531
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47957/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923145944
For a record,
- SparkR image building is broken in Jenkins and all other environments.
- After we recover SparkR, Jenkins machine will still fail at SparkR test case because it has only R 3.x currently.
On the system with R 4.0+, `SparkR` test case is recovered correctly like the following.
```
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with no resources & statefulset allocation
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local hostpath storage on statefulsets
- PVs with local hostpath and storageClass on statefulsets
- PVs with local storage
- Launcher client dependencies
- SPARK-33615: Launcher client archives
- SPARK-33748: Launcher python client respecting PYSPARK_PYTHON
- SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python
- Launcher python client dependencies using a zip file
- Test basic decommissioning
- Test basic decommissioning with shuffle cleanup
- Test decommissioning with dynamic allocation & shuffle cleanups *** FAILED ***
The code passed to eventually never returned normally. Attempted 212 times over 4.005780755083333 minutes. Last failure
- Test decommissioning timeouts
- Run SparkR on simple dataframe.R example
Run completed in 18 minutes, 23 seconds.
Total number of tests run: 30
Suites: completed 2, aborted 0
Tests: succeeded 29, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922677521
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47957/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922654052
**[Test build #143447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143447/testReport)** for PR 34048 at commit [`2137c18`](https://github.com/apache/spark/commit/2137c184ec8cc7adb71d7d2562dd2b02f571a656).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923601955
`branch-3.1` is recovered.
```
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local storage
- Launcher client dependencies
- SPARK-33615: Launcher client archives
- SPARK-33748: Launcher python client respecting PYSPARK_PYTHON
- SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python
- Launcher python client dependencies using a zip file
- Test basic decommissioning
- Test basic decommissioning with shuffle cleanup
- Test decommissioning with dynamic allocation & shuffle cleanups
- Run SparkR on simple dataframe.R example
Run completed in 14 minutes, 49 seconds.
Total number of tests run: 26
Suites: completed 2, aborted 0
Tests: succeeded 26, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922666058
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143449/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922681760
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922715135
Hi, @HyukjinKwon .
Could you review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922660473
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143447/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922685789
**[Test build #143450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143450/testReport)** for PR 34048 at commit [`f151234`](https://github.com/apache/spark/commit/f15123479bed6c17e3f80c4e38c3d3afe4a7ff74).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922661084
**[Test build #143449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143449/testReport)** for PR 34048 at commit [`4a9ba52`](https://github.com/apache/spark/commit/4a9ba52ffe50a8f5b6729d6a92eb99653d51b38f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922681809
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922665916
**[Test build #143449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143449/testReport)** for PR 34048 at commit [`4a9ba52`](https://github.com/apache/spark/commit/4a9ba52ffe50a8f5b6729d6a92eb99653d51b38f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #34048:
URL: https://github.com/apache/spark/pull/34048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922683354
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47957/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922661084
**[Test build #143449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143449/testReport)** for PR 34048 at commit [`4a9ba52`](https://github.com/apache/spark/commit/4a9ba52ffe50a8f5b6729d6a92eb99653d51b38f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922675985
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47955/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922660330
**[Test build #143447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143447/testReport)** for PR 34048 at commit [`2137c18`](https://github.com/apache/spark/commit/2137c184ec8cc7adb71d7d2562dd2b02f571a656).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922743846
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923145944
For a record,
- SparkR image building is broken in Jenkins and all other environments.
- After we recover SparkR, Jenkins machine will still fail at SparkR test case because it has only R 3.x currently.
On the system with R 4.0+, `Run SparkR on simple dataframe.R example` test case is recovered correctly like the following.
```
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with no resources & statefulset allocation
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local hostpath storage on statefulsets
- PVs with local hostpath and storageClass on statefulsets
- PVs with local storage
- Launcher client dependencies
- SPARK-33615: Launcher client archives
- SPARK-33748: Launcher python client respecting PYSPARK_PYTHON
- SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python
- Launcher python client dependencies using a zip file
- Test basic decommissioning
- Test basic decommissioning with shuffle cleanup
- Test decommissioning with dynamic allocation & shuffle cleanups *** FAILED ***
The code passed to eventually never returned normally. Attempted 212 times over 4.005780755083333 minutes. Last failure
- Test decommissioning timeouts
- Run SparkR on simple dataframe.R example
Run completed in 18 minutes, 23 seconds.
Total number of tests run: 30
Suites: completed 2, aborted 0
Tests: succeeded 29, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
```
Please note that `Test decommissioning` failure is irrelevant one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922672338
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47955/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-923145944
For a record,
- SparkR image building is broken in Jenkins and all other environments.
- After we recover SparkR, Jenkins machine will still fail at SparkR test case because it has only R 3.x currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922686013
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143450/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922678155
**[Test build #143450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143450/testReport)** for PR 34048 at commit [`f151234`](https://github.com/apache/spark/commit/f15123479bed6c17e3f80c4e38c3d3afe4a7ff74).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922675985
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47955/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922743846
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34048: [SPARK-36806][K8S][R] Use R 4.0.4 in K8s R image
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34048:
URL: https://github.com/apache/spark/pull/34048#issuecomment-922654052
**[Test build #143447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143447/testReport)** for PR 34048 at commit [`2137c18`](https://github.com/apache/spark/commit/2137c184ec8cc7adb71d7d2562dd2b02f571a656).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org