You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2023/02/22 22:33:10 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request, #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn documentation with v1.2
dongjoon-hyun opened a new pull request, #40132:
URL: https://github.com/apache/spark/pull/40132
### What changes were proposed in this pull request?
This PR aims to update `YuniKorn` documentation with the latest v1.2.0 and fix codify issues in doc.
### Why are the changes needed?
- https://yunikorn.apache.org/release-announce/1.2.0
### Does this PR introduce _any_ user-facing change?
This is a documentation-only change.
### How was this patch tested?
```
$ build/sbt -Psparkr -Pkubernetes -Pkubernetes-integration-tests -Dspark.kubernetes.test.deployMode=docker-desktop "kubernetes-integration-tests/test" -Dtest.exclude.tags=minikube,local,decom -Dtest.default.exclude.tags=''
[info] KubernetesSuite:
[info] - SPARK-42190: Run SparkPi with local[*] (10 seconds, 832 milliseconds)
[info] - Run SparkPi with no resources (12 seconds, 421 milliseconds)
[info] - Run SparkPi with no resources & statefulset allocation (17 seconds, 861 milliseconds)
[info] - Run SparkPi with a very long application name. (12 seconds, 531 milliseconds)
[info] - Use SparkLauncher.NO_RESOURCE (17 seconds, 697 milliseconds)
[info] - Run SparkPi with a master URL without a scheme. (12 seconds, 499 milliseconds)
[info] - Run SparkPi with an argument. (18 seconds, 734 milliseconds)
[info] - Run SparkPi with custom labels, annotations, and environment variables. (12 seconds, 520 milliseconds)
[info] - All pods have the same service account by default (17 seconds, 504 milliseconds)
[info] - Run extraJVMOptions check on driver (9 seconds, 402 milliseconds)
[info] - SPARK-42474: Run extraJVMOptions JVM GC option check - G1GC (9 seconds, 389 milliseconds)
[info] - SPARK-42474: Run extraJVMOptions JVM GC option check - Other GC (9 seconds, 330 milliseconds)
[info] - Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j2.properties (17 seconds, 710 milliseconds)
[info] - Run SparkPi with env and mount secrets. (19 seconds, 797 milliseconds)
[info] - Run PySpark on simple pi.py example (18 seconds, 568 milliseconds)
[info] - Run PySpark to test a pyfiles example (15 seconds, 622 milliseconds)
[info] - Run PySpark with memory customization (18 seconds, 507 milliseconds)
[info] - Run in client mode. (6 seconds, 185 milliseconds)
[info] - Start pod creation from template (17 seconds, 696 milliseconds)
[info] - SPARK-38398: Schedule pod creation from template (12 seconds, 585 milliseconds)
[info] - Run SparkR on simple dataframe.R example (19 seconds, 639 milliseconds)
[info] YuniKornSuite:
[info] - SPARK-42190: Run SparkPi with local[*] (12 seconds, 421 milliseconds)
[info] - Run SparkPi with no resources (20 seconds, 465 milliseconds)
[info] - Run SparkPi with no resources & statefulset allocation (15 seconds, 516 milliseconds)
[info] - Run SparkPi with a very long application name. (20 seconds, 532 milliseconds)
[info] - Use SparkLauncher.NO_RESOURCE (15 seconds, 545 milliseconds)
[info] - Run SparkPi with a master URL without a scheme. (20 seconds, 575 milliseconds)
[info] - Run SparkPi with an argument. (16 seconds, 462 milliseconds)
[info] - Run SparkPi with custom labels, annotations, and environment variables. (20 seconds, 568 milliseconds)
[info] - All pods have the same service account by default (15 seconds, 630 milliseconds)
[info] - Run extraJVMOptions check on driver (12 seconds, 483 milliseconds)
[info] - SPARK-42474: Run extraJVMOptions JVM GC option check - G1GC (12 seconds, 665 milliseconds)
[info] - SPARK-42474: Run extraJVMOptions JVM GC option check - Other GC (11 seconds, 615 milliseconds)
[info] - Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j2.properties (20 seconds, 810 milliseconds)
[info] - Run SparkPi with env and mount secrets. (24 seconds, 622 milliseconds)
[info] - Run PySpark on simple pi.py example (16 seconds, 650 milliseconds)
[info] - Run PySpark to test a pyfiles example (23 seconds, 662 milliseconds)
[info] - Run PySpark with memory customization (15 seconds, 450 milliseconds)
[info] - Run in client mode. (5 seconds, 121 milliseconds)
[info] - Start pod creation from template (20 seconds, 552 milliseconds)
[info] - SPARK-38398: Schedule pod creation from template (15 seconds, 847 milliseconds)
[info] - Run SparkR on simple dataframe.R example (22 seconds, 739 milliseconds)
[info] Run completed in 15 minutes, 41 seconds.
[info] Total number of tests run: 42
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 42, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 1306 s (21:46), completed Feb 22, 2023, 2:28:18 PM
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn docs with v1.2
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #40132:
URL: https://github.com/apache/spark/pull/40132#issuecomment-1440969626
Thank you so much, @viirya . Merged to master/3.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn docs with v1.2
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #40132:
URL: https://github.com/apache/spark/pull/40132#issuecomment-1440934308
Looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn docs with v1.2
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn docs with v1.2
URL: https://github.com/apache/spark/pull/40132
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #40132: [SPARK-42532][K8S][DOCS] Update YuniKorn documentation with v1.2
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #40132:
URL: https://github.com/apache/spark/pull/40132#issuecomment-1440918027
Could you review this too, @viirya ? I fixed some K8s documentation issues while reviewing and testing 3.4.0 RC1 and updated YuniKorn documentation with the latest information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org