You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/09/21 20:00:19 UTC
[GitHub] [spark] sarutak opened a new pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
sarutak opened a new pull request #29827:
URL: https://github.com/apache/spark/pull/29827
### What changes were proposed in this pull request?
This PR adds a GitHub Actions job to run WebUI tests with Chrome.
`ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite` are added to test WebUI with better JavaScript support but they are disabled on the CI environment.
Now that we use GitHub Actions, let's add a job to run them.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
To ensure WebUI related changes can work properly and don't make regressions.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I confirmed the new job works on my repository.
I also confirmed we can download `unit-tests.log` if the WebUI tests fails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696500528
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492467982
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you pig-back something like this important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something like this important thing silently?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696500528
**[Test build #128959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128959/testReport)** for PR 29827 at commit [`cb6aef4`](https://github.com/apache/spark/commit/cb6aef49d44ed102db620ecd4c49f6ba194ab491).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492453481
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
@sarutak . Could you tell us why you share the cache key? This task seems to save different cache into this key and corrupt the existing Java 11 Maven full build.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
Thanks!
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you pig-back something like this important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you pig-back something like this important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something like this important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something important thing silently? :)
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
If the test suite is broken with the default Apache Spark distribution, we need to revert the root cause which broke the test suite. IIRC, this test suite is tested with the default configuration initially.
> Actually, I noticed the current ChromeDriver and RemoteWebDriver can't use guava 14.0.1
Do we have a JIRA issue for that test suite failure?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Thank you for the pointer, @sarutak . BTW, there are many request for Guava update. I believe we can handle this in the same group.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754416567
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38225/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696535996
**[Test build #128958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128958/testReport)** for PR 29827 at commit [`5206561`](https://github.com/apache/spark/commit/52065619764756af8c28593956c2a2f9f8118e36).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754444554
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38225/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696500796
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492467982
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you pig-back something like this important thing silently?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492469526
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I noticed the current ChromeDriver and RemoteWebDriver can't use `guava 14.0.1` so I intended to use `25.0` only for `ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492488827
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
I think ideally we should just include `ChromeUITest`tag in `core` above without changing the guava version ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696490482
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r493753499
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
What about just not adding this test? :)
How important is it? does this cover new ground?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-753709007
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696344007
**[Test build #128948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128948/testReport)** for PR 29827 at commit [`1c9af36`](https://github.com/apache/spark/commit/1c9af3669d051b7b94b7eb5252fcc25e2d56bfb7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-698677232
@dongjoon-hyun I agree.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492654755
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` and running GitHub Actions jobs on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert SPARK-31765 as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492456768
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
Ah, I don't intend to reuse it. I just forgot to change the name.
I've renamed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754389686
**[Test build #133637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133637/testReport)** for PR 29827 at commit [`cb6aef4`](https://github.com/apache/spark/commit/cb6aef49d44ed102db620ecd4c49f6ba194ab491).
* This patch **fails Spark unit tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492484703
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
> BTW, there are many request for Guava update.
Yeah, I know. So, should we revert that PR? It's a security fix but for the test code, or waiting for Guava updated?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754337920
**[Test build #133637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133637/testReport)** for PR 29827 at commit [`cb6aef4`](https://github.com/apache/spark/commit/cb6aef49d44ed102db620ecd4c49f6ba194ab491).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-698677232
@dongjoon-hyun I agree.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492467363
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696417333
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696536878
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696340949
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696500796
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-820819674
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696340949
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-697730993
@sarutak , @HyukjinKwon , @srowen .
If this is only test issue, I guess we can revisit this after we upgrade Apache Spark's Guava version successfully.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492453481
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
@sarutak . Could you tell us why you share the cache key? This task seems to save different cache into this key and corrupt the existing Java 11 Maven full build.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492469526
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I noticed the current ChromeDriver can't use `guava 14.0.1` so I intended to use `25.0` only for `ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492456768
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,50 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: java11-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ java11-maven-
Review comment:
Ah, I don't intend to reuse it. I just forgot to change the name.
I've renamed it.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I noticed the current ChromeDriver can't use `guava 14.0.1` so I intended to use `25.0` only for `ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Yes. I know the official version of Guava is `14.0.1`. Actually, I noticed the current ChromeDriver and RemoteWebDriver can't use `guava 14.0.1` so I intended to use `25.0` only for `ChromeUISeleniumSuite` and `ChromeUIHistoryServerSuite`.
Do you have any better idea rather than use `25.0`?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might to need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
> BTW, there are many request for Guava update.
Yeah, I know. So, should we revert that PR? It's a security fix but for the test code, or waiting for Guava updated?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
Yes. I tried to do so first but I noticed those tests need the newer Guava so I made a separate job...
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert that change as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert SPARK-31765 as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` and running GitHub Actions jobs on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert SPARK-31765 as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696344007
**[Test build #128948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128948/testReport)** for PR 29827 at commit [`1c9af36`](https://github.com/apache/spark/commit/1c9af3669d051b7b94b7eb5252fcc25e2d56bfb7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754444554
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38225/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r494703217
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
It can be one option but I'm wondering if we can make much better coverage for UI tests somehow.
I often see that UI tests are done by "manual" especially when the change is related to JavaScript.
`UISeleniumSuite`, `HistoryServerSuite` and other similar suites are enough for the static part of the UI like but not enough for the dynamic part which dynamically changes DOM or CSS by JavaScript.
Those existing suites use `HtmlUnitDriver` but its JavaScript support is not enough.
So I think it's better if we can run test with real browsers in the CI.
`ChromeDriver` is similar to `HtmlUnitDriver` but it uses the real Chrome so we can test JavaScript related stuffs.
`ChromeUISeleniumSute` and `ChromeUIHistoryServerSuite` use `ChromeDriver` rather than `HtmlUnitDriver` and all tests in the suites are what need JavaScript support so I think they cover new ground.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696417333
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696344007
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492654755
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert that change as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492654755
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492467982
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something important thing silently?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
@sarutak . Apache Spark 3.1's official Guava version is still `<guava.version>14.0.1</guava.version>`, isn't it?
Why do you piggy-back something important thing silently? :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492480542
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might to need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we might need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696340949
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696344007
**[Test build #128948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128948/testReport)** for PR 29827 at commit [`1c9af36`](https://github.com/apache/spark/commit/1c9af3669d051b7b94b7eb5252fcc25e2d56bfb7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696417333
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #29827:
URL: https://github.com/apache/spark/pull/29827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754397799
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133637/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492490262
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
Yes. I tried to do so first but I noticed those tests need the newer Guava so I made a separate job...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696535805
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696416526
**[Test build #128948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128948/testReport)** for PR 29827 at commit [`1c9af36`](https://github.com/apache/spark/commit/1c9af3669d051b7b94b7eb5252fcc25e2d56bfb7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-698075428
+1 for @dongjoon-hyun's...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696535805
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754397799
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133637/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r494703217
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
It can be one option but I'm wondering if we can make much better coverage for UI tests somehow.
I often see that UI tests are done by "manual" especially when the change is related to JavaScript.
`UISeleniumSuite`, `HistoryServerSuite` and other similar suites are enough for the static part of the UI like but not enough for the dynamic part which dynamically changes DOM or CSS by JavaScript.
Those existing suites use `HtmlUnitDriver` but its JavaScript support is not enough.
So I think it's better if we can run test with real browsers in the CI.
`ChromeDriver` is similar to `HtmlUnitDriver` but it uses the real Chrome so we can test JavaScript related stuffs.
`ChromeUISeleniumSute` and `ChromeUIHistoryServerSuite` use `ChromeDriver` rather than `HtmlUnitDriver` and all tests in the suites are what need JavaScript support so I think they cover new ground.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696344007
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696535289
**[Test build #128959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128959/testReport)** for PR 29827 at commit [`cb6aef4`](https://github.com/apache/spark/commit/cb6aef49d44ed102db620ecd4c49f6ba194ab491).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492654755
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` on my repository but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`. Or, revert SPARK-31765 as I mentioned [here](https://github.com/apache/spark/pull/29827#discussion_r492484703).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #29827:
URL: https://github.com/apache/spark/pull/29827
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492654755
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
@HyukjinKwon I've tried including `ChromeUITest` tag in `core` but as I expected, tests failed due to the Guava version compatibility.
```
[info] org.apache.spark.deploy.history.ChromeUIHistoryServerSuite *** ABORTED *** (7 milliseconds)
[info] java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
[info] at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
[info] at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
[info] at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
[info] at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
[info] at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
[info] at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157)
[info] at org.apache.spark.deploy.history.ChromeUIHistoryServerSuite.beforeAll(ChromeUIHistoryServerSuite.scala:38)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[error] Uncaught exception when running org.apache.spark.deploy.history.ChromeUIHistoryServerSuite: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
```
So, we need to specify `25.0-jre` only for tests tagged with `ChromeUITest`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696416526
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696340949
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-696417333
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492483254
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
Thank you for the pointer, @sarutak . BTW, there are many request for Guava update. I believe we can handle this in the same group.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492475749
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
If the test suite is broken with the default Apache Spark distribution, we need to revert the root cause which broke the test suite. IIRC, this test suite is tested with the default configuration initially.
> Actually, I noticed the current ChromeDriver and RemoteWebDriver can't use guava 14.0.1
Do we have a JIRA issue for that test suite failure?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492480542
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
+ export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_CLI_OPTS="--no-transfer-progress"
+ mkdir -p ~/.m2
+ ./build/mvn -Dspark.test.webdriver.chrome.driver=/usr/bin/chromedriver \
+ -Dguava.version=25.0-jre -Djava.version=11 -Dtest.default.exclude.tags= -Dtest=none \
Review comment:
The first time those tests are added, they works with `guava 14.0.1`.
I believe #28585 (SPARK-31765) requires upgraded RemoteWebDriver and it requires `25.0-jre`.
That PR is for a small security issue.
> Do we have a JIRA issue for that test suite failure?
Yes. But we need to modify the title.
https://issues.apache.org/jira/browse/SPARK-31996
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r492488827
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+ name: WebUI tests with chrome
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Spark repository
+ uses: actions/checkout@v2
+ - name: Cache Maven local repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: webui-tests-with-chrome-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ webui-tests-with-chrome-maven-
+ - name: Install Java 11
+ uses: actions/setup-java@v1
+ with:
+ java-version: 11
+ - name: Install Chrome and ChromeDriver
+ run: |
+ sudo apt update
+ sudo apt install google-chrome-stable
+ sudo apt install chromium-chromedriver
+ - name: Run WebUI tests with Maven
+ run: |
Review comment:
I think ideally we should just include `ChromeUITest`tag in `core` above without changing the guava version ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754337920
**[Test build #133637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133637/testReport)** for PR 29827 at commit [`cb6aef4`](https://github.com/apache/spark/commit/cb6aef49d44ed102db620ecd4c49f6ba194ab491).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-754425231
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38225/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org