You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/08/13 18:19:37 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
LuciferYang opened a new pull request, #37508:
URL: https://github.com/apache/spark/pull/37508
### What changes were proposed in this pull request?
This pr revert SPARK-40047 due to mvn test still need this dependency.
### Why are the changes needed?
mvn test still need `xalan` dependency although GA passed before this pr.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Manual test:
```
mvn clean install -DskipTests -pl core -am
build/mvn clean test -pl core -Dtest=noen -DwildcardSuites=org.apache.spark.ui.UISeleniumSuite
```
**Before**
```
UISeleniumSuite:
*** RUN ABORTED ***
java.lang.NoClassDefFoundError: org/apache/xml/utils/PrefixResolver
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getField0(Class.java:2975)
at java.lang.Class.getField(Class.java:1701)
at com.gargoylesoftware.htmlunit.svg.SvgElementFactory.<clinit>(SvgElementFactory.java:64)
at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.<clinit>(HtmlUnitNekoHtmlParser.java:77)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.<clinit>(DefaultPageCreator.java:93)
at com.gargoylesoftware.htmlunit.WebClient.<init>(WebClient.java:191)
at com.gargoylesoftware.htmlunit.WebClient.<init>(WebClient.java:273)
at com.gargoylesoftware.htmlunit.WebClient.<init>(WebClient.java:263)
...
Cause: java.lang.ClassNotFoundException: org.apache.xml.utils.PrefixResolver
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getField0(Class.java:2975)
at java.lang.Class.getField(Class.java:1701)
at com.gargoylesoftware.htmlunit.svg.SvgElementFactory.<clinit>(SvgElementFactory.java:64)
at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.<clinit>(HtmlUnitNekoHtmlParser.java:77)
...
```
**After**
```
UISeleniumSuite:
- all jobs page should be rendered even though we configure the scheduling mode to fair
- effects of unpersist() / persist() should be reflected
- failed stages should not appear to be active
- spark.ui.killEnabled should properly control kill button display
- jobs page should not display job group name unless some job was submitted in a job group
- job progress bars should handle stage / task failures
- job details page should display useful information for stages that haven't started
- job progress bars / cells reflect skipped stages / tasks
- stages that aren't run appear as 'skipped stages' after a job finishes
- jobs with stages that are skipped should show correct link descriptions on all jobs page
- attaching and detaching a new tab
- kill stage POST/GET response is correct
- kill job POST/GET response is correct
- stage & job retention
- live UI json application list
- job stages should have expected dotfile under DAG visualization
- stages page should show skipped stages
- Staleness of Spark UI should not last minutes or hours
- description for empty jobs
- Support disable event timeline
Run completed in 17 seconds, 986 milliseconds.
Total number of tests run: 20
Suites: completed 2, aborted 0
Tests: succeeded 20, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #37508:
URL: https://github.com/apache/spark/pull/37508#issuecomment-1214202605
GA passed before this pr due to dependencyTree still include `xalan`:
[test-tree.txt](https://github.com/apache/spark/files/9331980/test-tree.txt)
```
[info] +-net.sourceforge.htmlunit:htmlunit:2.62.0
[info] | +-com.shapesecurity:salvation2:3.0.0
[info] | +-commons-codec:commons-codec:1.15
[info] | +-commons-io:commons-io:2.10.0 (evicted by: 2.11.0)
[info] | +-commons-io:commons-io:2.11.0
[info] | +-commons-logging:commons-logging:1.2
[info] | +-commons-net:commons-net:3.8.0
[info] | +-net.sourceforge.htmlunit:htmlunit-core-js:2.62.0
[info] | +-net.sourceforge.htmlunit:htmlunit-cssparser:1.12.0
[info] | +-net.sourceforge.htmlunit:neko-htmlunit:2.62.0
[info] | | +-xerces:xercesImpl:2.12.0
[info] | | +-xml-apis:xml-apis:1.4.01
[info] | |
[info] | +-org.apache.commons:commons-lang3:3.12.0
[info] | +-org.apache.commons:commons-text:1.9
[info] | | +-org.apache.commons:commons-lang3:3.11 (evicted by: 3.12.0)
[info] | | +-org.apache.commons:commons-lang3:3.12.0
[info] | |
[info] | +-org.apache.httpcomponents:httpmime:4.5.13
[info] | | +-org.apache.httpcomponents:httpclient:4.5.13
[info] | | +-commons-codec:commons-codec:1.11 (evicted by: 1.15)
[info] | | +-commons-codec:commons-codec:1.15
[info] | | +-commons-logging:commons-logging:1.2
[info] | | +-org.apache.httpcomponents:httpcore:4.4.13 (evicted by: 4.4.14)
[info] | | +-org.apache.httpcomponents:httpcore:4.4.14
[info] | |
[info] | +-org.brotli:dec:0.1.2
[info] | +-org.eclipse.jetty.websocket:websocket-client:9.4.46.v20220331
[info] | | +-org.eclipse.jetty.websocket:websocket-common:9.4.46.v20220331
[info] | | | +-org.eclipse.jetty.websocket:websocket-api:9.4.46.v20220331
[info] | | | +-org.eclipse.jetty:jetty-io:9.4.46.v20220331 (evicted by: 9.4.48.v202..
[info] | | | +-org.eclipse.jetty:jetty-io:9.4.48.v20220622
[info] | | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | | |
[info] | | | +-org.eclipse.jetty:jetty-util:9.4.46.v20220331 (evicted by: 9.4.48.v2..
[info] | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | |
[info] | | +-org.eclipse.jetty:jetty-client:9.4.46.v20220331 (evicted by: 9.4.48.v2..
[info] | | +-org.eclipse.jetty:jetty-client:9.4.48.v20220622
[info] | | | +-org.eclipse.jetty:jetty-http:9.4.48.v20220622
[info] | | | | +-org.eclipse.jetty:jetty-io:9.4.48.v20220622
[info] | | | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | | | |
[info] | | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | | |
[info] | | | +-org.eclipse.jetty:jetty-io:9.4.48.v20220622
[info] | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | |
[info] | | +-org.eclipse.jetty:jetty-io:9.4.46.v20220331 (evicted by: 9.4.48.v20220..
[info] | | +-org.eclipse.jetty:jetty-io:9.4.48.v20220622
[info] | | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | | |
[info] | | +-org.eclipse.jetty:jetty-util:9.4.46.v20220331 (evicted by: 9.4.48.v202..
[info] | | +-org.eclipse.jetty:jetty-util:9.4.48.v20220622
[info] | |
[info] | +-xalan:xalan:2.7.2
[info] | +-xalan:serializer:2.7.2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #37508:
URL: https://github.com/apache/spark/pull/37508#issuecomment-1214203032
cc @dongjoon-hyun We may need to revert SPARK-40047 first. I found that mvn test also needs `xalan`.
I'm very sorry. I haven't done enough testing. also cc @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
URL: https://github.com/apache/spark/pull/37508
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #37508:
URL: https://github.com/apache/spark/pull/37508#issuecomment-1214204213
Oh, got it. I saw the dependency tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #37508: Revert "[SPARK-40047][TEST] Exclude unused `xalan` transitive dependency from `htmlunit`"
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #37508:
URL: https://github.com/apache/spark/pull/37508#issuecomment-1214204575
Yes, the exclude in `pom.xml` does not work for `sbt`. I need more checks in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org