You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/25 07:00:42 UTC
[GitHub] [spark] LuciferYang opened a new pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
LuciferYang opened a new pull request #34376:
URL: https://github.com/apache/spark/pull/34376
### What changes were proposed in this pull request?
The main change of this pr as follows:
- Add `extraJavaTestArgs` to `scalatest-maven-plugin` in `sql/hive/pom.xml`
- Add `--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED` to default java module options because some UTs failed due to
```
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private volatile int java.util.concurrent.atomic.AtomicBoolean.value accessible: module java.base does not "opens java.util.concurrent.atomic" to unnamed module @2744cc4a
```
### Why are the changes needed?
Pass UT with JDK 17
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Pass the Jenkins or GitHub Action
- Manual test use Java17 build + Java17 run (both openjdk version "17" 2021-09-14 LTS)
Add `extraJavaTestArgs` to `scalatest-maven-plugin` in `sql/hive/pom.xml` first and use Java 17 to run the following two commands:
```
build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phive -pl sql/hive -am -DskipTests
build/mvn test -Phadoop-3.2 -Phive-2.3 -Phive -pl sql/hive
```
**Before**
```
Run completed in 1 hour, 2 minutes, 3 seconds.
Total number of tests run: 3547
Suites: completed 117, aborted 0
Tests: succeeded 3525, failed 22, canceled 6, ignored 605, pending 0
*** 22 TESTS FAILED ***
```
**After**
```
Run completed in 1 hour, 4 minutes, 41 seconds.
Total number of tests run: 3547
Suites: completed 117, aborted 0
Tests: succeeded 3547, failed 0, canceled 6, ignored 605, pending 0
All tests passed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952504795
> From my side, it seems to fail due to [ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way.
Is this an x86 macos or an M1 macos? I'm sorry, I don't have an available x86 MacOS , so all tests were run on X86 Linux
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952587999
@srowen @dongjoon-hyun I tried 4 versions of JDK to test
```
mvn clean install -DskipTests -pl common/network-common -am
mvn test -pl common/network-common
```
on my M1:
- Oracle 17 x64 version: passed
- Oracle 17 arm64 version: passed
- zulu 17 x64 version: Can reproduce the problem @dongjoon-hyun said
- zulu 17 arm64 version: passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954412711
> The used environment was the following.
>
> $ brew list openssl
> /usr/local/Cellar/openssl@3/3.0.0_1/.bottle/etc/ (7 files)
But I install openssl@1.1
```
brew list openssl@1.1
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/.bottle/etc/ (7 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/bin/c_rehash
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/bin/openssl
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/include/openssl/ (104 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/libcrypto.1.1.dylib
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/libssl.1.1.dylib
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/engines-1.1/ (2 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/pkgconfig/ (3 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/ (4 other files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/share/doc/ (3971 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/share/man/ (3971 files)
```
> /usr/local/Cellar/openssl@3/3.0.0_1/bin/c_rehash
> /usr/local/Cellar/openssl@3/3.0.0_1/bin/openssl
> /usr/local/Cellar/openssl@3/3.0.0_1/include/openssl/ (135 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/libcrypto.3.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/libssl.3.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/engines-3/ (3 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/ossl-modules/legacy.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/pkgconfig/ (3 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/ (4 other files)
> /usr/local/Cellar/openssl@3/3.0.0_1/share/doc/ (784 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/share/man/ (5467 files)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952300589
Thank you, @LuciferYang !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950688673
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49050/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950670344
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49050/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954412498
Ya, it might be MacOS issue. I'm now upgrading to `macOS Monterey`.
- https://www.apple.com/macos/monterey/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954406379
Thank you, @toujours33 . Could you try that on the latest OS, `macOS Big Sur`? I really want to prove that my environment has some issues.
```
$ sw_vers
ProductName: macOS
ProductVersion: 11.6
BuildVersion: 20G165
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951872500
@srowen @dongjoon-hyun @wangyum after merge this, I run
```
build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
```
to build and test whole project(87591c9b22157cfd241af6ea2533359c3fba1bb2) with Java 17. It seems that all the UTs have passed.
```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 1.971 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 2.170 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 14.008 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 2.466 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 49.650 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.095 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 1.826 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 1.851 s]
[INFO] Spark Project Core ................................. SUCCESS [24:40 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 17.816 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:27 min]
[INFO] Spark Project Streaming ............................ SUCCESS [04:57 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [07:56 min]
[INFO] Spark Project SQL .................................. SUCCESS [ 01:01 h]
[INFO] Spark Project ML Library ........................... SUCCESS [16:46 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 0.748 s]
[INFO] Spark Project Hive ................................. SUCCESS [ 01:11 h]
[INFO] Spark Project REPL ................................. SUCCESS [01:26 min]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 0.967 s]
[INFO] Spark Project YARN ................................. SUCCESS [06:54 min]
[INFO] Spark Project Mesos ................................ SUCCESS [ 46.913 s]
[INFO] Spark Project Kubernetes ........................... SUCCESS [01:08 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [19:12 min]
[INFO] Spark Ganglia Integration .......................... SUCCESS [ 4.610 s]
[INFO] Spark Project Hadoop Cloud Integration ............. SUCCESS [ 11.400 s]
[INFO] Spark Project Assembly ............................. SUCCESS [ 2.496 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 19.870 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:20 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [35:06 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [ 29.667 s]
[INFO] Spark Project Examples ............................. SUCCESS [ 32.189 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [ 0.949 s]
[INFO] Spark Avro ......................................... SUCCESS [01:55 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 1.104 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:19 h
[INFO] Finished at: 2021-10-26T20:02:56+08:00
[INFO] ------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951872500
@srowen @dongjoon-hyun @wangyum after merge this, I run
```
build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
```
to build and test whole project(head is 87591c9b22157cfd241af6ea2533359c3fba1bb2) with Java 17. It seems that all the UTs have passed.
```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 1.971 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 2.170 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 14.008 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 2.466 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 49.650 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.095 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 1.826 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 1.851 s]
[INFO] Spark Project Core ................................. SUCCESS [24:40 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 17.816 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:27 min]
[INFO] Spark Project Streaming ............................ SUCCESS [04:57 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [07:56 min]
[INFO] Spark Project SQL .................................. SUCCESS [ 01:01 h]
[INFO] Spark Project ML Library ........................... SUCCESS [16:46 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 0.748 s]
[INFO] Spark Project Hive ................................. SUCCESS [ 01:11 h]
[INFO] Spark Project REPL ................................. SUCCESS [01:26 min]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 0.967 s]
[INFO] Spark Project YARN ................................. SUCCESS [06:54 min]
[INFO] Spark Project Mesos ................................ SUCCESS [ 46.913 s]
[INFO] Spark Project Kubernetes ........................... SUCCESS [01:08 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [19:12 min]
[INFO] Spark Ganglia Integration .......................... SUCCESS [ 4.610 s]
[INFO] Spark Project Hadoop Cloud Integration ............. SUCCESS [ 11.400 s]
[INFO] Spark Project Assembly ............................. SUCCESS [ 2.496 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 19.870 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:20 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [35:06 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [ 29.667 s]
[INFO] Spark Project Examples ............................. SUCCESS [ 32.189 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [ 0.949 s]
[INFO] Spark Avro ......................................... SUCCESS [01:55 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 1.104 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:19 h
[INFO] Finished at: 2021-10-26T20:02:56+08:00
[INFO] ------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952334445
From my side, it seems to fail due to `[ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way`.
Could you elaborate your environment a little bit more, @LuciferYang ?
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 s - in org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Running org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.537 s - in org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Running org.apache.spark.network.crypto.AuthEngineSuite
[ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way
[ERROR] /bin/sh: line 1: 35403 Abort trap: 6 /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java -ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=128m -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-exports=java.base/jdk.internal.util.random=ALL-UNNAMED -Dio.netty.tryReflectionSetAccessible=t
rue -jar /Users/dongjoon/.jenkins/workspace/apache-spark-master-java17/common/network-common/target/surefire/surefirebooter17624427968863835516.jar /Users/dongjoon/.jenkins/workspace/apache-spark-master-java17/common/network-common/target/surefire 2021-10-26T14-15-31_852-jvmRun1 surefire2545686690811816554tmp surefire_311767453177444919737tmp
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 4.796 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 9.075 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 21.820 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 9.031 s]
[INFO] Spark Project Networking ........................... FAILURE [ 4.293 s]
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954410477
If this is only my issue, please ignore it. This is irrelevant to this PR completely. I'll try to investigate independently.
Until now, I tested on
- M1 Mac Mini (Big Sur)
- Intel MacBook Pro (BigSur)
- intel MacPro (BigSur)
Tomorrow, I'll get another MacBook Pro (M1 Max). I hope I can build a fresh environment there.
Thank you anyway, @LuciferYang and @srowen and @toujours33 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953082370
Actually, it's not mismatched architecture issue.
- The above was `Intel MacBook Pro` with `zulu17.30.15-ca-jdk17.0.1-macosx_x64`.
- The reason I tested it on `Intel MacBook Pro` wast that `M1 Mini` with `zulu17.28.13-ca-jdk17.0.0-macosx_aarch64` fails at Spark `core` module before
In short, the full Spark test run fails in both environments for me. I didn't get a success on Mac yet. (I didn't try the Linux environment yet.)
> build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954411774
> @srowen @dongjoon-hyun I tried 4 versions of JDK to test
> mvn clean install -DskipTests -pl common/network-common -am
> mvn test -pl common/network-common
> on my M1:
>
> Oracle 17 x64 version: passed
> Oracle 17 arm64 version: passed
> zulu 17 x64 version: Can reproduce the problem @dongjoon-hyun said
> zulu 17 arm64 version: passed
@dongjoon-hyun I found my macos is `Big Sur`, but it is version 11.4
```
sw_vers
ProductName: macOS
ProductVersion: 11.4
BuildVersion: 20F71
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] toujours33 commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
toujours33 commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953485304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954406379
Thank you, @toujours33 . Could you try that on the latest OS, `macOS Big Sur`?
```
$ sw_vers
ProductName: macOS
ProductVersion: 11.6
BuildVersion: 20G165
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952510648
> This seems specific to M1? E.g. cl-plus-ssl/cl-plus-ssl#114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end
It may be that using x86 JDK on M1 will cause this problem？ I use M1 JDK on M1 will not trigger this problem. Let me test `x86 JDK on M1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950939792
This seems fine; I suppose after one change goes in, the change for other modules is mostly just the POM change to tests. We could put many of those together into one rather than review a bunch of modules individually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950688673
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49050/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953082370
Actually, it's not mismatched architecture issue.
- The above was `Intel MacBook Pro` with `zulu17.30.15-ca-jdk17.0.1-macosx_x64`.
- The reason I tested it on `Intel MacBook Pro` wast that `M1 Mini` with `zulu17.28.13-ca-jdk17.0.0-macosx_aarch64` failed at Spark `core` module before
In short, the full Spark test run fails in both environments for me. I didn't get a success on Mac yet. (I didn't try the Linux environment yet.)
> build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953084787
~I'll try to use AdoptOpenJDK today and share the result to you folks.~
Oops. I realized that AdoptOpenJDK doesn't have Java 17 yet.
- https://adoptopenjdk.net
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954412711
> The used environment was the following.
>
> $ brew list openssl
> /usr/local/Cellar/openssl@3/3.0.0_1/.bottle/etc/ (7 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/bin/c_rehash
> /usr/local/Cellar/openssl@3/3.0.0_1/bin/openssl
> /usr/local/Cellar/openssl@3/3.0.0_1/include/openssl/ (135 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/libcrypto.3.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/libssl.3.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/engines-3/ (3 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/ossl-modules/legacy.dylib
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/pkgconfig/ (3 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/lib/ (4 other files)
> /usr/local/Cellar/openssl@3/3.0.0_1/share/doc/ (784 files)
> /usr/local/Cellar/openssl@3/3.0.0_1/share/man/ (5467 files)
But I install openssl@1.1
```
brew list openssl@1.1
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/.bottle/etc/ (7 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/bin/c_rehash
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/bin/openssl
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/include/openssl/ (104 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/libcrypto.1.1.dylib
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/libssl.1.1.dylib
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/engines-1.1/ (2 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/pkgconfig/ (3 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/lib/ (4 other files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/share/doc/ (3971 files)
/opt/homebrew/Cellar/openssl@1.1/1.1.1l_1/share/man/ (3971 files)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950591384
**[Test build #144579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144579/testReport)** for PR 34376 at commit [`b63e2d7`](https://github.com/apache/spark/commit/b63e2d79fde0ee406338d977427e8247d4a865dc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950906371
cc @srowen @dongjoon-hyun @wangyum @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952991167
If it's just a specific scenario and zulu's problem, I think we can ignore it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952301184
BTW, @LuciferYang . We will add GitHub Action job only. Could you revise, SPARK-37120?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952509642
This seems specific to M1? E.g. https://github.com/cl-plus-ssl/cl-plus-ssl/issues/114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954412962
`brew` changed it recently~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950745087
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144579/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950733206
**[Test build #144579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144579/testReport)** for PR 34376 at commit [`b63e2d7`](https://github.com/apache/spark/commit/b63e2d79fde0ee406338d977427e8247d4a865dc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #34376:
URL: https://github.com/apache/spark/pull/34376
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953447834
It seems that I still need to prepare an Intel MacBook Pro ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952587999
@srowen @dongjoon-hyun I tried 4 versions of JDK on my M1:
- Oracle 17 x64 version: passed
- Oracle 17 arm64 version: passed
- zulu 17 x64 version: Can reproduce the problem @dongjoon-hyun said
- zulu 17 arm64 version: passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] toujours33 commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
toujours33 commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954410435
> Thank you, @toujours33 . Could you try that on the latest OS, `macOS Big Sur`? I really want to prove that my environment has some issues.
>
> ```
> $ sw_vers
> ProductName: macOS
> ProductVersion: 11.6
> BuildVersion: 20G165
> ```
Of course, I will try it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954410477
If this is only my issue, please ignore it. This is irrelevant to this PR at all. I'll try to investigate independently.
Until now, I tested on
- M1 Mac Mini (Big Sur)
- Intel MacBook Pro (BigSur)
- intel MacPro (BigSur)
Tomorrow, I'll get another MacBook Pro (M1 Max). I hope I can build a fresh environment there.
Thank you anyway, @LuciferYang and @srowen and @toujours33 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952506879
@dongjoon-hyun and I test this module on a M1
```
mvn clean install -DskipTests -pl common/network-common -am
mvn test -pl common/network-common
```
```
[INFO] Running org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.083 s - in org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Running org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.19 s - in org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Running org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.087 s - in org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Running org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.153 s - in org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Running org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 s - in org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Running org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.786 s - in org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Running org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s - in org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Running org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Running org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Running org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.304 s - in org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Running org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.141 s - in org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Running org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 s - in org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Running org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Running org.apache.spark.network.ProtocolSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.ProtocolSuite
[INFO] Running org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Running org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.043 s - in org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Running org.apache.spark.network.protocol.EncodersSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.EncodersSuite
[INFO] Running org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Running org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Running org.apache.spark.network.StreamSuite
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.091 s - in org.apache.spark.network.StreamSuite
[INFO] Running org.apache.spark.network.RpcIntegrationSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 s - in org.apache.spark.network.RpcIntegrationSuite
[INFO] Running org.apache.spark.network.client.TransportClientFactorySuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.278 s - in org.apache.spark.network.client.TransportClientFactorySuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (test) @ spark-network-common_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO]
[INFO] --- scalatest-maven-plugin:2.0.2:test (test) @ spark-network-common_2.12 ---
Discovery starting.
Discovery completed in 106 milliseconds.
Run starting. Expected test count is: 0
DiscoverySuite:
Run completed in 128 milliseconds.
Total number of tests run: 0
Suites: completed 1, aborted 0
Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
No tests were executed.
```
```
java -version
openjdk version "17" 2021-09-14 LTS
OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS)
OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952504795
> From my side, it seems to fail due to [ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way.
@dongjoon-hyun Is this an x86 macos or an M1 macos? I'm sorry, I don't have an available x86 MacOS , so all tests were run on X86 Linux
```
uname -a
Linux $hostname 4.14.0_1-0-0-42 #1 SMP Tue Jul 20 16:37:22 CST 2021 x86_64 x86_64 x86_64 GNU/Linux
```
```
lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.3 (Final)
Release: 6.3
Codename: Final
```
```
java -version
openjdk version "17" 2021-09-14 LTS
OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS)
OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952506879
@dongjoon-hyun and I test this module on a M1
```
uname -a
Darwin B000000396921U 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:xnu-7195.121.3~9/RELEASE_ARM64_T8101 arm64
```
```
mvn clean install -DskipTests -pl common/network-common -am
mvn test -pl common/network-common
```
```
[INFO] Running org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.083 s - in org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Running org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.19 s - in org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Running org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.087 s - in org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Running org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.153 s - in org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Running org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 s - in org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Running org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.786 s - in org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Running org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s - in org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Running org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Running org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Running org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.304 s - in org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Running org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.141 s - in org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Running org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 s - in org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Running org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Running org.apache.spark.network.ProtocolSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.ProtocolSuite
[INFO] Running org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Running org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.043 s - in org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Running org.apache.spark.network.protocol.EncodersSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.EncodersSuite
[INFO] Running org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Running org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Running org.apache.spark.network.StreamSuite
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.091 s - in org.apache.spark.network.StreamSuite
[INFO] Running org.apache.spark.network.RpcIntegrationSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 s - in org.apache.spark.network.RpcIntegrationSuite
[INFO] Running org.apache.spark.network.client.TransportClientFactorySuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.278 s - in org.apache.spark.network.client.TransportClientFactorySuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (test) @ spark-network-common_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO]
[INFO] --- scalatest-maven-plugin:2.0.2:test (test) @ spark-network-common_2.12 ---
Discovery starting.
Discovery completed in 106 milliseconds.
Run starting. Expected test count is: 0
DiscoverySuite:
Run completed in 128 milliseconds.
Total number of tests run: 0
Suites: completed 1, aborted 0
Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
No tests were executed.
```
```
java -version
openjdk version "17" 2021-09-14 LTS
OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS)
OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950906371
cc @srowen @dongjoon-hyun @wangyum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952883157
OK can we ignore this? It's specific to one JDK and 'mismatched' architecture, and I don't think we can fix it on this end even so
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954413550
BTW, please note that `M1 Mini on BigSur` passed the above and failed at `core` with the different reason with LevelDB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952504795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952504795
> From my side, it seems to fail due to [ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way.
@dongjoon-hyun Is this test on x86 or M1? I'm sorry, I don't have an available x86 MacOS to test this, so all tests were run on X86 Linux
```
uname -a
Linux $hostname 4.14.0_1-0-0-42 #1 SMP Tue Jul 20 16:37:22 CST 2021 x86_64 x86_64 x86_64 GNU/Linux
```
```
lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.3 (Final)
Release: 6.3
Codename: Final
```
```
java -version
openjdk version "17" 2021-09-14 LTS
OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS)
OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951872500
@srowen @dongjoon-hyun @wangyum after merge this, I run
```
build/mvn clean install -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
```
to build and test whole project(Head is 87591c9b22157cfd241af6ea2533359c3fba1bb2) with Java 17. It seems that all the UTs have passed.
```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 1.971 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 2.170 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 14.008 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 2.466 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 49.650 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.095 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 1.826 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 1.851 s]
[INFO] Spark Project Core ................................. SUCCESS [24:40 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 17.816 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:27 min]
[INFO] Spark Project Streaming ............................ SUCCESS [04:57 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [07:56 min]
[INFO] Spark Project SQL .................................. SUCCESS [ 01:01 h]
[INFO] Spark Project ML Library ........................... SUCCESS [16:46 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 0.748 s]
[INFO] Spark Project Hive ................................. SUCCESS [ 01:11 h]
[INFO] Spark Project REPL ................................. SUCCESS [01:26 min]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 0.967 s]
[INFO] Spark Project YARN ................................. SUCCESS [06:54 min]
[INFO] Spark Project Mesos ................................ SUCCESS [ 46.913 s]
[INFO] Spark Project Kubernetes ........................... SUCCESS [01:08 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [19:12 min]
[INFO] Spark Ganglia Integration .......................... SUCCESS [ 4.610 s]
[INFO] Spark Project Hadoop Cloud Integration ............. SUCCESS [ 11.400 s]
[INFO] Spark Project Assembly ............................. SUCCESS [ 2.496 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 19.870 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:20 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [35:06 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [ 29.667 s]
[INFO] Spark Project Examples ............................. SUCCESS [ 32.189 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [ 0.949 s]
[INFO] Spark Avro ......................................... SUCCESS [01:55 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 1.104 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:19 h
[INFO] Finished at: 2021-10-26T20:02:56+08:00
[INFO] ------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954431710
> BTW, please note that M1 Mini on BigSur passed the above and failed at core with the different reason with LevelDB.
I know this. I tried to compile an available levedldbjni but failed 😣
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951888490
I created a JIRA SPARK-37120 to tracking `Add a Jenkins build and test job for Java 17`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951888490
I'm not sure if we should add a Jenkins build and test job for Java 17, but I create a new JIRA SPARK-37120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952011985
Thank you @LuciferYang.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950745087
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144579/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950591384
**[Test build #144579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144579/testReport)** for PR 34376 at commit [`b63e2d7`](https://github.com/apache/spark/commit/b63e2d79fde0ee406338d977427e8247d4a865dc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953483623
@toujours33 Because you use Intel MacBook Pro, can you help verify this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-954406379
Thank you, @toujours33 . Could you try that on the latest OS, `macOS Big Sur`? I really want to prove that my environment has some issues.
```
$ sw_vers
ProductName: macOS
ProductVersion: 11.6
BuildVersion: 20G165
```
The used environment was the following.
```
$ brew list openssl
/usr/local/Cellar/openssl@3/3.0.0_1/.bottle/etc/ (7 files)
/usr/local/Cellar/openssl@3/3.0.0_1/bin/c_rehash
/usr/local/Cellar/openssl@3/3.0.0_1/bin/openssl
/usr/local/Cellar/openssl@3/3.0.0_1/include/openssl/ (135 files)
/usr/local/Cellar/openssl@3/3.0.0_1/lib/libcrypto.3.dylib
/usr/local/Cellar/openssl@3/3.0.0_1/lib/libssl.3.dylib
/usr/local/Cellar/openssl@3/3.0.0_1/lib/engines-3/ (3 files)
/usr/local/Cellar/openssl@3/3.0.0_1/lib/ossl-modules/legacy.dylib
/usr/local/Cellar/openssl@3/3.0.0_1/lib/pkgconfig/ (3 files)
/usr/local/Cellar/openssl@3/3.0.0_1/lib/ (4 other files)
/usr/local/Cellar/openssl@3/3.0.0_1/share/doc/ (784 files)
/usr/local/Cellar/openssl@3/3.0.0_1/share/man/ (5467 files)
```
- Oracle Java 17
```
$ java -version
java version "17.0.1" 2021-10-19 LTS
Java(TM) SE Runtime Environment (build 17.0.1+12-LTS-39)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.1+12-LTS-39, mixed mode, sharing)
```
- Oracle Failure
```
[INFO] Running org.apache.spark.network.crypto.AuthEngineSuite
[ERROR] WARNING: /Users/dongjoon/A/jdk-release/jdk-17.0.1.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way
[ERROR] /bin/sh: line 1: 95181 Abort trap: 6 /Users/dongjoon/A/jdk-release/jdk-17.0.1.jdk/Contents/Home/bin/java -ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=128m -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-exports=java.base/jdk.internal.util.random=ALL-UNNAMED -Dio.netty.tryReflectionSetAccessible=true -jar /Users/dongjoon/APACHE/sp
ark-merge/common/network-common/target/surefire/surefirebooter1249551704044360879.jar /Users/dongjoon/APACHE/spark-merge/common/network-common/target/surefire 2021-10-28T20-48-58_948-jvmRun1 surefire14051958684954209309tmp surefire_16417539655447883687tmp
```
- Zulu 17
```
$ java -version
openjdk version "17.0.1" 2021-10-19 LTS
OpenJDK Runtime Environment Zulu17.30+15-CA (build 17.0.1+12-LTS)
OpenJDK 64-Bit Server VM Zulu17.30+15-CA (build 17.0.1+12-LTS, mixed mode, sharing)
```
- Zulu Failure
```
[ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way
[ERROR] /bin/sh: line 1: 96409 Abort trap: 6 /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java -ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=128m -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-exports=java.base/jdk.internal.util.random=ALL-UNNAMED -Dio.netty.tryReflectionSetAccessible=t
rue -jar /Users/dongjoon/APACHE/spark-merge/common/network-common/target/surefire/surefirebooter7191129155144461475.jar /Users/dongjoon/APACHE/spark-merge/common/network-common/target/surefire 2021-10-28T20-50-23_169-jvmRun1 surefire16306363663564275511tmp surefire_14930262903376525483tmp
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953084787
I'll try to use AdoptOpenJDK today and share the result to you folks.
- https://adoptopenjdk.net
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953084787
~I'll try to use AdoptOpenJDK today and share the result to you folks.~
Oops. I realized that AdoptOpenJDK doesn't have Java 17 yet.
- https://adoptopenjdk.net
I'll try to use Oracle JDK instead.
- https://www.oracle.com/java/technologies/downloads/#java17
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952504795
> From my side, it seems to fail due to [ERROR] WARNING: /Users/dongjoon/A/jdk-release/zulu17.30.15-ca-jdk17.0.1-macosx_x64/zulu-17.jdk/Contents/Home/bin/java is loading libcrypto in an unsafe way.
@dongjoon-hyun Is this an x86 macos or an M1 macos? I'm sorry, I don't have an available x86 MacOS , so all tests were run on X86 Linux
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952506879
@dongjoon-hyun and I test this module on a M1
```
mvn clean install -DskipTests -pl common/network-common -am
mvn test -pl common/network-common
```
```
[INFO] Running org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.083 s - in org.apache.spark.network.crypto.AuthMessagesSuite
[INFO] Running org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.19 s - in org.apache.spark.network.crypto.TransportCipherSuite
[INFO] Running org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.087 s - in org.apache.spark.network.crypto.AuthEngineSuite
[INFO] Running org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.153 s - in org.apache.spark.network.crypto.AuthIntegrationSuite
[INFO] Running org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 s - in org.apache.spark.network.ChunkFetchIntegrationSuite
[INFO] Running org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.786 s - in org.apache.spark.network.util.TransportFrameDecoderSuite
[INFO] Running org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s - in org.apache.spark.network.util.NettyMemoryMetricsSuite
[INFO] Running org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.util.TimerWithCustomUnitSuite
[INFO] Running org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.util.CryptoUtilsSuite
[INFO] Running org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.304 s - in org.apache.spark.network.RequestTimeoutIntegrationSuite
[INFO] Running org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.141 s - in org.apache.spark.network.sasl.SparkSaslSuite
[INFO] Running org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 s - in org.apache.spark.network.ChunkFetchRequestHandlerSuite
[INFO] Running org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in org.apache.spark.network.server.OneForOneStreamManagerSuite
[INFO] Running org.apache.spark.network.ProtocolSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s - in org.apache.spark.network.ProtocolSuite
[INFO] Running org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.MessageWithHeaderSuite
[INFO] Running org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.043 s - in org.apache.spark.network.protocol.MergedBlockMetaSuccessSuite
[INFO] Running org.apache.spark.network.protocol.EncodersSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.spark.network.protocol.EncodersSuite
[INFO] Running org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.apache.spark.network.TransportResponseHandlerSuite
[INFO] Running org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in org.apache.spark.network.TransportRequestHandlerSuite
[INFO] Running org.apache.spark.network.StreamSuite
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.091 s - in org.apache.spark.network.StreamSuite
[INFO] Running org.apache.spark.network.RpcIntegrationSuite
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 s - in org.apache.spark.network.RpcIntegrationSuite
[INFO] Running org.apache.spark.network.client.TransportClientFactorySuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.278 s - in org.apache.spark.network.client.TransportClientFactorySuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (test) @ spark-network-common_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO]
[INFO] --- scalatest-maven-plugin:2.0.2:test (test) @ spark-network-common_2.12 ---
Discovery starting.
Discovery completed in 106 milliseconds.
Run starting. Expected test count is: 0
DiscoverySuite:
Run completed in 128 milliseconds.
Total number of tests run: 0
Suites: completed 1, aborted 0
Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
No tests were executed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951436882
> This seems fine; I suppose after one change goes in, the change for other modules is mostly just the POM change to tests. We could put many of those together into one rather than review a bunch of modules individually.
OK～thank you for your advice～
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952510648
> This seems specific to M1? E.g. cl-plus-ssl/cl-plus-ssl#114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end
May be using x86 JDK on M1 will cause this problem？ I use M1 JDK on M1 will not trigger this problem. Let me test `x86 JDK on M1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-952510648
> This seems specific to M1? E.g. cl-plus-ssl/cl-plus-ssl#114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end
May be using x64 JDK on M1 will cause this problem？ I use M1 JDK on M1 will not trigger this problem. Let me test `x86 JDK on M1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951888490
I'm not sure should we need add a Jenkins build and test job for Java 17, but I create a new JIRA SPARK-37120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] toujours33 commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
toujours33 commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953485056
It seems ok on Intel MacBook Pro with zulu 17
```
build/mvn clean install -DskipTests -pl common/network-common -am
build/mvn test -pl common/network-common
```
environment
```
MacOs: Mojave 10.14.6
Cpu: 2.3 GHz Intel Core i5
Java version:
openjdk version "17.0.1" 2021-10-19 LTS
OpenJDK Runtime Environment 21.10-(Zulu-17.30+15-macosx_x64)-Microsoft-Azure-restricted (build 17.0.1+12-LTS)
OpenJDK 64-Bit Server VM 21.10-(Zulu-17.30+15-macosx_x64)-Microsoft-Azure-restricted (build 17.0.1+12-LTS, mixed mode, sharing)
```
@LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-953483623
@toujours33 Since you use Intel MacBook Pro, can you help verify this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-951121986
Let me merge this first, @srowen . For the rest of modules, please follow @srowen 's advice, @LuciferYang .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34376:
URL: https://github.com/apache/spark/pull/34376#issuecomment-950628046
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49050/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org