[GitHub] [spark] dongjoon-hyun edited a comment on issue #25443: [WIP][SPARK-28723][test-hadoop3.2][test-maven] Test JDK 11 with Hadoop-3.2/Hive 2.3.6 on jenkins

[GitBox <gi...@apache.org>]

dongjoon-hyun edited a comment on issue #25443: [WIP][SPARK-28723][test-hadoop3.2][test-maven] Test JDK 11 with Hadoop-3.2/Hive 2.3.6 on jenkins
URL: https://github.com/apache/spark/pull/25443#issuecomment-521083767
 
 
   If we build and test with both JDK11, it will pass. The current Jenkins is building with JDK8 and running on JDK11 and hit this known issue.
   ```
   $ build/sbt "core/testOnly *.ExternalSorterSuite"
   [info] ExternalSorterSuite:
   WARNING: An illegal reflective access operation has occurred
   WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/Users/dhyun/PRS/SPARK-HIVE-2.3.6/common/unsafe/target/scala-2.12/spark-unsafe_2.12-3.0.0-SNAPSHOT.jar) to constructor java.nio.DirectByteBuffer(long,int)
   WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
   WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
   WARNING: All illegal access operations will be denied in a future release
   [info] - empty data stream with kryo ser (1 second, 457 milliseconds)
   [info] - empty data stream with java ser (89 milliseconds)
   [info] - few elements per partition with kryo ser (89 milliseconds)
   [info] - few elements per partition with java ser (74 milliseconds)
   [info] - empty partitions with spilling with kryo ser (329 milliseconds)
   [info] - empty partitions with spilling with java ser (156 milliseconds)
   [info] - spilling in local cluster with kryo ser (4 seconds, 296 milliseconds)
   [info] - spilling in local cluster with java ser (4 seconds, 372 milliseconds)
   [info] - spilling in local cluster with many reduce tasks with kryo ser (5 seconds, 718 milliseconds)
   [info] - spilling in local cluster with many reduce tasks with java ser (6 seconds, 51 milliseconds)
   [info] - cleanup of intermediate files in sorter (113 milliseconds)
   [info] - cleanup of intermediate files in sorter with failures (111 milliseconds)
   [info] - cleanup of intermediate files in shuffle (297 milliseconds)
   [info] - cleanup of intermediate files in shuffle with failures (121 milliseconds)
   [info] - no sorting or partial aggregation with kryo ser (58 milliseconds)
   [info] - no sorting or partial aggregation with java ser (53 milliseconds)
   [info] - no sorting or partial aggregation with spilling with kryo ser (62 milliseconds)
   [info] - no sorting or partial aggregation with spilling with java ser (68 milliseconds)
   [info] - sorting, no partial aggregation with kryo ser (63 milliseconds)
   [info] - sorting, no partial aggregation with java ser (54 milliseconds)
   [info] - sorting, no partial aggregation with spilling with kryo ser (58 milliseconds)
   [info] - sorting, no partial aggregation with spilling with java ser (61 milliseconds)
   [info] - partial aggregation, no sorting with kryo ser (52 milliseconds)
   [info] - partial aggregation, no sorting with java ser (51 milliseconds)
   [info] - partial aggregation, no sorting with spilling with kryo ser (55 milliseconds)
   [info] - partial aggregation, no sorting with spilling with java ser (49 milliseconds)
   [info] - partial aggregation and sorting with kryo ser (44 milliseconds)
   [info] - partial aggregation and sorting with java ser (44 milliseconds)
   [info] - partial aggregation and sorting with spilling with kryo ser (48 milliseconds)
   [info] - partial aggregation and sorting with spilling with java ser (49 milliseconds)
   [info] - sort without breaking sorting contracts with kryo ser (1 second, 904 milliseconds)
   [info] - sort without breaking sorting contracts with java ser (1 second, 860 milliseconds)
   [info] - sort without breaking timsort contracts for large arrays !!! IGNORED !!!
   [info] - spilling with hash collisions (208 milliseconds)
   [info] - spilling with many hash collisions (589 milliseconds)
   [info] - spilling with hash collisions using the Int.MaxValue key (168 milliseconds)
   [info] - spilling with null keys and values (226 milliseconds)
   [info] - sorting updates peak execution memory (1 second, 347 milliseconds)
   [info] - force to spill for external sorter (800 milliseconds)
   [info] ScalaTest
   [info] Run completed in 34 seconds, 99 milliseconds.
   [info] Total number of tests run: 38
   [info] Suites: completed 1, aborted 0
   [info] Tests: succeeded 38, failed 0, canceled 0, ignored 1, pending 0
   [info] All tests passed.
   ```
   
   cc @srowen 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org