You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/03/18 05:23:22 UTC
[GitHub] [spark] HeartSaVioR commented on issue #24113: [MINOR][CORE]
Replace Scala forkjoin package to Java bundled one
HeartSaVioR commented on issue #24113: [MINOR][CORE] Replace Scala forkjoin package to Java bundled one
URL: https://github.com/apache/spark/pull/24113#issuecomment-473773947
I just realized that the code would not compile with `-Pscala-2.11` profile, since `ForkJoinTaskSupport` in Scala 2.11 requires Scala version of `ForkJoinPool` instead of Java bundled one - so we cannot do this unless we set lower bar of Scala to `scala-2.12`.
Btw, once we want to both support Scala 2.12 and Scala 2.13, current code will break since they look to simply remove the alias in Scala 2.13 (They seem to remove ForkJoinTaskSupport in scala 2.13, too). Maybe broader fix should be needed at that time then.
One interesting thing is, some claims performance regression on changing ForkJoinPool from Scala one to Java bundled one (which is done Scala 2.12), and Akka team decided to keep Scala 2.11 version of ForkJoinPool in their repository. I'm not sure I can follow the discussions and changes related to this (they stated that the perf difference came from using busy-wait), but the implementation of ForkJoinPool on Scala 2.11 and Java bundled one sounds to be different, and perf. regression might happen silently between Scala 2.11 vs Scala 2.12.
http://www.scala-archive.org/ForkJoinPool-ExecutionContext-Implicits-global-performance-problem-for-Scala-2-12-td4647851.html
https://github.com/scala/bug/issues/10083
https://akka.io/blog/news/2017/04/13/akka-2.5.0-released
https://github.com/akka/akka/blob/master/akka-actor/src/main/java/akka/dispatch/forkjoin/ForkJoinPool.java
Anyway let me close the PR for now. Thanks all for spending time to review, and thanks @dongjoon-hyun to let me realize about supporting Scala versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org