You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/04/15 03:47:33 UTC

[GitHub] [spark] dongjoon-hyun edited a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers

dongjoon-hyun edited a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
URL: https://github.com/apache/spark/pull/27695#issuecomment-613799273
 
 
   @onursatici . I'm still not sure about this approach. 
   - First, K8s environment is different from on-prem environment. I don't think the user of Apache Spark 3.1.0 will use this kind of old JDKs (JDK 8u120 or older). We have reached the milestone of JDK11 at Apache Spark 3.0.0.
   - Second, even in old JDKs, this original YARN code is made by the following PR for `Netty`. In K8s environment, the available number of core is the same with the requested amount.
     - https://github.com/apache/spark/pull/21885 ([SPARK-24926][CORE] Ensure numCores is used consistently in all netty configurations)
   
   So, do you want to have bigger parallelism on the small container?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org