[GitHub] [spark] LucaCanali commented on pull request #29477: [SPARK-32661][K8S] Spark executors on K8S should request extra memory for off-heap allocations.

[GitBox <gi...@apache.org>]

LucaCanali commented on pull request #29477:
URL: https://github.com/apache/spark/pull/29477#issuecomment-702604124


   @dongjoon-hyun I see that this is a minor bug. I'd say that if there is no time or necessity to fix this during the current phase of Spark on K8s GA preparation, another option could be just to add a note in the documentation, with somehting like  "Executors on K8S will not allocate extra memory for off-heap allocations based on the configuration `spark.memory.offHeap.size`, differently from what happens with executors on YARN. Users will have to allocate the extra memory needed for off-heap allocations on K8S using the configuration parameter `spark.executorMemoryOverhead` and/or `spark.kubernetes.memoryOverheadFactor`."


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org