You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/12/11 16:29:31 UTC

[GitHub] [spark] xkrogen commented on pull request #30725: [SPARK-33753][CORE] Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

xkrogen commented on pull request #30725:
URL: https://github.com/apache/spark/pull/30725#issuecomment-743293158


   Do we really need a separate copy of the JobConf cached for each partition ID? Is there any opportunity for us to reduce the number of JobConfs to begin with? It seems like all of the partitions should be able to safely share the same conf object...?
   
   Regardless, weak references seem more appropriate here than soft.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org