[GitHub] [spark] tgravescs commented on pull request #31942: [SPARK-34834][NETWORK] Fix a potential Netty memory leak in TransportResponseHandler.

[GitBox <gi...@apache.org>]

tgravescs commented on pull request #31942:
URL: https://github.com/apache/spark/pull/31942#issuecomment-810267053


   it would be nice to have more description in this pr and jira.  When is this possible, what are the ramifications of this, was this actually hit in real application, etc.  
   
   > And in the case of ChunkFetchSuccess, I wonder we may release the buffer here too early since the listener.onSuccess(...) is executed asynchronously:
   
   That shouldn't happen as the caller is supposed to retain or copy it like you referenced, but yeah in the shuffle fetcher at first glance it doesn't look like its cleaned up in the good case after calling retain on it. I'll have to look a bit deeper.
   
   It would really be nice to have a test for this, even if its manual once doing heap dumps or changing code to see if we have some actual leaks or if there is a path we are missing here.  This comes back to my original request of having more information in the jira.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org