[GitHub] [hadoop] snvijaya commented on pull request #5117: HADOOP-18521. ABFS ReadBufferManager must not reuse in-progress buffers

[GitBox <gi...@apache.org>]

snvijaya commented on PR #5117:
URL: https://github.com/apache/hadoop/pull/5117#issuecomment-1313734822

   Hi @steveloughran, Wanted to get your opinion on below change as a possible replacement for this change :
   [https://github.com/apache/hadoop/pull/5133](url)
   
   A ReadBuffer with a valid Buffer assigned to it can be in certain states when stream is closed, and with the above change, I am trying to address it as below :
   1. Is in QueueReadAheadList - No change, the earlier purge takes care of it
   2. Is in CompletedList - No change again, the earlier purge takes care of it
   3. Is InProgressList but yet to make the network call - If stream is closed, stop network call and move the ReadBuffer as a failure into completed list
   4. Is InProgressList , just finished the network call - If stream is closed, network call was successful or not, move the ReadBuffer as a failure into completed list
   
   Now, when in state 3 or 4, the purge method might not pick it as it might have executed first. In that case, to prioritize these ReadBuffers for eviction, have added the check for stream is closed in the eviction code as well. 
   
   Please let me know if you see value in this fix and I could pursue further changes to incorporate validation code at queuing time and when getBlock finds a hit in completed list, and will also add related test code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org