[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #159: HDDS-2477. TableCache cleanup issue for OM non-HA.:

[GitBox <gi...@apache.org>]

bharatviswa504 opened a new pull request #159: HDDS-2477. TableCache cleanup issue for OM non-HA.:
URL: https://github.com/apache/hadoop-ozone/pull/159
 
 
   ## What changes were proposed in this pull request?
   
   In OM in non-HA case, the ratisTransactionLogIndex is generated by OmProtocolServersideTranslatorPB.java. And in OM non-HA validateAndUpdateCache is called from multipleHandler threads. So think of a case where one thread which has an index - 10 has added to doubleBuffer. (0-9 still have not added). DoubleBuffer flush thread flushes and call cleanup. (So, now cleanup will go and cleanup all cache entries with less than 10 epoch) This should not have cleanup those which might have put in to cache later and which are in process of flush to DB. This will cause inconsitency for few OM requests.
   
   Example:
   
   4 threads Committing 4 parts.
   
   1st thread - part 1 - ratis Index - 3
   
   2nd thread - part 2 - ratis index - 2
   
   3rd thread - part3 - ratis index - 1
   
    
   
   First thread got lock, and put in to doubleBuffer and cache with OmMultipartInfo (with part1). And cleanup is called to cleanup all entries in cache with less than 3. In the mean time 2nd thread and 1st thread put 2,3 parts in to OmMultipartInfo in to Cache and doubleBuffer. But first thread might cleanup those entries, as it is called with index 3 for cleanup.
   
    
   
   Now when the 4th part upload came -> when it is commit Multipart upload when it gets multipartinfo it get Only part1 in OmMultipartInfo, as the OmMultipartInfo (with 1,2,3 is still in process of committing to DB). So now after 4th part upload is complete in DB and Cache we will have 1,4 parts only. We will miss part2,3 information.
   
    
   
   So for non-HA case cleanup will be called with list of epochs that need to be cleanedup.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-2477
   
   ## How was this patch tested?
   
   Added UT.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org