[GitHub] [carbondata] akashrn5 commented on pull request #3733: [CARBONDATA-3789]Fix cache issue in case of compaction failure in compaction post listeners

[GitBox <gi...@apache.org>]

akashrn5 commented on pull request #3733:
URL: https://github.com/apache/carbondata/pull/3733#issuecomment-623396522


   > @akashrn5 : Issue is happening when a segment is removed from main table but cache is not cleaned right?
   > 
   > When the SI table compaction disable why not just disable SI, why need to remove whole segment of main table ?
   
   @ajantha-bhat , once the main table compaction is finished, say example, 0.1 for segments 1, 2, then 0.1 segment data of the main table is selected and loaded to SI table's 0.1 segment. So if while loading 0.1 to SI fails, then 0.1 entry won't be present in the main table, and the cache will have this entry for the main table.
   
   So when next time again the same 0.1 is compacted for 0,1 segments as it was failed before, in SI it will fire a select query to 0.1 of the main table, since the cache is not cleared, it will try for old carbondata files and say file not found.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org