[GitHub] [carbondata] dhatchayani opened a new pull request #3226: [CARBONDATA-3393] Merge Index Job Failure should not fail the LOAD. Load status should not consider the merge index job status.

[GitBox <gi...@apache.org>]

dhatchayani opened a new pull request #3226: [CARBONDATA-3393] Merge Index Job Failure should not fail the LOAD. Load status should not consider the merge index job status.
URL: https://github.com/apache/carbondata/pull/3226
 
 
   **Problem:**
   If the merge index job is failed, then the LOAD is also failing.
   
   **Solution:**
   Merge index job status should not be considered for the actual LOAD STATUS. Load should pass, as the merge index is only the extra optimization and not the actual load stage. Merge index can be triggered later if needed manually by the ALTER TABLE <table_name> COMPACT 'SEGMENT_INDEX' command.
   
   **Changes:**
   (1) Merge index job failure will not be propagated to the caller. And will only be LOGGED.
   (2) Implement a new method to write the SegmentFile based on the current load timestamp. This helps in case of merge index failures and writing merge index for old store.
   
    - [ ] Any interfaces changed?
    
    - [ ] Any backward compatibility impacted?
    
    - [ ] Document update required?
   
    - [x] Testing done
           Existing UT
   
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. 
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services