[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...

[akashrn5 <gi...@git.apache.org>]

Github user akashrn5 commented on the issue:

    https://github.com/apache/carbondata/pull/2113
  
    When the loading is triggered after creation of table and datamap , it failed with lock exception.
    Analysis:
    The index write location for the lucene is same, and to IndexWriter will take a lock file called write.lock in write location while writing the index files. In carbon loading the writer tasks are launched parallel and those many writers are opened,
    Since the write.lock file is acquired by one writer, all other tasks will fail and dataloading will fail.
    
    Solution
    so in this PR, the write path can be configured on the fly, with task id and timestamp mapping.
    
    
    Handling in reader part:
    In query side, there was a single indexSearch object which does search on index file present in a single location.
    Now with thsi dataload fix, there will be multiple directories which will have index files, and cannot give a single path to search object, as serch object is one.
    
    Probable Solutions:
    1. create those many number of search objects, as number of index file directories created in data load. and finally return all the final Blocklets.
    2. you can have one searcher in one executor to search the local folder, this part im working how to do



---