You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by BJangir <gi...@git.apache.org> on 2018/11/05 11:10:49 UTC

[GitHub] carbondata pull request #2897: [WIP] Supporting local dictionary enable by d...

GitHub user BJangir opened a pull request:

    https://github.com/apache/carbondata/pull/2897

    [WIP] Supporting local dictionary enable by default for SDK

    Be sure to do all of the following checklist to help us incorporate 
    your contribution quickly and easily:
    
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
    
     - [ ] Testing done
            Please provide details on 
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. 
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/BJangir/incubator-carbondata sdk_local_dic_default

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2897
    
----
commit 83d11643f1c9acc2b8d3f206fd304ae8ee686160
Author: BJangir <ba...@...>
Date:   2018-11-05T11:11:16Z

    Supporting local dictionary enable by default for SDK

----


---

[GitHub] carbondata pull request #2897: [WIP] Supporting local dictionary enable by d...

Posted by BJangir <gi...@git.apache.org>.
Github user BJangir commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r230736008
  
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
    @@ -64,7 +64,8 @@
       private Map<String, String> options;
       private String taskNo;
       private int localDictionaryThreshold;
    -  private boolean isLocalDictionaryEnabled;
    +  private boolean isLocalDictionaryEnabled = Boolean.parseBoolean(
    --- End diff --
    
    it is already handled in below method when isLocalDictionaryEnabled is true
    org.apache.carbondata.core.metadata.schema.table.TableSchemaBuilder#build


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10150/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10091/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1779/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9538/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10042/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1500/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10181/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1285/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10163/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1780/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1794/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2116/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed  with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10155/



---

[GitHub] carbondata issue #2897: [WIP] Supporting local dictionary enable by default ...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1274/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    @BJangir ...please go through the discussion and then decide whether for SDK be default we should enable local dictionary or not. We can further discuss on it based on your view
    http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Feature-Proposal-Proposal-for-offline-and-DDL-local-dictionary-support-td67620.html


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed  with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10159/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r243810091
  
    --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/util/CarbonVectorizedRecordReader.java ---
    @@ -68,6 +67,10 @@
       private AbstractDetailQueryResultIterator iterator;
     
       private QueryModel queryModel;
    +  //This holds mapping of  fetch index with respect to project col index.
    +  // it is used when same col is used in projection many times.So need to fetch only that col.
    +  private List<Integer> projectionMapping = new ArrayList<>();
    --- End diff --
    
    Better use  an array to have fast lookups instead of list


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1993/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r242423291
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/LocalDictDimensionDataChunkStore.java ---
    @@ -94,10 +93,9 @@ public void fillVector(int[] invertedIndex, int[] invertedIndexReverse, byte[] d
       }
     
       @Override public void fillRow(int rowId, CarbonColumnVector vector, int vectorRow) {
    -    if (!dictionary.isDictionaryUsed()) {
    -      vector.setDictionary(dictionary);
    -      dictionary.setDictionaryUsed();
    -    }
    +    // always set dictionary otherwise
    +    // empty dictionary will get set if same col is called again in projection.
    +    vector.setDictionary(dictionary);
    --- End diff --
    
    @BJangir 
    1. Please check and confirm if the same problem occurs with CarbonSession also
    2. Modify the PR description and specify the details for bug fixed in this PR after completion of point 1


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1895/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by BJangir <gi...@git.apache.org>.
Github user BJangir commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r243812352
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/memory/UnsafeMemoryManager.java ---
    @@ -173,6 +174,7 @@ public synchronized void freeMemoryAll(String taskId) {
               "Freeing offheap working memory of size %d. Current available memory is %d",
               occuppiedMemory, totalMemory - memoryUsed));
         }
    +    ThreadLocalTaskInfo.clearCarbonTaskInfo();
    --- End diff --
    
    OK..


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1900/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1782/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2042/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by BJangir <gi...@git.apache.org>.
Github user BJangir commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r243812841
  
    --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/util/CarbonVectorizedRecordReader.java ---
    @@ -68,6 +67,10 @@
       private AbstractDetailQueryResultIterator iterator;
     
       private QueryModel queryModel;
    +  //This holds mapping of  fetch index with respect to project col index.
    +  // it is used when same col is used in projection many times.So need to fetch only that col.
    +  private List<Integer> projectionMapping = new ArrayList<>();
    --- End diff --
    
    During initBatch only it is done(not frequent lookup/creation ).  


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2120/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1928/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2135/



---

[GitHub] carbondata issue #2897: [WIP] Supporting local dictionary enable by default ...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1489/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1834/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10040/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9546/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10054/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1995/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2110/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1925/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10178/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2104/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1909/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1905/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r243810009
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/memory/UnsafeMemoryManager.java ---
    @@ -173,6 +174,7 @@ public synchronized void freeMemoryAll(String taskId) {
               "Freeing offheap working memory of size %d. Current available memory is %d",
               occuppiedMemory, totalMemory - memoryUsed));
         }
    +    ThreadLocalTaskInfo.clearCarbonTaskInfo();
    --- End diff --
    
    Better call explicitly from methods where we call `freeMemoryAll, thread clearing should not be part of it.


---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2897


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1497/



---

[GitHub] carbondata pull request #2897: [CARBONDATA-3080] Supporting local dictionary...

Posted by BJangir <gi...@git.apache.org>.
Github user BJangir commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r242629291
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/LocalDictDimensionDataChunkStore.java ---
    @@ -94,10 +93,9 @@ public void fillVector(int[] invertedIndex, int[] invertedIndexReverse, byte[] d
       }
     
       @Override public void fillRow(int rowId, CarbonColumnVector vector, int vectorRow) {
    -    if (!dictionary.isDictionaryUsed()) {
    -      vector.setDictionary(dictionary);
    -      dictionary.setDictionaryUsed();
    -    }
    +    // always set dictionary otherwise
    +    // empty dictionary will get set if same col is called again in projection.
    +    vector.setDictionary(dictionary);
    --- End diff --
    
    1. For Session it is ok, issue happens only in SDK reader , Now hanlded in org.apache.carbondata.hadoop.util.CarbonVectorizedRecordReader.  LocalDictDimensionDataChunkStore.java reverted back.
    2. OK. Done


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1992/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2138/



---

[GitHub] carbondata pull request #2897: [WIP] Supporting local dictionary enable by d...

Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2897#discussion_r230729869
  
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
    @@ -64,7 +64,8 @@
       private Map<String, String> options;
       private String taskNo;
       private int localDictionaryThreshold;
    -  private boolean isLocalDictionaryEnabled;
    +  private boolean isLocalDictionaryEnabled = Boolean.parseBoolean(
    --- End diff --
    
    So what about the `localDictionaryThreshold`? To keep the same behavior, each configurations should be assigned the default value while init.


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Failed  with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10039/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1279/



---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    LGTM


---

[GitHub] carbondata issue #2897: [CARBONDATA-3080] Supporting local dictionary enable...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2897
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2006/



---