You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by jackylk <gi...@git.apache.org> on 2018/02/08 13:45:27 UTC

[GitHub] carbondata pull request #1808: [CARBONDATA-2023][DataLoad] Add size base blo...

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1808#discussion_r166936712
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonLoadOptionConstants.java ---
    @@ -114,4 +114,14 @@
        */
       public static final int MAX_EXTERNAL_DICTIONARY_SIZE = 10000000;
     
    +  /**
    +   * enable block size based block allocation while loading data. By default, carbondata assigns
    +   * blocks to node based on block number. If this option is set to `true`, carbondata will
    +   * consider block size first and make sure that all the nodes will process almost equal size of
    +   * data. This option is especially useful when you encounter skewed data.
    +   */
    +  @CarbonProperty
    +  public static final String ENABLE_CARBON_LOAD_SKEWED_DATA_OPTIMIZATION
    +      = "carbon.load.skewed.data.optimization";
    --- End diff --
    
    change to `carbon.load.skewedDataOptimization.enabled`


---