[GitHub] [carbondata] MarvinLitt commented on a change in pull request #4210: [CARBONDATA-4240]: Added missing properties on the configurations page

[GitBox <gi...@apache.org>]

MarvinLitt commented on a change in pull request #4210:
URL: https://github.com/apache/carbondata/pull/4210#discussion_r697783621



##########
File path: docs/configuration-parameters.md
##########
@@ -52,6 +52,18 @@ This section provides the details of all the configurations required for the Car
 | carbon.trash.retention.days | 7 | This parameter specifies the number of days after which the timestamp based subdirectories are expired in the trash folder. Allowed Min value = 0, Allowed Max Value = 365 days|
 | carbon.clean.file.force.allowed | false | This parameter specifies if the clean files operation with force option is allowed or not.|
 | carbon.cdc.minmax.pruning.enabled | false | This parameter defines whether the min max pruning to be performed on the target table based on the source data. It will be useful when data is not sparse across target table which results in better pruning.|
+| carbon.storelocation | (none) | This parameter defines the path on DFS where carbondata files and metadata will be stored. |

Review comment:
       carbon.storelocation is need to configure now? does it change to spark.sql.warehouse.dir ?
   @QiangCai @kumarvishal09 

##########
File path: docs/configuration-parameters.md
##########
@@ -52,6 +52,18 @@ This section provides the details of all the configurations required for the Car
 | carbon.trash.retention.days | 7 | This parameter specifies the number of days after which the timestamp based subdirectories are expired in the trash folder. Allowed Min value = 0, Allowed Max Value = 365 days|
 | carbon.clean.file.force.allowed | false | This parameter specifies if the clean files operation with force option is allowed or not.|
 | carbon.cdc.minmax.pruning.enabled | false | This parameter defines whether the min max pruning to be performed on the target table based on the source data. It will be useful when data is not sparse across target table which results in better pruning.|
+| carbon.storelocation | (none) | This parameter defines the path on DFS where carbondata files and metadata will be stored. |
+| carbon.blocklet.size | 64 MB | Carbondata files consist of blocklets which further consists of column pages. As per the latest V3 format, the default size of a blocklet is 64 MB. In V2 format, the default size of a blocklet was 120000 rows. |
+| carbon.properties.filepath | conf/carbon.properties | This file is by default present in conf directory on your base project path. Users can configure all the carbondata related properties in this file. |
+| carbon.date.format | yyyy-MM-dd | This property specifies the format in which data will be stored in the column with DATE data type. |
+| carbon.lock.class | (none) | This specifies the implementation of ICarbonLock interface to be used for acquiring the locks in case of concurrent operations |
+| carbon.local.dictionary.enable | (none) | If set to true, this property enables the generation of local dictionary. Local dictionary enables to map string and varchar values to numbers which helps in storing the data efficiently. |
+| carbon.local.dictionary.decoder.fallback | true | Page Level data will not be maintained for the blocklet. During fallback, actual data will be retrieved from the encoded page data using local dictionary. NOTE: Memory footprint decreases significantly as compared to when this property is set to false |
+| spark.deploy.zookeeper.url | (none) | The zookeeper url to connect to for using zookeeper based locking |
+| carbon.data.file.version | V3 | This specifies carbondata file format version. Carbondata file format has evolved with time from V1 to V3 in terms of metadata storage and IO level pruning capabilities. You can find more details [here](https://carbondata.apache.org/file-structure-of-carbondata.html#carbondata-file-format). |
+| spark.carbon.hive.schema.store | false | Carbondata currently supports 2 different types of metastores for storing schemas. This property specifies if Hive metastore is to be used for storing and retrieving table schemas |
+| spark.carbon.sessionstate.classname | `org.apache.spark.sql.hive.CarbonInMemorySessionStateBuilder` | This parameter determines the implementation of carbon session state to override sql parser and for adding strategies. Currently 2 different implementations are provided out of the box - one implements in memory session state and second implements Hive aware session state. |

Review comment:
       org.apache.spark.sql.hive.CarbonInMemorySessionStateBuilder is not a commonly used value, give a commonly used value




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org