You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by GitBox <gi...@apache.org> on 2022/02/28 10:01:14 UTC
[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4250: [WIP] Update Data frame supported options in document and fix partition table creation with df spatial property
ShreelekhyaG opened a new pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250
### Why is this PR needed?
1. Only specific properties are supported using dataframe options. Need to update the documentation.
2. Create partition table fails with Spatial index property for carbon table created with dataframe in spark-shell.
### What changes were proposed in this PR?
1. Added data frame supported properties in the documentation.
2. Using spark-shell, the table gets created with carbon session and `catalogTable.properties` is empty here. Getting the properties from `catalogTable.storage.properties` to access the properties set.
### Does this PR introduce any user interface change?
- No
### Is any new testcase added?
- No, tested in cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057095550
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6223/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057678048
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057776159
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/615/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057943149
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/617/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057917071
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4482/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [WIP] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1054204913
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4477/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [WIP] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1054203863
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6222/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057122241
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/613/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057094426
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4478/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057976856
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058823503
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058097506
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4483/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#discussion_r817405056
##########
File path: docs/carbon-as-spark-datasource-guide.md
##########
@@ -96,6 +96,30 @@ df.write.format("carbon").save("/user/person_table")
val dfread = spark.read.format("carbon").load("/user/person_table")
dfread.show()
```
+## Supported OPTIONS
+
+| Property | Default Value | Description |
+|-----------------------------------|-------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| table_blocksize | 1024 | Size of blocks to write onto hdfs. For more details, see [Table Block Size Configuration](./ddl-of-carbondata.md#table-block-size-configuration). |
Review comment:
i think, since already there is one table above in this md file with same contents, you can add a link for those properties and add only the remaining properties in this table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058129026
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/618/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058300487
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4485/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057751945
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6225/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057915999
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6227/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058877136
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6231/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [WIP] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1054229814
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/612/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058905263
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/621/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058134786
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058308385
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6230/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058900981
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4486/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on a change in pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#discussion_r818315378
##########
File path: docs/carbon-as-spark-datasource-guide.md
##########
@@ -96,6 +96,30 @@ df.write.format("carbon").save("/user/person_table")
val dfread = spark.read.format("carbon").load("/user/person_table")
dfread.show()
```
+## Supported OPTIONS
+
+| Property | Default Value | Description |
+|-----------------------------------|-------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| table_blocksize | 1024 | Size of blocks to write onto hdfs. For more details, see [Table Block Size Configuration](./ddl-of-carbondata.md#table-block-size-configuration). |
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057791999
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058324697
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/620/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057672074
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1057756730
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4480/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4250: [CARBONDATA-4325] Update Data frame supported options in document and fix partition table creation with df spatial property
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4250:
URL: https://github.com/apache/carbondata/pull/4250#issuecomment-1058003237
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6228/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org