You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by qiuchenjian <gi...@git.apache.org> on 2018/09/02 13:41:34 UTC
[GitHub] carbondata pull request #2684: [CARBONDATA-2908]the option of sort_scope don...
GitHub user qiuchenjian opened a pull request:
https://github.com/apache/carbondata/pull/2684
[CARBONDATA-2908]the option of sort_scope don't effects while creatin…
…g table by data frame
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/qiuchenjian/carbondata master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2684.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2684
----
commit e29d4fd239d0f251c50bf9ebe71a6af0d2a91a26
Author: qiuchenjian <80...@...>
Date: 2018-09-02T13:39:18Z
[CARBONDATA-2908]the option of sort_scope don't effects while creating table by data frame
----
---
[GitHub] carbondata pull request #2684: [CARBONDATA-2908]the option of sort_scope don...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2684
---
[GitHub] carbondata pull request #2684: [CARBONDATA-2908]the option of sort_scope don...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2684#discussion_r214619170
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonDataFrameWriter.scala ---
@@ -81,6 +81,7 @@ class CarbonDataFrameWriter(sqlContext: SQLContext, val dataFrame: DataFrame) {
private def makeCreateTableString(schema: StructType, options: CarbonOption): String = {
val property = Map(
"SORT_COLUMNS" -> options.sortColumns,
+ "SORT_SCOPE" -> options.sortScope,
--- End diff --
From this PR, it seems that carbon will not output error message even if there is some table properties it can not recognized during create table by DataFrame API. I think it is better to add a check for it.
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/266/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8335/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/8/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8232/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by qiuchenjian <gi...@git.apache.org>.
Github user qiuchenjian commented on the issue:
https://github.com/apache/carbondata/pull/2684
retest this please
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/269/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/161/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by qiuchenjian <gi...@git.apache.org>.
Github user qiuchenjian commented on the issue:
https://github.com/apache/carbondata/pull/2684
@jackylk revert the change of thrift, only solve sortScope
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by qiuchenjian <gi...@git.apache.org>.
Github user qiuchenjian commented on the issue:
https://github.com/apache/carbondata/pull/2684
retest this please
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2684
LGTM
---
[GitHub] carbondata pull request #2684: [CARBONDATA-2908]the option of sort_scope don...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2684#discussion_r214627013
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonDataFrameWriter.scala ---
@@ -81,6 +81,7 @@ class CarbonDataFrameWriter(sqlContext: SQLContext, val dataFrame: DataFrame) {
private def makeCreateTableString(schema: StructType, options: CarbonOption): String = {
val property = Map(
"SORT_COLUMNS" -> options.sortColumns,
+ "SORT_SCOPE" -> options.sortScope,
--- End diff --
I think you can refer to CREATE TABLE section in http://carbondata.apache.org/data-management-on-carbondata.html
---
[GitHub] carbondata pull request #2684: [CARBONDATA-2908]the option of sort_scope don...
Posted by qiuchenjian <gi...@git.apache.org>.
Github user qiuchenjian commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2684#discussion_r214623582
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonDataFrameWriter.scala ---
@@ -81,6 +81,7 @@ class CarbonDataFrameWriter(sqlContext: SQLContext, val dataFrame: DataFrame) {
private def makeCreateTableString(schema: StructType, options: CarbonOption): String = {
val property = Map(
"SORT_COLUMNS" -> options.sortColumns,
+ "SORT_SCOPE" -> options.sortScope,
--- End diff --
I can't list all of the properties of table, any class record this ?
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2684
merged, thanks for working on this
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by chenliang613 <gi...@git.apache.org>.
Github user chenliang613 commented on the issue:
https://github.com/apache/carbondata/pull/2684
@qiuchenjian please update the pr's title, doesn't display completely.
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/97/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by qiuchenjian <gi...@git.apache.org>.
Github user qiuchenjian commented on the issue:
https://github.com/apache/carbondata/pull/2684
@jackylk properties of table can be add to TableProperty in schema.thrift
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/101/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8339/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8306/
---
[GitHub] carbondata issue #2684: [CARBONDATA-2908]the option of sort_scope don't effe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2684
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/236/
---