You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/09/09 10:37:11 UTC
[GitHub] [incubator-kyuubi] ulysses-you opened a new issue #1070: [FEATURE] Support insert zorder by table properties
ulysses-you opened a new issue #1070:
URL: https://github.com/apache/incubator-kyuubi/issues/1070
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
Add zorder before writing dynamically.
This feature support:
- insert into datasource table
- create datasource table as select
- insert into hive table
- create hive table as select
### Motivation
Zorder is a popular approach of data skipping. We have supported `OPTIMIZE <table> [WHERE <predicate>] ZORDER BY col1, col2, ...` but it's not enough for user.
Let's say, If we have a hot table that adds a partition every day so we decide to apply zorder to optimize it. The table properties become to a good place to store the zorder info. When you realize that this is marked as a zorder table, we can optimize it.
### Describe the solution
Support parse two keys in table properties:
- kyuubi.zorder.enabled
decide if we should add zorder before writing this table
- kyuubi.zorder.cols
which columns should be included with zorder
And we also add a global config `spark.sql.optimizer.insertZorderBeforeWriting.enabled`.
### Additional context
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed issue #1070: [FEATURE] Support insert zorder by table properties
Posted by GitBox <gi...@apache.org>.
ulysses-you closed issue #1070:
URL: https://github.com/apache/incubator-kyuubi/issues/1070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org