You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/07/20 12:58:47 UTC
[GitHub] [incubator-kyuubi] liaoyt opened a new pull request, #3106: Correct `RelMetadataProvider` used in flink-sql-engine
liaoyt opened a new pull request, #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Currently flink-sql-engine can not run sql statement like followed, which is a normal case in flink sql.
```sql
select count(*) from tbl_src;
```
In Flink the default RelMetadataProvider is `org.apache.flink.table.planner.plan.metadata.FlinkDefaultRelMetadataProvider`, and in Kyuubi we use `org.apache.calcite.rel.metadata.DefaultRelMetadataProvider`.
We should change the RelMetadataProvider to the flink one to support all flink sql syntax.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1193548154
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] liaoyt commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
liaoyt commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1192434847
> You can run dev/reformat to format all Java and Scala code. There are some information that can help you. https://github.com/apache/incubator-kyuubi/blob/f1312ea439d288009200a105db36aa8431f27f8a/docs/develop_tools/developer.md
Thanks for the help, I will try it later.
I found that some unit test case failed with flink version 1.15 due to the difference between flink 1.14 and 1.15.
I'm thinking about whether I should fix it in this PR or open another issue and fix it in the future.
Any suggestions? @SteNicholas @zhaomin1423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1192485779
> I found that some unit test case failed with flink version 1.15 due to the difference between flink 1.14 and 1.15.
This is because Kyuubi engine uses vanilla `org.apache.calcite` classes, it's not a correct way, we need to use reflect to access the Flink runtime shaded calcite classes, and fallback to vanilla one(only for ut.)
> I'm thinking about whether I should fix it in this PR or open another issue and fix it in the future.
Either is fine. Choose the one you like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
pan3793 closed pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
URL: https://github.com/apache/incubator-kyuubi/pull/3106
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1191664491
You can run dev/reformat to format all Java and Scala code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1190960124
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3106](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (324ce15) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/b0685f9b48b038bea681ddd71266349f580ff730?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b0685f9) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #3106 +/- ##
============================================
- Coverage 51.33% 51.31% -0.02%
Complexity 6 6
============================================
Files 458 458
Lines 25389 25389
Branches 3536 3536
============================================
- Hits 13034 13029 -5
- Misses 11113 11115 +2
- Partials 1242 1245 +3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWN0bC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jdGwvY21kL2xvZy9Mb2dCYXRjaENvbW1hbmQuc2NhbGE=) | `80.00% <0.00%> (-2.00%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `80.82% <0.00%> (-1.37%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `68.91% <0.00%> (-1.36%)` | :arrow_down: |
| [...yuubi/server/metadata/jdbc/JDBCMetadataStore.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3106/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvbWV0YWRhdGEvamRiYy9KREJDTWV0YWRhdGFTdG9yZS5zY2FsYQ==) | `89.27% <0.00%> (-0.70%)` | :arrow_down: |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] liaoyt commented on pull request #3106: Correct `RelMetadataProvider` used in flink-sql-engine
Posted by GitBox <gi...@apache.org>.
liaoyt commented on PR #3106:
URL: https://github.com/apache/incubator-kyuubi/pull/3106#issuecomment-1190268461
@SteNicholas
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org