You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/01/13 09:56:53 UTC
[GitHub] [incubator-kyuubi] pan3793 opened a new pull request #1756: Clean up code of FlinkOperation
pan3793 opened a new pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756
### _Why are the changes needed?_
Minor code cleanup.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] SteNicholas commented on a change in pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756#discussion_r784519925
##########
File path: externals/kyuubi-flink-sql-engine/src/main/scala/org/apache/kyuubi/engine/flink/operation/GetTables.scala
##########
@@ -38,49 +39,33 @@ class GetTables(
try {
val tableEnv = sessionContext.getExecutionContext.getTableEnvironment
- var catalogName = catalog
- if (catalog == null || catalog.isEmpty) {
- catalogName = tableEnv.getCurrentCatalog
- }
-
- val schemaPattern = toJavaRegex(schema).r.pattern
- val tableNamePattern = toJavaRegex(tableName).r.pattern
+ val catalogName = if (StringUtils.isEmpty(catalog)) tableEnv.getCurrentCatalog else catalog
- var tables = List[String]()
+ val schemaPattern = toJavaRegex(schema).r
+ val tableNamePattern = toJavaRegex(tableName).r
- val optional = tableEnv.getCatalog(catalogName)
- if (optional.isPresent) {
- val currCatalog = optional.get()
- tables = currCatalog.listDatabases().asScala
- .filter(database =>
- schemaPattern.matcher(database).matches())
- .flatMap { database =>
- currCatalog.listTables(database).asScala
- .filter(identifier =>
- tableNamePattern.matcher(identifier).matches())
- .filter(identifier => {
- // only table or view
- if (!tableTypes.contains(Constants.TABLE_TYPE) || !tableTypes.contains(
- Constants.VIEW_TYPE)) {
- // try to get table kind
- Try(currCatalog.getTable(ObjectIdentifier.of(
- catalogName,
- database,
- identifier).toObjectPath)) match {
- case Success(table) => tableTypes.contains(table.getTableKind.name())
+ var tables = tableEnv.getCatalog(catalogName).asScala.toSeq.flatMap { flinkCatalog =>
+ flinkCatalog.listDatabases().asScala
+ .filter { case schemaPattern(_*) => true; case _ => false }
Review comment:
This could use the `StringUtils.fliterPattern` in #1749 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
pan3793 closed pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
pan3793 commented on pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756#issuecomment-1013847356
Thanks, merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yanghua commented on a change in pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
yanghua commented on a change in pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756#discussion_r784471369
##########
File path: externals/kyuubi-flink-sql-engine/src/main/scala/org/apache/kyuubi/engine/flink/operation/GetTables.scala
##########
@@ -38,49 +39,33 @@ class GetTables(
try {
val tableEnv = sessionContext.getExecutionContext.getTableEnvironment
- var catalogName = catalog
- if (catalog == null || catalog.isEmpty) {
- catalogName = tableEnv.getCurrentCatalog
- }
-
- val schemaPattern = toJavaRegex(schema).r.pattern
- val tableNamePattern = toJavaRegex(tableName).r.pattern
+ val catalogName = if (StringUtils.isEmpty(catalog)) tableEnv.getCurrentCatalog else catalog
- var tables = List[String]()
+ val schemaPattern = toJavaRegex(schema).r
+ val tableNamePattern = toJavaRegex(tableName).r
- val optional = tableEnv.getCatalog(catalogName)
- if (optional.isPresent) {
- val currCatalog = optional.get()
- tables = currCatalog.listDatabases().asScala
- .filter(database =>
- schemaPattern.matcher(database).matches())
- .flatMap { database =>
- currCatalog.listTables(database).asScala
- .filter(identifier =>
- tableNamePattern.matcher(identifier).matches())
- .filter(identifier => {
- // only table or view
- if (!tableTypes.contains(Constants.TABLE_TYPE) || !tableTypes.contains(
- Constants.VIEW_TYPE)) {
- // try to get table kind
- Try(currCatalog.getTable(ObjectIdentifier.of(
- catalogName,
- database,
- identifier).toObjectPath)) match {
- case Success(table) => tableTypes.contains(table.getTableKind.name())
+ var tables = tableEnv.getCatalog(catalogName).asScala.toSeq.flatMap { flinkCatalog =>
+ flinkCatalog.listDatabases().asScala
+ .filter { case schemaPattern(_*) => true; case _ => false }
+ .flatMap { _schema =>
+ flinkCatalog.listTables(_schema).asScala
Review comment:
In flink, it uses `database` to represent `schema`, e.g. `listTables(database)`. But, on the other side, it's ok to follow general understood. I am ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yanghua closed pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
yanghua closed pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1756: Clean up code of FlinkOperation
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1756:
URL: https://github.com/apache/incubator-kyuubi/pull/1756#issuecomment-1013843747
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1756](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (319a985) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/783fc16d3a93004b9524ca7fc29219e91af763f9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (783fc16) will **decrease** coverage by `0.06%`.
> The diff coverage is `81.81%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1756 +/- ##
============================================
- Coverage 59.52% 59.45% -0.07%
Complexity 240 240
============================================
Files 273 273
Lines 13415 13403 -12
Branches 1695 1694 -1
============================================
- Hits 7985 7969 -16
- Misses 4746 4747 +1
- Partials 684 687 +3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...kyuubi/engine/flink/operation/FlinkOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vRmxpbmtPcGVyYXRpb24uc2NhbGE=) | `53.96% <0.00%> (-1.04%)` | :arrow_down: |
| [...ache/kyuubi/engine/flink/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `84.00% <87.50%> (-2.49%)` | :arrow_down: |
| [...he/kyuubi/engine/flink/operation/GetCatalogs.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vR2V0Q2F0YWxvZ3Muc2NhbGE=) | `83.33% <100.00%> (-4.17%)` | :arrow_down: |
| [.../kyuubi/engine/flink/operation/GetTableTypes.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vR2V0VGFibGVUeXBlcy5zY2FsYQ==) | `100.00% <100.00%> (ø)` | |
| [...he/kyuubi/engine/spark/repl/KyuubiSparkILoop.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9yZXBsL0t5dXViaVNwYXJrSUxvb3Auc2NhbGE=) | `90.16% <0.00%> (-3.28%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `92.50% <0.00%> (-2.50%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [783fc16...319a985](https://codecov.io/gh/apache/incubator-kyuubi/pull/1756?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org