You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/03/27 03:07:30 UTC
[GitHub] [incubator-kyuubi] Nick-0723 opened a new pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Nick-0723 opened a new pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
support newly added data types: TimestampNTZType
since Spark3.3.0
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836009684
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/kyuubi/engine/spark/operation/SparkOperationSuite.scala
##########
@@ -76,6 +77,10 @@ class SparkOperationSuite extends WithSparkSQLEngine with HiveMetadataTests with
.add("c16", "binary", nullable = false, "16")
.add("c17", "struct<X: string>", nullable = true, "17")
+ if (SPARK_ENGINE_MAJOR_MINOR_VERSION._1 >= 3 && SPARK_ENGINE_MAJOR_MINOR_VERSION._2 > 2) {
Review comment:
this does not work for 4.x, shall be (x>3 || (x==3 && y >=3)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#issuecomment-1081709959
thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#issuecomment-1079833046
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2224](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (56dfb2d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/86cd685de784f6ade61ea78502aae561cdb48a7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86cd685) will **increase** coverage by `0.09%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #2224 +/- ##
============================================
+ Coverage 61.80% 61.89% +0.09%
Complexity 69 69
============================================
Files 331 331
Lines 16016 16055 +39
Branches 2031 2041 +10
============================================
+ Hits 9899 9938 +39
+ Misses 5295 5283 -12
- Partials 822 834 +12
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/engine/spark/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvUm93U2V0LnNjYWxh) | `91.13% <0.00%> (-1.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/schema/SchemaHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvU2NoZW1hSGVscGVyLnNjYWxh) | `90.10% <0.00%> (-4.15%)` | :arrow_down: |
| [.../credentials/HadoopFsDelegationTokenProvider.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jcmVkZW50aWFscy9IYWRvb3BGc0RlbGVnYXRpb25Ub2tlblByb3ZpZGVyLnNjYWxh) | `95.00% <0.00%> (-2.30%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `82.35% <0.00%> (-1.78%)` | :arrow_down: |
| [...cala/org/apache/kyuubi/ctl/ServiceControlCli.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWN0bC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jdGwvU2VydmljZUNvbnRyb2xDbGkuc2NhbGE=) | `77.57% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `96.16% <0.00%> (+0.02%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `85.59% <0.00%> (+0.24%)` | :arrow_up: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `96.07% <0.00%> (+1.96%)` | :arrow_up: |
| [.../org/apache/kyuubi/engine/hive/HiveSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvSGl2ZVNRTEVuZ2luZS5zY2FsYQ==) | `77.77% <0.00%> (+2.77%)` | :arrow_up: |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `75.20% <0.00%> (+10.71%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86cd685...56dfb2d](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] Nick-0723 commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
Nick-0723 commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836403122
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/kyuubi/engine/spark/operation/SparkOperationSuite.scala
##########
@@ -76,6 +77,10 @@ class SparkOperationSuite extends WithSparkSQLEngine with HiveMetadataTests with
.add("c16", "binary", nullable = false, "16")
.add("c17", "struct<X: string>", nullable = true, "17")
+ if (SPARK_ENGINE_MAJOR_MINOR_VERSION._1 >= 3 && SPARK_ENGINE_MAJOR_MINOR_VERSION._2 > 2) {
Review comment:
oops，I get you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] Nick-0723 commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
Nick-0723 commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836074660
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/kyuubi/engine/spark/operation/SparkOperationSuite.scala
##########
@@ -76,6 +77,10 @@ class SparkOperationSuite extends WithSparkSQLEngine with HiveMetadataTests with
.add("c16", "binary", nullable = false, "16")
.add("c17", "struct<X: string>", nullable = true, "17")
+ if (SPARK_ENGINE_MAJOR_MINOR_VERSION._1 >= 3 && SPARK_ENGINE_MAJOR_MINOR_VERSION._2 > 2) {
Review comment:
>
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] Nick-0723 commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
Nick-0723 commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836072847
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/schema/SchemaHelper.scala
##########
@@ -39,6 +44,7 @@ object SchemaHelper {
case _: DecimalType => TTypeId.DECIMAL_TYPE
case DateType => TTypeId.DATE_TYPE
case TimestampType => TTypeId.TIMESTAMP_TYPE
+ case ntz if ntz.getClass.getName.equals(TIMESTAMP_NTZ) => TTypeId.TIMESTAMP_TYPE
Review comment:
> why not use simple class name as intervals？
I found intervals did not handle the getColumnSize ,and I'll fix it in another pr。 So I would replace the string with constant variables ，but maybe simple class name is a good choice。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836008938
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/schema/SchemaHelper.scala
##########
@@ -39,6 +44,7 @@ object SchemaHelper {
case _: DecimalType => TTypeId.DECIMAL_TYPE
case DateType => TTypeId.DATE_TYPE
case TimestampType => TTypeId.TIMESTAMP_TYPE
+ case ntz if ntz.getClass.getName.equals(TIMESTAMP_NTZ) => TTypeId.TIMESTAMP_TYPE
Review comment:
why not use simple class name as intervals？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#issuecomment-1079833046
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2224](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54ef78d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/86cd685de784f6ade61ea78502aae561cdb48a7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86cd685) will **decrease** coverage by `0.02%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #2224 +/- ##
============================================
- Coverage 61.79% 61.77% -0.03%
Complexity 69 69
============================================
Files 331 331
Lines 16016 16022 +6
Branches 2031 2036 +5
============================================
Hits 9897 9897
- Misses 5297 5298 +1
- Partials 822 827 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/engine/spark/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvUm93U2V0LnNjYWxh) | `91.13% <0.00%> (-1.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/schema/SchemaHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvU2NoZW1hSGVscGVyLnNjYWxh) | `90.10% <0.00%> (-4.15%)` | :arrow_down: |
| [...org/apache/kyuubi/ha/client/ServiceDiscovery.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9TZXJ2aWNlRGlzY292ZXJ5LnNjYWxh) | `89.28% <0.00%> (-3.58%)` | :arrow_down: |
| [...rg/apache/kyuubi/engine/trino/TrinoStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS10cmluby1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL3RyaW5vL1RyaW5vU3RhdGVtZW50LnNjYWxh) | `69.87% <0.00%> (+2.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86cd685...54ef78d](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#issuecomment-1079833046
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2224](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54ef78d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/86cd685de784f6ade61ea78502aae561cdb48a7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86cd685) will **decrease** coverage by `0.03%`.
> The diff coverage is `0.00%`.
> :exclamation: Current head 54ef78d differs from pull request most recent head 10b12f3. Consider uploading reports for the commit 10b12f3 to get more accurate results
```diff
@@ Coverage Diff @@
## master #2224 +/- ##
============================================
- Coverage 61.80% 61.77% -0.04%
Complexity 69 69
============================================
Files 331 331
Lines 16016 16022 +6
Branches 2031 2036 +5
============================================
- Hits 9899 9897 -2
- Misses 5295 5298 +3
- Partials 822 827 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/engine/spark/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvUm93U2V0LnNjYWxh) | `91.13% <0.00%> (-1.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/schema/SchemaHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvU2NoZW1hSGVscGVyLnNjYWxh) | `90.10% <0.00%> (-4.15%)` | :arrow_down: |
| [...org/apache/kyuubi/ha/client/ServiceDiscovery.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9TZXJ2aWNlRGlzY292ZXJ5LnNjYWxh) | `89.28% <0.00%> (-3.58%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86cd685...10b12f3](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#issuecomment-1079833046
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2224](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54ef78d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/86cd685de784f6ade61ea78502aae561cdb48a7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86cd685) will **decrease** coverage by `0.03%`.
> The diff coverage is `0.00%`.
> :exclamation: Current head 54ef78d differs from pull request most recent head 56dfb2d. Consider uploading reports for the commit 56dfb2d to get more accurate results
```diff
@@ Coverage Diff @@
## master #2224 +/- ##
============================================
- Coverage 61.80% 61.77% -0.04%
Complexity 69 69
============================================
Files 331 331
Lines 16016 16022 +6
Branches 2031 2036 +5
============================================
- Hits 9899 9897 -2
- Misses 5295 5298 +3
- Partials 822 827 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/engine/spark/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvUm93U2V0LnNjYWxh) | `91.13% <0.00%> (-1.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/schema/SchemaHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zY2hlbWEvU2NoZW1hSGVscGVyLnNjYWxh) | `90.10% <0.00%> (-4.15%)` | :arrow_down: |
| [...org/apache/kyuubi/ha/client/ServiceDiscovery.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9TZXJ2aWNlRGlzY292ZXJ5LnNjYWxh) | `89.28% <0.00%> (-3.58%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86cd685...56dfb2d](https://codecov.io/gh/apache/incubator-kyuubi/pull/2224?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] Nick-0723 commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
Nick-0723 commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836394719
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/kyuubi/engine/spark/operation/SparkOperationSuite.scala
##########
@@ -76,6 +77,10 @@ class SparkOperationSuite extends WithSparkSQLEngine with HiveMetadataTests with
.add("c16", "binary", nullable = false, "16")
.add("c17", "struct<X: string>", nullable = true, "17")
+ if (SPARK_ENGINE_MAJOR_MINOR_VERSION._1 >= 3 && SPARK_ENGINE_MAJOR_MINOR_VERSION._2 > 2) {
Review comment:
Isn't is this : (major == 3 && minor >= 3)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] Nick-0723 commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
Nick-0723 commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836072712
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/schema/SchemaHelper.scala
##########
@@ -39,6 +44,7 @@ object SchemaHelper {
case _: DecimalType => TTypeId.DECIMAL_TYPE
case DateType => TTypeId.DATE_TYPE
case TimestampType => TTypeId.TIMESTAMP_TYPE
+ case ntz if ntz.getClass.getName.equals(TIMESTAMP_NTZ) => TTypeId.TIMESTAMP_TYPE
Review comment:
I found intervals did not handle the getColumnSize ,and I'll fix it in another pr。 So I would replace the string with constant variables ，but maybe simple class name is a good choice。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224#discussion_r836390783
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/schema/SchemaHelper.scala
##########
@@ -39,6 +44,7 @@ object SchemaHelper {
case _: DecimalType => TTypeId.DECIMAL_TYPE
case DateType => TTypeId.DATE_TYPE
case TimestampType => TTypeId.TIMESTAMP_TYPE
+ case ntz if ntz.getClass.getName.equals(TIMESTAMP_NTZ) => TTypeId.TIMESTAMP_TYPE
Review comment:
+1 for simple class name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed pull request #2224: [KYUUBI #2207]Support newly added spark data types: TimestampNTZType
Posted by GitBox <gi...@apache.org>.
ulysses-you closed pull request #2224:
URL: https://github.com/apache/incubator-kyuubi/pull/2224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org