You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "ulysses-you (via GitHub)" <gi...@apache.org> on 2023/02/14 10:20:44 UTC
[GitHub] [kyuubi] ulysses-you opened a new pull request, #4328: Fix trino client type signature parameter
ulysses-you opened a new pull request, #4328:
URL: https://github.com/apache/kyuubi/pull/4328
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/CONTRIBUTING.html
2. If the PR is related to an issue in https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
according to `io.trino.jdbc.ColumnInfo`, there are some type requring signature parameter.
- varchar(n)
- char(n)
- decimal(precision, scale)
It failed with trino jdbc now
<img width="613" alt="image" src="https://user-images.githubusercontent.com/12025282/218707052-a2e9dc91-0333-483c-bc0a-96baec213578.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] ulysses-you commented on a diff in pull request #4328: Make Trino jdbc driver work
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on code in PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#discussion_r1107952772
##########
kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/api/v1/dto/VersionInfo.java:
##########
@@ -26,10 +29,13 @@ public class VersionInfo {
public VersionInfo() {}
- public VersionInfo(String version) {
+ // Explicitly specifies JsonProperty to be compatible if disable auto detect feature
+ @JsonCreator
+ public VersionInfo(@JsonProperty("version") String version) {
Review Comment:
This is the only one dto which is used by trino and kyuubi. Since we disable auto detect feature for Trino, Explicitly specifies JsonProperty here to be compatible both of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] ulysses-you commented on pull request #4328: Make Trino jdbc driver work
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#issuecomment-1431054555
cc @Yikf @iodone @pan3793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] ulysses-you commented on a diff in pull request #4328: Make Trino jdbc driver work
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on code in PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#discussion_r1106984653
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/server/trino/api/TrinoScalaObjectMapper.scala:
##########
@@ -19,14 +19,54 @@ package org.apache.kyuubi.server.trino.api
import javax.ws.rs.ext.ContextResolver
-import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper}
+import com.fasterxml.jackson.core.JsonGenerator
+import com.fasterxml.jackson.databind.{DeserializationFeature, JsonSerializer, Module, ObjectMapper, SerializerProvider}
+import com.fasterxml.jackson.databind.module.SimpleModule
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module
+import io.trino.client.{ClientTypeSignature, ClientTypeSignatureParameter}
class TrinoScalaObjectMapper extends ContextResolver[ObjectMapper] {
private lazy val mapper = new ObjectMapper()
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
+ .registerModule(serializeTrinoClientTypeModule())
.registerModule(new Jdk8Module)
+ /**
+ * Not sure why jackson annotations do not work for some Trino dto, e.g. `ClientTypeSignature`.
+ * Without this change, it will throw exception during serializing when return back response.
+ * This is a workaround to be compatible with Trino json serialization.
+ * TODO: Remove this when find root cause
+ */
+ private def serializeTrinoClientTypeModule(): Module = {
+ val module = new SimpleModule("trinoSerializer")
Review Comment:
ah, good point. I have validated it works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] iodone commented on a diff in pull request #4328: Make Trino jdbc driver work
Posted by "iodone (via GitHub)" <gi...@apache.org>.
iodone commented on code in PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#discussion_r1106976787
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/server/trino/api/TrinoScalaObjectMapper.scala:
##########
@@ -19,14 +19,54 @@ package org.apache.kyuubi.server.trino.api
import javax.ws.rs.ext.ContextResolver
-import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper}
+import com.fasterxml.jackson.core.JsonGenerator
+import com.fasterxml.jackson.databind.{DeserializationFeature, JsonSerializer, Module, ObjectMapper, SerializerProvider}
+import com.fasterxml.jackson.databind.module.SimpleModule
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module
+import io.trino.client.{ClientTypeSignature, ClientTypeSignatureParameter}
class TrinoScalaObjectMapper extends ContextResolver[ObjectMapper] {
private lazy val mapper = new ObjectMapper()
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
+ .registerModule(serializeTrinoClientTypeModule())
.registerModule(new Jdk8Module)
+ /**
+ * Not sure why jackson annotations do not work for some Trino dto, e.g. `ClientTypeSignature`.
+ * Without this change, it will throw exception during serializing when return back response.
+ * This is a workaround to be compatible with Trino json serialization.
+ * TODO: Remove this when find root cause
+ */
+ private def serializeTrinoClientTypeModule(): Module = {
+ val module = new SimpleModule("trinoSerializer")
Review Comment:
Can we refer the implementaion of `QueryResults` codec in trino client ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] codecov-commenter commented on pull request #4328: Make Trino jdbc driver work
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#issuecomment-1432540665
# [Codecov](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#4328](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ae1a796) into [master](https://codecov.io/gh/apache/kyuubi/commit/763c088d8197b168894b3e5d84aefc248fd3273b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (763c088) will **decrease** coverage by `0.13%`.
> The diff coverage is `47.72%`.
```diff
@@ Coverage Diff @@
## master #4328 +/- ##
============================================
- Coverage 53.66% 53.53% -0.13%
Complexity 13 13
============================================
Files 562 562
Lines 30753 30775 +22
Branches 4161 4149 -12
============================================
- Hits 16503 16475 -28
- Misses 12693 12758 +65
+ Partials 1557 1542 -15
```
| [Impacted Files](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../apache/kyuubi/server/trino/api/TrinoContext.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvdHJpbm8vYXBpL1RyaW5vQ29udGV4dC5zY2FsYQ==) | `53.87% <39.47%> (-4.47%)` | :arrow_down: |
| [...g/apache/kyuubi/client/api/v1/dto/VersionInfo.java](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXJlc3QtY2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvY2xpZW50L2FwaS92MS9kdG8vVmVyc2lvbkluZm8uamF2YQ==) | `30.76% <100.00%> (-23.08%)` | :arrow_down: |
| [...uubi/server/trino/api/TrinoScalaObjectMapper.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvdHJpbm8vYXBpL1RyaW5vU2NhbGFPYmplY3RNYXBwZXIuc2NhbGE=) | `92.85% <100.00%> (+12.85%)` | :arrow_up: |
| [...kyuubi/server/trino/api/v1/StatementResource.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvdHJpbm8vYXBpL3YxL1N0YXRlbWVudFJlc291cmNlLnNjYWxh) | `50.66% <100.00%> (+0.66%)` | :arrow_up: |
| [.../org/apache/kyuubi/server/trino/api/v1/dto/Ok.java](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvdHJpbm8vYXBpL3YxL2R0by9Pay5qYXZh) | `46.15% <100.00%> (-23.08%)` | :arrow_down: |
| [...ain/scala/org/apache/kyuubi/util/RowSetUtils.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS91dGlsL1Jvd1NldFV0aWxzLnNjYWxh) | `26.08% <0.00%> (-57.25%)` | :arrow_down: |
| [...apache/kyuubi/engine/JpsApplicationOperation.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvSnBzQXBwbGljYXRpb25PcGVyYXRpb24uc2NhbGE=) | `77.41% <0.00%> (-3.23%)` | :arrow_down: |
| [.../apache/kyuubi/server/api/v1/BatchesResource.scala](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0JhdGNoZXNSZXNvdXJjZS5zY2FsYQ==) | `70.09% <0.00%> (-2.81%)` | :arrow_down: |
| ... and [12 more](https://codecov.io/gh/apache/kyuubi/pull/4328?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] ulysses-you commented on pull request #4328: Make Trino jdbc driver work
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on PR #4328:
URL: https://github.com/apache/kyuubi/pull/4328#issuecomment-1432813351
thanks for review, merging to master/1.7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] ulysses-you closed pull request #4328: Make Trino jdbc driver work
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you closed pull request #4328: Make Trino jdbc driver work
URL: https://github.com/apache/kyuubi/pull/4328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org