You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/28 15:33:29 UTC
[GitHub] [incubator-kyuubi] wForget opened a new pull request, #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
wForget opened a new pull request, #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #2490
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1112904149
Thanks @yaooqinn , I will add the test case and reimplement it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113154143
thanks, merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113186796
> maybe we can move the test to the trait
trait HiveEngineTests ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113182788
oh you forgot assume(SystemUtils.isJavaVersionAtMost(JavaVersion.JAVA_1_8))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113192158
Sounds good, I will do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113184408
> oh you forgot assume(SystemUtils.isJavaVersionAtMost(JavaVersion.JAVA_1_8))
Thanks, I didn't notice this, I'll add it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113189280
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113081131
The NPE happens when we execute the command statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
URL: https://github.com/apache/incubator-kyuubi/pull/2509
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#discussion_r861456936
##########
kyuubi-common/src/main/scala/org/apache/kyuubi/service/AbstractBackendService.scala:
##########
@@ -160,7 +160,18 @@ abstract class AbstractBackendService(name: String)
val operation = sessionManager.operationManager.getOperation(operationHandle)
if (operation.shouldRunAsync) {
try {
- operation.getBackgroundHandle.get(timeout, TimeUnit.MILLISECONDS)
Review Comment:
the fix here is wrong, because operation.getBackgroundHandle is null forever
```diff
+++ b/externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/operation/ExecuteStatement.scala
@@ -30,7 +30,7 @@ class ExecuteStatement(
session: Session,
override val statement: String,
confOverlay: Map[String, String],
- override val shouldRunAsync: Boolean,
+ runAsync: Boolean,
queryTimeout: Long)
extends HiveOperation(OperationType.EXECUTE_STATEMENT, session) {
@@ -39,9 +39,11 @@ class ExecuteStatement(
hive,
statement,
confOverlay.asJava,
- shouldRunAsync,
+ runAsync,
queryTimeout)
}
EventBus.post(HiveOperationEvent(this))
+
+ override def shouldRunAsync: Boolean = runAsync && getBackgroundHandle != null
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1112873444
we can add a simple unit test with parser error to verify this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1112410265
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2509](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (afaf240) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/beb132f96fe055f0349e4db1315aaba3d385acae?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (beb132f) will **increase** coverage by `0.02%`.
> The diff coverage is `66.66%`.
```diff
@@ Coverage Diff @@
## master #2509 +/- ##
============================================
+ Coverage 63.53% 63.55% +0.02%
Complexity 69 69
============================================
Files 371 371
Lines 17710 17718 +8
Branches 2358 2360 +2
============================================
+ Hits 11252 11261 +9
+ Misses 5411 5409 -2
- Partials 1047 1048 +1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...apache/kyuubi/service/AbstractBackendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2aWNlL0Fic3RyYWN0QmFja2VuZFNlcnZpY2Uuc2NhbGE=) | `94.20% <66.66%> (-4.16%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `96.51% <0.00%> (-0.12%)` | :arrow_down: |
| [.../engine/spark/session/SparkSQLSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU1FMU2Vzc2lvbk1hbmFnZXIuc2NhbGE=) | `80.95% <0.00%> (+1.19%)` | :arrow_up: |
| [...rg/apache/kyuubi/engine/trino/TrinoStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS10cmluby1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL3RyaW5vL1RyaW5vU3RhdGVtZW50LnNjYWxh) | `71.26% <0.00%> (+4.59%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [beb132f...afaf240](https://codecov.io/gh/apache/incubator-kyuubi/pull/2509?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113176419
@yaooqinn There seems to be a jdk version compatibility issue, doesn't it matter?
```
09:24:53.334 HiveTBinaryFrontendHandler-Pool: Thread-34 ERROR HiveTBinaryFrontendService: Error opening session:
org.apache.kyuubi.KyuubiSQLException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:73) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:110) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.kyuubi.service.TFrontendService.getSessionHandle(TFrontendService.scala:162) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.kyuubi.service.TFrontendService.OpenSession(TFrontendService.scala:171) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[libthrift-0.9.3.jar:0.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:413) ~[hive-exec-3.1.3.jar:3.1.3]
at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:166) ~[hive-service-3.1.3.jar:3.1.3]
at org.apache.kyuubi.engine.hive.session.HiveSessionImpl.open(HiveSessionImpl.scala:51) ~[classes/:?]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:97) ~[kyuubi-common_2.12-1.6.0-SNAPSHOT.jar:1.6.0-SNAPSHOT]
... 12 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2509: [KYUUBI #2490] Fix NPE in getOperationStatus
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2509:
URL: https://github.com/apache/incubator-kyuubi/pull/2509#issuecomment-1113185211
maybe we can move the test to the trait
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org