You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/19 11:55:17 UTC
[GitHub] [incubator-kyuubi] zhaomin1423 opened a new pull request, #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
zhaomin1423 opened a new pull request, #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
add engine stop event;
fix initialized event partition;
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860508011
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
In hive engine we can simplify to use this startTime, so any event can hold the same start time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on pull request #2421: [draft][KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#issuecomment-1103458218
https://github.com/apache/incubator-kyuubi/pull/2412/files
The engine.stop do't be used. The engine state is not changed to STOPPED.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you closed pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
URL: https://github.com/apache/incubator-kyuubi/pull/2421
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860485360
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
Spark engine post an initialized event, its start time is start time of sparkContext
https://github.com/apache/incubator-kyuubi/blob/93f13ef662f497a0e02281cf09cbf8cd0d8577fd/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala#L203
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#issuecomment-1112951530
thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r855700297
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/HiveSQLEngine.scala:
##########
@@ -100,6 +100,9 @@ object HiveSQLEngine extends Logging {
EventBus.post(event)
Utils.addShutdownHook(() => {
engine.getServices.foreach(_.stop())
+ val event = HiveEngineEvent(engine)
+ .copy(state = ServiceState.STOPPED, endTime = System.currentTimeMillis())
+ EventBus.post(event)
Review Comment:
it's unreliable to do some io work in shutdown hook, can we do this in `stopServer` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860496194
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
ok, we can make a global variable at HiveEngineEvent, something like:
```scala
object HiveEngineEvent {
val startTime = System.currentTimeMillis()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#issuecomment-1102617986
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2421](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3d48176) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/6017917100c59e444dd7a9f89845106cd8616735?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6017917) will **decrease** coverage by `0.01%`.
> The diff coverage is `50.00%`.
```diff
@@ Coverage Diff @@
## master #2421 +/- ##
============================================
- Coverage 62.90% 62.89% -0.02%
Complexity 69 69
============================================
Files 362 362
Lines 17293 17299 +6
Branches 2334 2334
============================================
+ Hits 10878 10880 +2
- Misses 5389 5396 +7
+ Partials 1026 1023 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/kyuubi/engine/hive/HiveSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvSGl2ZVNRTEVuZ2luZS5zY2FsYQ==) | `70.14% <14.28%> (-5.66%)` | :arrow_down: |
| [...e/kyuubi/engine/hive/events/HiveSessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvZXZlbnRzL0hpdmVTZXNzaW9uRXZlbnQuc2NhbGE=) | `94.73% <66.66%> (ø)` | |
| [...he/kyuubi/engine/hive/events/HiveEngineEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvZXZlbnRzL0hpdmVFbmdpbmVFdmVudC5zY2FsYQ==) | `89.65% <100.00%> (+0.36%)` | :arrow_up: |
| [...e/kyuubi/engine/hive/session/HiveSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvc2Vzc2lvbi9IaXZlU2Vzc2lvbkltcGwuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/trino/TrinoStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS10cmluby1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL3RyaW5vL1RyaW5vU3RhdGVtZW50LnNjYWxh) | `68.96% <0.00%> (-2.30%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `96.56% <0.00%> (+0.11%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `80.74% <0.00%> (+0.62%)` | :arrow_up: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `96.29% <0.00%> (+1.85%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [6017917...3d48176](https://codecov.io/gh/apache/incubator-kyuubi/pull/2421?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #2421: [draft][KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#issuecomment-1105015926
does this pr still be draft ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860475189
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
do we cancel post event before engine start?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860478408
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
I think we can. I looked at Spark engine, it also does not post before start
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860389390
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/HiveSQLEngine.scala:
##########
@@ -100,6 +100,9 @@ object HiveSQLEngine extends Logging {
EventBus.post(event)
Utils.addShutdownHook(() => {
engine.getServices.foreach(_.stop())
+ val event = HiveEngineEvent(engine)
+ .copy(state = ServiceState.STOPPED, endTime = System.currentTimeMillis())
+ EventBus.post(event)
Review Comment:
I mean, can we move this before exit system ?
```scala
override protected def stopServer(): Unit = {
// #2351
// https://issues.apache.org/jira/browse/HIVE-23164
// Server is not properly terminated because of non-daemon threads
System.exit(0)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r855763297
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
After the engine is initialized, an event is posted.
https://github.com/apache/incubator-kyuubi/blob/14f675d28841cb7e049af187fcd1a339e24ffc6d/externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/HiveSQLEngine.scala#L96
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r855824953
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/HiveSQLEngine.scala:
##########
@@ -100,6 +100,9 @@ object HiveSQLEngine extends Logging {
EventBus.post(event)
Utils.addShutdownHook(() => {
engine.getServices.foreach(_.stop())
+ val event = HiveEngineEvent(engine)
+ .copy(state = ServiceState.STOPPED, endTime = System.currentTimeMillis())
+ EventBus.post(event)
Review Comment:
It will be not be executed in stopServer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#issuecomment-1105180313
> does this pr still be draft ?
I have been updated it, please help me to review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860500810
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
The startTime has been existed in AbstractService, it is set in start method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860588168
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/HiveSQLEngine.scala:
##########
@@ -100,6 +100,9 @@ object HiveSQLEngine extends Logging {
EventBus.post(event)
Utils.addShutdownHook(() => {
engine.getServices.foreach(_.stop())
+ val event = HiveEngineEvent(engine)
+ .copy(state = ServiceState.STOPPED, endTime = System.currentTimeMillis())
+ EventBus.post(event)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860389676
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
so shall we also fix that code path ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r860588355
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
add a engineStartTime
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2421: [KYUUBI #2028][FOLLOWUP] add engine stop event and fix the partition of initialized event
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2421:
URL: https://github.com/apache/incubator-kyuubi/pull/2421#discussion_r855700835
##########
externals/kyuubi-hive-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/events/HiveEngineEvent.scala:
##########
@@ -30,8 +30,14 @@ case class HiveEngineEvent(
diagnostic: String,
settings: Map[String, String]) extends KyuubiEvent {
- override def partitions: Seq[(String, String)] =
- ("day", Utils.getDateFromTimestamp(startTime)) :: Nil
+ override def partitions: Seq[(String, String)] = {
+ // before engine is started, the start time is 0L, the partition use current day.
+ if (startTime == 0) {
Review Comment:
if a engine is not started how can we go into this code path , is it a bug ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org