You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/08/20 07:56:20 UTC

[GitHub] [incubator-kyuubi] hddong opened a new pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

hddong opened a new pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968


   <!--
   Thanks for sending a pull request!
   
   Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
     2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
   -->
   
   ### _Why are the changes needed?_
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you add a feature, you can talk about the use case of it.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   Track operation state change and support output to custom system.
   
   ### _How was this patch tested?_
   - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
   
   - [ ] Add screenshots for manual tests if appropriate
   
   - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-905221246


   thank you @hddong , master to master for 1.4.0.
   
   Can you send a new PR for branch-1.3.0 only to fix the typo issue ?
   `var exeception: String = "") extends KyuubiEvent {` -> `var exception: String = "") extends KyuubiEvent {`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you edited a comment on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you edited a comment on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-905221246


   thank you @hddong , merging to master for 1.4.0. FYI @yaooqinn @pan3793 
   
   Can you send a new PR for branch-1.3.0 only to fix the typo issue ?
   `var exeception: String = "") extends KyuubiEvent {` -> `var exception: String = "") extends KyuubiEvent {`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-904475749


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#968](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a6ef14) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/4912542bdacc941dc1d72bcbeb6065d3d0b8d4ac?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4912542) will **decrease** coverage by `0.61%`.
   > The diff coverage is `82.00%`.
   
   > :exclamation: Current head 1a6ef14 differs from pull request most recent head c75b768. Consider uploading reports for the commit c75b768 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master     #968      +/-   ##
   ============================================
   - Coverage     79.81%   79.20%   -0.62%     
     Complexity       11       11              
   ============================================
     Files           147      150       +3     
     Lines          5454     5529      +75     
     Branches        654      650       -4     
   ============================================
   + Hits           4353     4379      +26     
   - Misses          720      774      +54     
   + Partials        381      376       -5     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <ø> (ø)` | |
   | [...org/apache/kyuubi/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...ala/org/apache/kyuubi/events/JsonEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvbkV2ZW50TG9nZ2VyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/kyuubi/events/JsonProtocol.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvblByb3RvY29sLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...n/scala/org/apache/kyuubi/events/KyuubiEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpRXZlbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
   | [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `85.00% <72.72%> (-2.50%)` | :arrow_down: |
   | [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `88.88% <88.88%> (ø)` | |
   | [...cala/org/apache/kyuubi/events/StatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `95.65% <95.65%> (ø)` | |
   | [...pache/kyuubi/engine/spark/events/EngineEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRW5naW5lRXZlbnQuc2NhbGE=) | `88.13% <100.00%> (ø)` | |
   | [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `83.33% <100.00%> (ø)` | |
   | ... and [15 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4912542...c75b768](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r692814914



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -682,13 +682,36 @@ object KyuubiConf {
       .booleanConf
       .createWithDefault(false)
 
+  val SERVER_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+    buildConf("backend.server.event.json.log.path")
+      .doc("The location of server events go for the builtin JSON logger")
+      .version("1.4.0")
+      .stringConf
+      .createWithDefault("/tmp/kyuubi/events")
+
   val ENGINE_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
     buildConf("engine.event.json.log.path")
-      .doc("The location of all the engine events go for the builtin JSON logger")
+      .doc("The location of all the engine events go for the builtin JSON logger.")
       .version("1.3.0")
       .stringConf
       .createWithDefault("/tmp/kyuubi/events")
 
+  val SERVER_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
+    buildConf("backend.server.event.loggers")
+      .doc("A comma separated list of server history loggers, where session/operation etc" +
+        " events go.<ul>" +
+        s" <li>JSON: the events will be written to the location of" +
+        s" ${SERVER_EVENT_JSON_LOG_PATH.key}</li>" +
+        s" <li>JDBC: to be done</li>" +
+        s" <li>CUSTOM: to be done.</li></ul>")
+      .version("1.4.0")
+      .stringConf
+      .transform(_.toUpperCase(Locale.ROOT))
+      .toSequence()
+      .checkValue(_.toSet.subsetOf(Set("JSON", "JDBC", "CUSTOM")),
+        "Unsupported event loggers")
+      .createWithDefault(Nil)
+
   val ENGINE_EVENT_LOGGERS: ConfigEntry[Seq[String]] =

Review comment:
       Do we have time to rename those configs to `kyuubi.event.[server|engine].xxx` before 1.3.0? cc @pan3793 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r698204120



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/KyuubiEvent.scala
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import java.util.Locale
+
+trait KyuubiEvent extends Product {
+  final lazy val eventType: String = {
+    this.getClass.getSimpleName.stripSuffix("Event")
+      .replaceAll("(.)([A-Z])", "$1-$2")

Review comment:
       > can we use `"$1_$2"`? `-` is unfriendly to sql
   
   Yep.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] hddong commented on pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-902562499


   Yeah, the mainly changes:
   1. Move `KyuubiEvent` to `kyuubi-common` and add two sub trait (`KyuubiServerEvent`, `KyuubiSparkEvent`) . 
     `SPARK` event logger is only supported in `KyuubiSparkEvent`.
   2. Move `EventLogger` and `JsonEventLogger` to `kyuubi-common`
   3. Create `AbstractEventLoggingService`, `kyuubi-server` and `kyuubi-engine` each has a subclass `EventLoggingService` to get their own event loggers and catch events.
   4. Add `OperationEvent` for server


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r693626770



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/EventLoggingService.scala
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.kyuubi.config.KyuubiConf
+import org.apache.kyuubi.events.EventLoggerType.EventLoggerType
+import org.apache.kyuubi.service.CompositeService
+
+abstract class AbstractEventLoggingService[T <: KyuubiEvent]
+  extends CompositeService("EventLogging") {
+
+  private val eventLoggers = new ArrayBuffer[EventLogger[T]]()
+
+  /**
+   * Get configured event log type.
+   */
+  protected def getLoggers(conf: KyuubiConf): Seq[EventLoggerType]
+
+  def onEvent(event: T): Unit = {
+    eventLoggers.foreach(_.logEvent(event))
+  }
+
+  override def initialize(conf: KyuubiConf): Unit = {
+    getLoggers(conf).foreach(analyseEventLoggerType)

Review comment:
       It seems overkill to add two methods here, how about override `initialize` in implementation ?

##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/StatementEvent.scala
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.operation.ExecuteStatement
+import org.apache.kyuubi.operation.OperationState.OperationState
+
+/**
+ *
+ * @param user: who connect to kyuubi server
+ * @param statementId: the identifier of operationHandler
+ * @param statement: the sql that you execute
+ * @param remoteIp: the ip of user
+ * @param sessionId: the identifier of a session
+ * @param createTime: the create time of this statement
+ * @param state: store each state that the sql has
+ * @param stateTime: the time that the sql's state change
+ * @param exception: caught exception if have
+ */
+case class StatementEvent(

Review comment:
       This enigne type is conflict with engine's `StatementEvent` if we write into same path.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r692759786



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/events/StatementEvent.scala
##########
@@ -42,7 +42,7 @@ case class StatementEvent(
     var state: String,
     var stateTime: Long,
     var queryExecution: String = "",
-    var exeception: String = "") extends KyuubiEvent {
+    var exception: String = "") extends KyuubiSparkEvent {

Review comment:
       nice catch




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you edited a comment on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you edited a comment on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-905221246


   thank you @hddong , meging to master for 1.4.0. FYI @yaooqinn @pan3793 
   
   Can you send a new PR for branch-1.3.0 only to fix the typo issue ?
   `var exeception: String = "") extends KyuubiEvent {` -> `var exception: String = "") extends KyuubiEvent {`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r694690069



##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/StatementEvent.scala
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.operation.ExecuteStatement
+import org.apache.kyuubi.operation.OperationState.OperationState
+
+/**
+ *
+ * @param user: who connect to kyuubi server
+ * @param statementId: the identifier of operationHandler
+ * @param statement: the sql that you execute
+ * @param remoteIp: the ip of user
+ * @param sessionId: the identifier of a session
+ * @param createTime: the create time of this statement
+ * @param state: store each state that the sql has
+ * @param stateTime: the time that the sql's state change
+ * @param exception: caught exception if have
+ */
+case class StatementEvent(

Review comment:
       Since they have different schema, if we want to create table on that path, how can we read them together ? I think we should use different event type for this case so that we can create tables for them.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r695416500



##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/StatementEvent.scala
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.operation.ExecuteStatement
+import org.apache.kyuubi.operation.OperationState.OperationState
+
+/**
+ *
+ * @param user: who connect to kyuubi server
+ * @param statementId: the identifier of operationHandler
+ * @param statement: the sql that you execute
+ * @param remoteIp: the ip of user
+ * @param sessionId: the identifier of a session
+ * @param createTime: the create time of this statement
+ * @param state: store each state that the sql has
+ * @param stateTime: the time that the sql's state change
+ * @param exception: caught exception if have
+ */
+case class StatementEvent(

Review comment:
       > Since ... I think we should use different event type for this case so that we can create tables for them.
   
   Added prifix for StatementEvents.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-904475749


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#968](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8be8f26) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/3ebb50f1cb9121217900987c02727912c9325a48?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3ebb50f) will **decrease** coverage by `0.67%`.
   > The diff coverage is `81.55%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master     #968      +/-   ##
   ============================================
   - Coverage     80.72%   80.05%   -0.68%     
     Complexity       11       11              
   ============================================
     Files           148      151       +3     
     Lines          5406     5484      +78     
     Branches        638      639       +1     
   ============================================
   + Hits           4364     4390      +26     
   - Misses          663      715      +52     
     Partials        379      379              
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...he/kyuubi/events/AbstractEventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvQWJzdHJhY3RFdmVudExvZ2dpbmdTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <ø> (ø)` | |
   | [...ala/org/apache/kyuubi/events/JsonEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvbkV2ZW50TG9nZ2VyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/kyuubi/events/JsonProtocol.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvblByb3RvY29sLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...n/scala/org/apache/kyuubi/events/KyuubiEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpRXZlbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
   | [...uubi/engine/spark/events/SparkStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtTdGF0ZW1lbnRFdmVudC5zY2FsYQ==) | `83.33% <33.33%> (ø)` | |
   | [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `90.00% <90.00%> (ø)` | |
   | [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `95.65% <95.65%> (ø)` | |
   | [...pache/kyuubi/engine/spark/events/EngineEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRW5naW5lRXZlbnQuc2NhbGE=) | `88.13% <100.00%> (ø)` | |
   | [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `86.36% <100.00%> (-1.14%)` | :arrow_down: |
   | ... and [15 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3ebb50f...8be8f26](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r694683190



##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/StatementEvent.scala
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.operation.ExecuteStatement
+import org.apache.kyuubi.operation.OperationState.OperationState
+
+/**
+ *
+ * @param user: who connect to kyuubi server
+ * @param statementId: the identifier of operationHandler
+ * @param statement: the sql that you execute
+ * @param remoteIp: the ip of user
+ * @param sessionId: the identifier of a session
+ * @param createTime: the create time of this statement
+ * @param state: store each state that the sql has
+ * @param stateTime: the time that the sql's state change
+ * @param exception: caught exception if have
+ */
+case class StatementEvent(

Review comment:
       > This event type is conflict with engine's `StatementEvent` if we write into same path.
   
   `StatementEvent` is a path too, they have differnet file name under `statement`. 
   And add a check for this case. 
   ```
   ll /var/folders/xl/jv1jvw6s6hv42ht1ty6gnk100000gn/T/kyuubi-bb4a5f34-451c-4783-9f50-1906889386cf/statement/day=20210824/
   -rw-rw----  1 *  staff  9472 Aug 24 16:27 local-1629793620908.json
   -rw-rw----  1 *  staff  3588 Aug 24 16:27 server-172.20.144.37.json
   ```

##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/EventLoggingService.scala
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.kyuubi.config.KyuubiConf
+import org.apache.kyuubi.events.EventLoggerType.EventLoggerType
+import org.apache.kyuubi.service.CompositeService
+
+abstract class AbstractEventLoggingService[T <: KyuubiEvent]
+  extends CompositeService("EventLogging") {
+
+  private val eventLoggers = new ArrayBuffer[EventLogger[T]]()
+
+  /**
+   * Get configured event log type.
+   */
+  protected def getLoggers(conf: KyuubiConf): Seq[EventLoggerType]
+
+  def onEvent(event: T): Unit = {
+    eventLoggers.foreach(_.logEvent(event))
+  }
+
+  override def initialize(conf: KyuubiConf): Unit = {
+    getLoggers(conf).foreach(analyseEventLoggerType)

Review comment:
       yeah, change to override `initialize`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r698134009



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/KyuubiEvent.scala
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import java.util.Locale
+
+trait KyuubiEvent extends Product {
+  final lazy val eventType: String = {
+    this.getClass.getSimpleName.stripSuffix("Event")
+      .replaceAll("(.)([A-Z])", "$1-$2")

Review comment:
       can we use `"$1_$2"`? `-` is unfriendly to sql




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r692760010



##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/kyuubi/engine/spark/events/EventLoggingServiceSuite.scala
##########
@@ -26,10 +26,11 @@ import org.scalatest.time.SpanSugar._
 import org.apache.kyuubi.Utils
 import org.apache.kyuubi.config.KyuubiConf
 import org.apache.kyuubi.engine.spark.{KyuubiSparkUtil, WithSparkSQLEngine}
+import org.apache.kyuubi.events.JsonProtocol
 import org.apache.kyuubi.operation.{JDBCTestUtils, OperationHandle}
 
 class EventLoggingServiceSuite extends WithSparkSQLEngine with JDBCTestUtils {
-  import EventLoggerType._
+  import org.apache.kyuubi.events.EventLoggerType._

Review comment:
       move up




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#discussion_r693625703



##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/StatementEvent.scala
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.operation.ExecuteStatement
+import org.apache.kyuubi.operation.OperationState.OperationState
+
+/**
+ *
+ * @param user: who connect to kyuubi server
+ * @param statementId: the identifier of operationHandler
+ * @param statement: the sql that you execute
+ * @param remoteIp: the ip of user
+ * @param sessionId: the identifier of a session
+ * @param createTime: the create time of this statement
+ * @param state: store each state that the sql has
+ * @param stateTime: the time that the sql's state change
+ * @param exception: caught exception if have
+ */
+case class StatementEvent(

Review comment:
       This event type is conflict with engine's `StatementEvent` if we write into same path.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] hddong commented on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-905223056


   @ulysses-you : Sure, I will.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-904475749


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#968](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c75b768) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/3ebb50f1cb9121217900987c02727912c9325a48?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3ebb50f) will **decrease** coverage by `0.61%`.
   > The diff coverage is `86.45%`.
   
   > :exclamation: Current head c75b768 differs from pull request most recent head 8be8f26. Consider uploading reports for the commit 8be8f26 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master     #968      +/-   ##
   ============================================
   - Coverage     80.72%   80.10%   -0.62%     
     Complexity       11       11              
   ============================================
     Files           148      149       +1     
     Lines          5406     5465      +59     
     Branches        638      637       -1     
   ============================================
   + Hits           4364     4378      +14     
   - Misses          663      714      +51     
   + Partials        379      373       -6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...he/kyuubi/events/AbstractEventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvQWJzdHJhY3RFdmVudExvZ2dpbmdTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <ø> (ø)` | |
   | [...ala/org/apache/kyuubi/events/JsonEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvbkV2ZW50TG9nZ2VyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/kyuubi/events/JsonProtocol.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvblByb3RvY29sLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...n/scala/org/apache/kyuubi/events/KyuubiEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpRXZlbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `90.00% <90.00%> (ø)` | |
   | [...cala/org/apache/kyuubi/events/StatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `95.65% <95.65%> (ø)` | |
   | [...pache/kyuubi/engine/spark/events/EngineEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRW5naW5lRXZlbnQuc2NhbGE=) | `88.13% <100.00%> (ø)` | |
   | [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `86.36% <100.00%> (-1.14%)` | :arrow_down: |
   | [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `83.33% <100.00%> (ø)` | |
   | ... and [19 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/968/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3ebb50f...8be8f26](https://codecov.io/gh/apache/incubator-kyuubi/pull/968?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you closed pull request #968: [KYUUBI #922] Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
ulysses-you closed pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #968: [KYUUBI#922]Add OperationEvent for server to track operation state

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #968:
URL: https://github.com/apache/incubator-kyuubi/pull/968#issuecomment-902512559


   can you detail the changes in your PR to help us review?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org