You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/05/17 01:19:27 UTC

[GitHub] [incubator-kyuubi] turboFei opened a new pull request, #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

turboFei opened a new pull request, #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669

   <!--
   Thanks for sending a pull request!
   
   Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
     2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
   -->
   
   ### _Why are the changes needed?_
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you add a feature, you can talk about the use case of it.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   
   
   ### _How was this patch tested?_
   - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
   
   - [ ] Add screenshots for manual tests if appropriate
   
   - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874355055


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$runner -ml" #| s"grep $tag"
       try {
         pb.lineStream_!.headOption
       } catch {

Review Comment:
   we should not suppress the error msg since the runner is not null



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128431728

   ```
   - JpsApplicationOperation with spark local mode *** FAILED ***
     response._1 was false Failed to terminate: 11656 org.apache.spark.launcher.Main org.apache.spark.deploy.SparkSubmit --class org.apache.kyuubi.engine.spark.SparkSQLEngine --conf spark.kyuubi.engine.operation.log.dir.root=target/engine_operation_logs --conf spark.kyuubi.session.idle.timeout=PT3M --conf spark.abc=f41c3b50-1025-49c4-b1a4-3f4e01e91345 --conf spark.kyuubi.testing=true --conf spark.kyuubi.frontend.bind.host=localhost --conf spark.master=local --conf spark.kyuubi.metrics.json.location=/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/metrics --conf spark.kyuubi.operation.log.dir.root=target/server_operation_logs --proxy-user runner /home/runner/work/incubator-kyuubi/incubator-kyuubi/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar, due to Nonzero exit code: 1 (JpsApplicationOperationSuite.scala:87)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128706558

   please update the PR desc


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874862450


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   > Is there some possibility, such as the engine startup is slow, more than 10s, or the kill command fails?
   It seems difficult for JpsApplicationOperation to handle it.
   



##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   > Is there some possibility, such as the engine startup is slow, more than 10s, or the kill command fails?
   
   It seems difficult for JpsApplicationOperation to handle it.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874438446


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,12 +81,11 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)
+      assert(response._1, response._2)
+      assert(response._2 startsWith "Succeeded to terminate:")
     }
 
-    val response = jps.killApplicationByTag(id)
-    assert(response._1, response._2)
-    assert(response._2 startsWith "Succeeded to terminate:")
-
     val desc2 = jps.getApplicationInfoByTag(id)

Review Comment:
   thanks, make sense



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] cfmcgrady commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874878489


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   I think the `JpsApplicationOperation.getApplicationInfoByTag` should make sure the started JVM application is `org.apache.spark.deploy.SparkSubmit`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874286591


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$jps -ml" #| s"grep $tag"

Review Comment:
   Sure,  let me refactor it to.
   ```
       runner =
         try {
           jps.!!
           jps
         } catch {
           case _: Throwable => null
         }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874357341


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$runner -ml" #| s"grep $tag"
       try {
         pb.lineStream_!.headOption
       } catch {

Review Comment:
   let me remove the catch block and have a try.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128396648

   seems still flaky .. 
   
   ```
   - JpsApplicationOperation with spark local mode *** FAILED ***
     response._1 was false APPLICATION_NOT_FOUND (JpsApplicationOperationSuite.scala:87)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128535646

   ```
   SparkOperationProgressSuite:
   2022-05-17 07:19:51,657 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", "onMismatch"
   2022-05-17 07:19:51,660 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", "onMismatch"
   - test operation progress *** FAILED ***
     The code passed to eventually never returned normally. Attempted 329 times over 25.002496456 seconds. Last failure message: checkFlag1 was true, but checkFlag2 was false. (SparkOperationProgressSuite.scala:55)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874357870


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$runner -ml" #| s"grep $tag"
       try {
         pb.lineStream_!.headOption
       } catch {

Review Comment:
   ```
   
   scala> val pb = "jps -ml" #| s"grep 123 456"
   val pb: scala.sys.process.ProcessBuilder =  ( [jps, -ml] #| [grep, 123, 456] )
   
   scala> pb.lineStream_!.headOption
             ^
          warning: method lineStream_! in trait ProcessBuilder is deprecated (since 2.13.0): use lazyLines_!
   grep: 456: No such file or directory
   java.io.IOException: Pipe closed
           at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:260)
           at java.io.PipedInputStream.receive(PipedInputStream.java:226)
           at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
           at scala.sys.process.BasicIO$.loop$1(BasicIO.scala:263)
           at scala.sys.process.BasicIO$.transferFullyImpl(BasicIO.scala:269)
           at scala.sys.process.BasicIO$.transferFully(BasicIO.scala:250)
           at scala.sys.process.ProcessImpl$PipeThread.runloop(ProcessImpl.scala:181)
           at scala.sys.process.ProcessImpl$PipeSource.go$1(ProcessImpl.scala:197)
           at scala.sys.process.ProcessImpl$PipeSource.run(ProcessImpl.scala:200)
   val res0: Option[String] = None
   
   scala> 
   ```
   
   It seems that we do not need catch the exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128459034

   > ```
   > kill: (11656): No such process
   > - JpsApplicationOperation with spark local mode *** FAILED ***
   >   response._1 was false Failed to terminate: 11656 org.apache.spark.launcher.Main org.apache.spark.deploy.SparkSubmit --class org.apache.kyuubi.engine.spark.SparkSQLEngine --conf spark.kyuubi.engine.operation.log.dir.root=target/engine_operation_logs --conf spark.kyuubi.session.idle.timeout=PT3M --conf spark.abc=f41c3b50-1025-49c4-b1a4-3f4e01e91345 --conf spark.kyuubi.testing=true --conf spark.kyuubi.frontend.bind.host=localhost --conf spark.master=local --conf spark.kyuubi.metrics.json.location=/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/metrics --conf spark.kyuubi.operation.log.dir.root=target/server_operation_logs --proxy-user runner /home/runner/work/incubator-kyuubi/incubator-kyuubi/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar, due to Nonzero exit code: 1 (JpsApplicationOperationSuite.scala:87)
   > ```
   
   It seems that the process should not terminate itself before be killed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874284749


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$jps -ml" #| s"grep $tag"

Review Comment:
   shall we use runner here？



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128342300

   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#2669](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a6a8633) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/a163f3a86b8ab4a0b27e9d7b534d82cbd105b95a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a163f3a) will **decrease** coverage by `0.07%`.
   > The diff coverage is `100.00%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #2669      +/-   ##
   ============================================
   - Coverage     64.29%   64.21%   -0.08%     
     Complexity       82       82              
   ============================================
     Files           385      385              
     Lines         18656    18673      +17     
     Branches       2529     2531       +2     
   ============================================
   - Hits          11994    11991       -3     
   - Misses         5515     5535      +20     
     Partials       1147     1147              
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...apache/kyuubi/engine/JpsApplicationOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvSnBzQXBwbGljYXRpb25PcGVyYXRpb24uc2NhbGE=) | `83.87% <100.00%> (ø)` | |
   | [...rg/apache/kyuubi/engine/trino/TrinoStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS10cmluby1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL3RyaW5vL1RyaW5vU3RhdGVtZW50LnNjYWxh) | `66.66% <0.00%> (-4.60%)` | :arrow_down: |
   | [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `83.60% <0.00%> (-1.64%)` | :arrow_down: |
   | [.../org/apache/kyuubi/jdbc/hive/KyuubiConnection.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhpdmUtamRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3l1dWJpL2pkYmMvaGl2ZS9LeXV1YmlDb25uZWN0aW9uLmphdmE=) | `4.02% <0.00%> (-0.11%)` | :arrow_down: |
   | [.../org/apache/kyuubi/config/KyuubiReservedKeys.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpUmVzZXJ2ZWRLZXlzLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...e/kyuubi/server/KyuubiTBinaryFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpVEJpbmFyeUZyb250ZW5kU2VydmljZS5zY2FsYQ==) | `84.37% <0.00%> (ø)` | |
   | [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `96.73% <0.00%> (+0.11%)` | :arrow_up: |
   | [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `84.90% <0.00%> (+0.62%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a163f3a...a6a8633](https://codecov.io/gh/apache/incubator-kyuubi/pull/2669?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] pan3793 commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128301236

   Also cc @cxzl25 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128722956

   > please update the PR desc
   
   updated
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874357870


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,7 +50,7 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
+      val pb = s"$runner -ml" #| s"grep $tag"
       try {
         pb.lineStream_!.headOption
       } catch {

Review Comment:
   ```
   
   scala> val pb = "jps -ml" #| s"grep 123 456"
   val pb: scala.sys.process.ProcessBuilder =  ( [jps, -ml] #| [grep, 123, 456] )
   
   scala> pb.lineStream_!.headOption
             ^
          warning: method lineStream_! in trait ProcessBuilder is deprecated (since 2.13.0): use lazyLines_!
   grep: 456: No such file or directory
   java.io.IOException: Pipe closed
           at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:260)
           at java.io.PipedInputStream.receive(PipedInputStream.java:226)
           at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
           at scala.sys.process.BasicIO$.loop$1(BasicIO.scala:263)
           at scala.sys.process.BasicIO$.transferFullyImpl(BasicIO.scala:269)
           at scala.sys.process.BasicIO$.transferFully(BasicIO.scala:250)
           at scala.sys.process.ProcessImpl$PipeThread.runloop(ProcessImpl.scala:181)
           at scala.sys.process.ProcessImpl$PipeSource.go$1(ProcessImpl.scala:197)
           at scala.sys.process.ProcessImpl$PipeSource.run(ProcessImpl.scala:200)
   val res0: Option[String] = None
   
   scala> 
   ```
   
   It seems that we do not need catch the exception, it will not throw exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128459379

   the UT passed.
   
   Let me re-run them again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] cfmcgrady commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874431588


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,12 +81,11 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)
+      assert(response._1, response._2)
+      assert(response._2 startsWith "Succeeded to terminate:")
     }
 
-    val response = jps.killApplicationByTag(id)
-    assert(response._1, response._2)
-    assert(response._2 startsWith "Succeeded to terminate:")
-
     val desc2 = jps.getApplicationInfoByTag(id)

Review Comment:
   Since we kill the application with signal `SIGTERM`, we should surround L90-L93 with `eventually` too.
   
   [SIGTERM vs SIGKILL: What's the Difference?](https://linuxhandbook.com/sigterm-vs-sigkill/)
   ```
   SIGTERM gracefully kills the process whereas SIGKILL kills the process immediately.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874854697


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   thanks for the comments @cxzl25
   cc @cfmcgrady 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] cfmcgrady commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874878489


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   I think the `JpsApplicationOperation. getApplicationInfoByTag` should make sure the started JVM application is `org.apache.spark.deploy.SparkSubmit`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] cfmcgrady commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874884981


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   also cc @yaooqinn 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128566800

   The UT passed except above `SparkOperationProgressSuite`.
   
   <img width="978" alt="image" src="https://user-images.githubusercontent.com/6757692/168765034-dc436ab0-6bf2-4651-9a3b-1e9c3f146e59.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874536222


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,12 +49,8 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
-      try {

Review Comment:
   According to the comments, it seems not needed.
   
   <img width="687" alt="image" src="https://user-images.githubusercontent.com/6757692/168768500-d4b4fea1-46b4-4e8f-9368-28ac18d7d7dc.png">
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874545091


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,12 +49,8 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
-      try {

Review Comment:
   don't trust it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874547306


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,12 +49,8 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
-      try {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128464534

   > > ```
   > > kill: (11656): No such process
   > > - JpsApplicationOperation with spark local mode *** FAILED ***
   > >   response._1 was false Failed to terminate: 11656 org.apache.spark.launcher.Main org.apache.spark.deploy.SparkSubmit --class org.apache.kyuubi.engine.spark.SparkSQLEngine --conf spark.kyuubi.engine.operation.log.dir.root=target/engine_operation_logs --conf spark.kyuubi.session.idle.timeout=PT3M --conf spark.abc=f41c3b50-1025-49c4-b1a4-3f4e01e91345 --conf spark.kyuubi.testing=true --conf spark.kyuubi.frontend.bind.host=localhost --conf spark.master=local --conf spark.kyuubi.metrics.json.location=/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/metrics --conf spark.kyuubi.operation.log.dir.root=target/server_operation_logs --proxy-user runner /home/runner/work/incubator-kyuubi/incubator-kyuubi/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar, due to Nonzero exit code: 1 (JpsApplicationOperationSuite.scala:87)
   > > ```
   > 
   > It seems that the process should not terminate itself before being killed.
   
   ```
   22/05/17 05:00:38 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   2022-05-17 05:00:38.915 INFO util.SignalRegister: Registering signal handler for TERM
   2022-05-17 05:00:38.917 INFO util.SignalRegister: Registering signal handler for HUP
   2022-05-17 05:00:38.917 INFO util.SignalRegister: Registering signal handler for INT
   2022-05-17 05:00:39.285 INFO conf.HiveConf: Found configuration file null
   2022-05-17 05:00:39.543 INFO spark.SparkContext: Running Spark version 3.1.3
   2022-05-17 05:00:39.621 INFO resource.ResourceUtils: ==============================================================
   2022-05-17 05:00:39.621 INFO resource.ResourceUtils: No custom resources configured for spark.driver.
   2022-05-17 05:00:39.622 INFO resource.ResourceUtils: ==============================================================
   2022-05-17 05:00:39.622 INFO spark.SparkContext: Submitted application: org.apache.kyuubi.engine.spark.SparkSQLEngine
   2022-05-17 05:00:39.680 INFO resource.ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
   2022-05-17 05:00:39.683 INFO resource.ResourceProfile: Limiting resource is cpu
   2022-05-17 05:00:39.690 INFO resource.ResourceProfileManager: Added ResourceProfile id: 0
   2022-05-17 05:00:39.776 INFO spark.SecurityManager: Changing view acls to: runner
   2022-05-17 05:00:39.776 INFO spark.SecurityManager: Changing modify acls to: runner
   2022-05-17 05:00:39.776 INFO spark.SecurityManager: Changing view acls groups to: 
   2022-05-17 05:00:39.777 INFO spark.SecurityManager: Changing modify acls groups to: 
   2022-05-17 05:00:39.777 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(runner); groups with view permissions: Set(); users  with modify permissions: Set(runner); groups with modify permissions: Set()
   2022-05-17 05:00:40.202 INFO util.Utils: Successfully started service 'sparkDriver' on port 43437.
   2022-05-17 05:00:40.269 INFO spark.SparkEnv: Registering MapOutputTracker
   2022-05-17 05:00:40.375 INFO spark.SparkEnv: Registering BlockManagerMaster
   2022-05-17 05:00:40.413 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
   2022-05-17 05:00:40.414 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
   2022-05-17 05:00:40.419 INFO spark.SparkEnv: Registering BlockManagerMasterHeartbeat
   2022-05-17 05:00:40.438 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-f7e1c27f-c2ec-4752-b9fd-5a02a842acf1
   2022-05-17 05:00:40.472 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MiB
   2022-05-17 05:00:40.503 INFO spark.SparkEnv: Registering OutputCommitCoordinator
   2022-05-17 05:00:40.615 INFO util.log: Logging initialized @4628ms to org.sparkproject.jetty.util.log.Slf4jLog
   2022-05-17 05:00:40.755 INFO server.Server: jetty-9.4.40.v20210413; built: 2021-04-13T20:42:42.668Z; git: b881a572662e1943a14ae12e7e1207989f218b74; jvm 1.8.0_332-b09
   2022-05-17 05:00:40.851 INFO server.Server: Started @4864ms
   2022-05-17 05:00:41.059 INFO server.AbstractConnector: Started ServerConnector@6bfdb014{HTTP/1.1, (http/1.1)}{localhost:42741}
   2022-05-17 05:00:41.059 INFO util.Utils: Successfully started service 'SparkUI' on port 42741.
   2022-05-17 05:00:41.102 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@367795c7{/jobs,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.106 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@75699e35{/jobs/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.109 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4aeaadc1{/jobs/job,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.112 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7daa61f3{/jobs/job/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.115 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6e4ea0bd{/stages,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.125 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@78f9ed3e{/stages/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.126 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b0964b2{/stages/stage,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.128 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@64040287{/stages/stage/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.128 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6f89f665{/stages/pool,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.129 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4925f4f5{/stages/pool/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.130 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3a43d133{/storage,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.130 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5f2afe62{/storage/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.131 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@28782602{/storage/rdd,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.143 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@68105edc{/storage/rdd/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.147 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@38b972d7{/environment,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.147 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3935e9a8{/environment/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.148 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5b56b654{/executors,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.149 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@534243e4{/executors/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.149 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@470a9030{/executors/threadDump,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.151 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27494e46{/executors/threadDump/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.166 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e411d81{/static,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.168 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39ab59f8{/,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.172 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@111610e6{/api,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.179 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4beddc56{/jobs/job/kill,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.180 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1b812421{/stages/stage/kill,null,AVAILABLE,@Spark}
   2022-05-17 05:00:41.182 INFO ui.SparkUI: Bound SparkUI to localhost, and started at http://localhost:42741
   2022-05-17 05:00:41.204 INFO spark.SparkContext: Added JAR file:/home/runner/work/incubator-kyuubi/incubator-kyuubi/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar at spark://localhost:43437/jars/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar with timestamp 1652763639524
   2022-05-17 05:00:41.500 INFO executor.Executor: Starting executor ID driver on host localhost
   2022-05-17 05:00:41.528 INFO executor.Executor: Using REPL class URI: spark://localhost:43437/classes
   2022-05-17 05:00:41.555 INFO executor.Executor: Fetching spark://localhost:43437/jars/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar with timestamp 1652763639524
   2022-05-17 05:00:41.619 INFO client.TransportClientFactory: Successfully created connection to localhost/127.0.0.1:43437 after 42 ms (0 ms spent in bootstraps)
   2022-05-17 05:00:41.631 INFO util.Utils: Fetching spark://localhost:43437/jars/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar to /tmp/spark-fa12c304-8837-445c-864c-e2a3522e8f69/userFiles-1df90a08-5586-4f46-9271-562e051c5790/fetchFileTemp2804306972146769908.tmp
   2022-05-17 05:00:41.737 INFO executor.Executor: Adding file:/tmp/spark-fa12c304-8837-445c-864c-e2a3522e8f69/userFiles-1df90a08-5586-4f46-9271-562e051c5790/kyuubi-spark-sql-engine_2.12-1.6.0-SNAPSHOT.jar to class loader
   2022-05-17 05:00:41.761 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 40877.
   2022-05-17 05:00:41.761 INFO netty.NettyBlockTransferService: Server created on localhost:40877
   2022-05-17 05:00:41.763 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
   2022-05-17 05:00:41.780 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, localhost, 40877, None)
   2022-05-17 05:00:41.785 INFO storage.BlockManagerMasterEndpoint: Registering block manager localhost:40877 with 366.3 MiB RAM, BlockManagerId(driver, localhost, 40877, None)
   2022-05-17 05:00:41.792 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, localhost, 40877, None)
   2022-05-17 05:00:41.793 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, localhost, 40877, None)
   2022-05-17 05:00:42.341 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@776802b0{/metrics/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:42.993 INFO internal.SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/work/runner/spark-warehouse/').
   2022-05-17 05:00:42.994 INFO internal.SharedState: Warehouse path is 'file:/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/work/runner/spark-warehouse/'.
   2022-05-17 05:00:43.080 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6c796cc1{/SQL,null,AVAILABLE,@Spark}
   2022-05-17 05:00:43.081 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1cb7936c{/SQL/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:43.083 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7123be6c{/SQL/execution,null,AVAILABLE,@Spark}
   2022-05-17 05:00:43.086 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77a2aa4a{/SQL/execution/json,null,AVAILABLE,@Spark}
   2022-05-17 05:00:43.172 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30c0d731{/static/sql,null,AVAILABLE,@Spark}
   2022-05-17 05:00:50.995 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 2.3.7 using Spark classes.
   2022-05-17 05:00:51.266 INFO conf.HiveConf: Found configuration file null
   2022-05-17 05:00:52.449 INFO session.SessionState: Created local directory: /tmp/runner
   2022-05-17 05:00:52.488 INFO session.SessionState: Created HDFS directory: /tmp/hive/runner/4b4bfd19-ddc4-456f-b07a-6f0905e56c93
   2022-05-17 05:00:52.513 INFO session.SessionState: Created local directory: /tmp/runner/4b4bfd19-ddc4-456f-b07a-6f0905e56c93
   2022-05-17 05:00:52.540 INFO session.SessionState: Created HDFS directory: /tmp/hive/runner/4b4bfd19-ddc4-456f-b07a-6f0905e56c93/_tmp_space.db
   2022-05-17 05:00:52.576 INFO client.HiveClientImpl: Warehouse location for Hive client (version 2.3.7) is file:/home/runner/work/incubator-kyuubi/incubator-kyuubi/kyuubi-server/target/work/runner/spark-warehouse/
   2022-05-17 05:00:54.465 WARN conf.HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist
   2022-05-17 05:00:54.466 WARN conf.HiveConf: HiveConf of name hive.stats.retries.wait does not exist
   2022-05-17 05:00:54.466 INFO metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
   2022-05-17 05:00:54.538 INFO metastore.ObjectStore: ObjectStore, initialize called
   2022-05-17 05:00:54.947 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
   2022-05-17 05:00:54.952 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
   2022-05-17 05:00:58.994 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
   2022-05-17 05:01:03.444 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
   2022-05-17 05:01:03.460 INFO metastore.ObjectStore: Initialized ObjectStore
   2022-05-17 05:01:03.643 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
   2022-05-17 05:01:03.643 WARN metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore runner@10.1.0.178
   2022-05-17 05:01:03.717 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
   2022-05-17 05:01:04.185 INFO metastore.HiveMetaStore: Added admin role in metastore
   2022-05-17 05:01:04.187 INFO metastore.HiveMetaStore: Added public role in metastore
   2022-05-17 05:01:04.300 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
   2022-05-17 05:01:04.597 INFO metastore.HiveMetaStore: 0: get_all_functions
   2022-05-17 05:01:04.598 INFO HiveMetaStore.audit: ugi=runner	ip=unknown-ip-addr	cmd=get_all_functions	
   2022-05-17 05:01:04.732 INFO metastore.HiveMetaStore: 0: get_database: default
   2022-05-17 05:01:04.732 INFO HiveMetaStore.audit: ugi=runner	ip=unknown-ip-addr	cmd=get_database: default	
   2022-05-17 05:01:04.787 INFO metastore.HiveMetaStore: 0: get_databases: *
   2022-05-17 05:01:04.787 INFO HiveMetaStore.audit: ugi=runner	ip=unknown-ip-addr	cmd=get_databases: *	
   2022-05-17 05:01:05.401 INFO codegen.CodeGenerator: Code generated in 211.927261 ms
   2022-05-17 05:01:05.467 INFO codegen.CodeGenerator: Code generated in 7.722799 ms
   2022-05-17 05:01:05.518 INFO util.ThreadUtils: SparkSQLSessionManager-exec-pool: pool size: 100, wait queue size: 100, thread keepalive time: 60000 ms
   2022-05-17 05:01:05.522 INFO operation.SparkSQLOperationManager: Service[SparkSQLOperationManager] is initialized.
   2022-05-17 05:01:05.524 INFO session.SparkSQLSessionManager: Service[SparkSQLSessionManager] is initialized.
   2022-05-17 05:01:05.524 INFO spark.SparkSQLBackendService: Service[SparkSQLBackendService] is initialized.
   2022-05-17 05:01:05.565 INFO spark.SparkTBinaryFrontendService: Initializing SparkTBinaryFrontend on localhost:40031 with [9, 999] worker threads
   2022-05-17 05:01:05.566 INFO spark.SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is initialized.
   2022-05-17 05:01:05.566 INFO spark.SparkSQLEngine: Service[SparkSQLEngine] is initialized.
   2022-05-17 05:01:05.580 INFO operation.SparkSQLOperationManager: Service[SparkSQLOperationManager] is started.
   2022-05-17 05:01:05.580 INFO session.SparkSQLSessionManager: Service[SparkSQLSessionManager] is started.
   2022-05-17 05:01:05.580 INFO spark.SparkSQLBackendService: Service[SparkSQLBackendService] is started.
   2022-05-17 05:01:05.580 INFO spark.SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is started.
   2022-05-17 05:01:05.580 INFO spark.SparkSQLEngine: Service[SparkSQLEngine] is started.
   2022-05-17 05:01:05.594 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13cc3984{/kyuubi,null,AVAILABLE,@Spark}
   2022-05-17 05:01:05.594 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5625ba2{/kyuubi/json,null,AVAILABLE,@Spark}
   2022-05-17 05:01:05.597 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4bd1b07d{/kyuubi/session,null,AVAILABLE,@Spark}
   2022-05-17 05:01:05.598 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fafd099{/kyuubi/session/json,null,AVAILABLE,@Spark}
   2022-05-17 05:01:05.613 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2512155e{/kyuubi/stop,null,AVAILABLE,@Spark}
   2022-05-17 05:01:05.616 INFO spark.SparkSQLEngine: 
       Spark application name: org.apache.kyuubi.engine.spark.SparkSQLEngine
             application ID:  local-1652763641274
             application web UI: http://localhost:42741
             master: local
             version: 3.1.3
             driver: [cpu: 1, mem: 1g]
             executor: [cpu: 2, mem: 1g, maxNum: 2]
       Start time: Tue May 17 05:00:39 UTC 2022
       
       User: runner (shared mode: USER)
       State: STARTED
       
   
   ````
   
   the engine log looks pretty normal


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128567925

   trigger re-run again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] cxzl25 commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
cxzl25 commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874831326


##########
kyuubi-server/src/test/scala/org/apache/kyuubi/engine/JpsApplicationOperationSuite.scala:
##########
@@ -81,16 +81,18 @@ class JpsApplicationOperationSuite extends KyuubiFunSuite {
       assert(desc1.contains("id"))
       assert(desc1("name").contains(id))
       assert(desc1("state") === "RUNNING")
+      val response = jps.killApplicationByTag(id)

Review Comment:
   The root cause of this problem is that `spark-submit` will generate two processes.
   First start `org.apache.spark.launcher.Main org.apache.spark.deploy.SparkSubmit` to generate submission parameters,
   Then start `org.apache.spark.deploy.SparkSubmit` to execute the main logic.
   
   So the call to `getApplicationInfoByTag` is successful, it sees process 1, but sometimes `killApplicationByTag` happens to be executed when the two processes are being switched. At this time, jps cannot see the process of the relevant tag.
   
   Now add in the eventually block and it seems to work ok.
   
   Is there some possibility, such as the engine startup is slow, more than 10s, or the kill command fails?
   
   However, after this investigation, the next time we encounter this case, it should be quickly fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874520226


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,12 +49,8 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
-      try {

Review Comment:
   shall we keep it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128341485

   It seems that this PR can not fix the issue.
   
   It is confused that, we can get engineId when invoking `getApplicationInfoByTag` but fail to get engineId when invoking `killApplicationByTag`.
   ```
       assert(jps.isSupported(builder.clusterManager()))
       eventually(Timeout(10.seconds)) {
         val desc1 = jps.getApplicationInfoByTag(id)
         assert(desc1.contains("id"))
         assert(desc1("name").contains(id))
         assert(desc1("state") === "RUNNING")
       }
   
       val response = jps.killApplicationByTag(id)
       assert(response._1, response._2)
       assert(response._2 startsWith "Succeeded to terminate:")
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128539530

   >```
   >SparkOperationProgressSuite:
   >2022-05-17 07:19:51,657 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", >"onMismatch"
   >2022-05-17 07:19:51,660 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", >"onMismatch"
   >- test operation progress *** FAILED ***
   > The code passed to eventually never returned normally. Attempted 329 times over 25.002496456 seconds. Last failure >message: checkFlag1 was true, but checkFlag2 was false. (SparkOperationProgressSuite.scala:55)
   >```
   
   cc @wForget 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on code in PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#discussion_r874533241


##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala:
##########
@@ -49,12 +49,8 @@ class JpsApplicationOperation extends ApplicationOperation {
     if (runner == null) {
       None
     } else {
-      val pb = "jps -ml" #| s"grep $tag"
-      try {

Review Comment:
   ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei commented on pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2669:
URL: https://github.com/apache/incubator-kyuubi/pull/2669#issuecomment-1128790872

   re-run again and succeed.
   
   Thanks for your review, merging to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [incubator-kyuubi] turboFei closed pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode

Posted by GitBox <gi...@apache.org>.
turboFei closed pull request #2669: [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode
URL: https://github.com/apache/incubator-kyuubi/pull/2669


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org