You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2023/03/15 07:08:39 UTC
[kyuubi] branch branch-1.7 updated: [KYUUBI #4519] Update metadata when batch application state changed
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.7 by this push:
new 382ed8879 [KYUUBI #4519] Update metadata when batch application state changed
382ed8879 is described below
commit 382ed8879aadba43e7b66024ea440c295acb4382
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Wed Mar 15 15:08:11 2023 +0800
[KYUUBI #4519] Update metadata when batch application state changed
### _Why are the changes needed?_
Kyuubi Server should update the metadata store when the batch application state changes, otherwise the peers see the outdated application state.
The following inconsistent happened on a dual Kyuubi Server deployment
```
23/03/15 11:15:36 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: null
23/03/15 11:15:41 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: null, appStatus: null
23/03/15 11:15:46 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:15:51 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:15:56 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:01 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:06 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:11 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:16 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:21 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:26 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:31 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:37 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:42 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:47 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:52 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes #4519 from pan3793/batch-metadata.
Closes #4519
6b9b3e706 [Cheng Pan] Update metadata when batch application state changed
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
(cherry picked from commit 7b8ca1235c842accb81f195bc0dc9db006cbbf8a)
Signed-off-by: Cheng Pan <ch...@apache.org>
---
.../src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala | 1 +
1 file changed, 1 insertion(+)
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala
index 883afcf58..3cbb16907 100644
--- a/kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala
+++ b/kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala
@@ -300,6 +300,7 @@ class BatchJobSubmission(
val newApplicationStatus = currentApplicationInfo
if (newApplicationStatus.map(_.state) != _applicationInfo.map(_.state)) {
_applicationInfo = newApplicationStatus
+ updateBatchMetadata()
info(s"Batch report for $batchId, ${_applicationInfo}")
}
}