You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by GitBox <gi...@apache.org> on 2022/07/17 15:06:26 UTC
[GitHub] [incubator-linkis] CCweixiao opened a new pull request, #2479: [Feature-2228] The ECM kill engine needs to be able to complete the kill of the yarn appid
CCweixiao opened a new pull request, #2479:
URL: https://github.com/apache/incubator-linkis/pull/2479
https://github.com/apache/incubator-linkis/issues/2228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] CCweixiao merged pull request #2479: [Feature-2228] The ECM kill engine needs to be able to complete the kill of the yarn appid
Posted by GitBox <gi...@apache.org>.
CCweixiao merged PR #2479:
URL: https://github.com/apache/incubator-linkis/pull/2479
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] peacewong commented on a diff in pull request #2479: [Feature-2228] The ECM kill engine needs to be able to complete the kill of the yarn appid
Posted by GitBox <gi...@apache.org>.
peacewong commented on code in PR #2479:
URL: https://github.com/apache/incubator-linkis/pull/2479#discussion_r924133139
##########
linkis-engineconn-plugins/shell/src/main/scala/org/apache/linkis/manager/engineplugin/shell/executor/YarnAppIdExtractor.scala:
##########
@@ -116,7 +116,11 @@ class YarnAppIdExtractor extends Thread with Logging{
def addYarnAppIds(yarnAppIds: Array[String]): Unit = {
if (yarnAppIds != null && yarnAppIds.length != 0) {
appIdList.synchronized {
- yarnAppIds.foreach(id => if(!appIdList.contains()) appIdList.add(id))
+ yarnAppIds.foreach(id => if (!appIdList.contains()) {
Review Comment:
Modified to hashset
##########
linkis-engineconn-plugins/shell/src/main/resources/bin/kill-yarn-jobs.sh:
##########
@@ -35,9 +35,15 @@ STATUS_NOT_FOUND="NOT_FOUND"
STATUS_ERR="ERROR"
YARN_APP_STATUS_KILLED="KILLED"
+source $LINKIS_CONF_DIR/linkis-env.sh
+YARN_EXE_PATH="/appcom/Install/hadoop/bin/yarn"
Review Comment:
Modified default value to yarn
##########
linkis-engineconn-plugins/shell/src/main/resources/bin/kill-yarn-jobs.sh:
##########
@@ -35,9 +35,15 @@ STATUS_NOT_FOUND="NOT_FOUND"
STATUS_ERR="ERROR"
YARN_APP_STATUS_KILLED="KILLED"
Review Comment:
can remove this file
##########
linkis-dist/package/sbin/kill-yarn-jobs.sh:
##########
@@ -0,0 +1,259 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+# http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Query and kill unfinished YARN tasks based on yarnID （根据yarnID查询并杀死未结束的yarn任务）
+set -x
+
+declare -A is_unfinish_state=(\
+ ['ACCEPTED']=true \
+ ['RUNNING']=true \
+ ['SUBMITTED']=true \
+)
+
+declare -A kill_status
+KILLED=0
+UNFIN=1
+FIN=2
+NOTFOUND=3
+SENT=4
+ERR=255
+
+STATUS_NOT_FOUND="NOT_FOUND"
+STATUS_ERR="ERROR"
+YARN_APP_STATUS_KILLED="KILLED"
+
+source $LINKIS_CONF_DIR/linkis-env.sh
+YARN_EXE_PATH="/appcom/Install/hadoop/bin/yarn"
Review Comment:
update default value to yarn
##########
linkis-computation-governance/linkis-engineconn-manager/linkis-engineconn-manager-server/src/main/scala/org/apache/linkis/ecm/server/service/impl/DefaultEngineConnKillService.java:
##########
@@ -79,6 +92,94 @@ public EngineStopResponse dealEngineConnStop(EngineStopRequest engineStopRequest
return response;
}
+ private synchronized void killYarnAppIdOfOneEc(EngineConn engineConn) {
+ String engineConnInstance = engineConn.getServiceInstance().toString();
+ logger.info("try to kill yarn app ids in the engine of ({}).", engineConnInstance);
+ String engineLogDir = engineConn.getEngineConnManagerEnv().engineConnLogDirs();
+ final String errEngineLogPath = engineLogDir.concat(File.separator).concat("stderr");
+ logger.info("try to parse the yarn app id from the engine err log file path: {}", errEngineLogPath);
+ Thread t = new Thread(() -> {
+ BufferedReader in = null;
+ try {
+ in = new BufferedReader(new FileReader(errEngineLogPath));
+ String line;
+ String regex = getYarnAppRegexByEngineType(engineConn);
+ if (StringUtils.isBlank(regex)) {
+ return;
+ }
+ Pattern pattern = Pattern.compile(regex);
+ List<String> appIds = new ArrayList<>();
+ while ((line = in.readLine()) != null) {
+ if (StringUtils.isNotBlank(line)) {
+ Matcher mApp = pattern.matcher(line);
+ if (mApp.find()) {
+ String candidate1 = mApp.group(mApp.groupCount());
+ if (!appIds.contains(candidate1)) {
+ appIds.add(candidate1);
+ }
+ }
+ }
+ }
+ if (!appIds.isEmpty()) {
+ String yarnAppKillScriptPath = EngineConnConf.ENGINE_CONN_YARN_APP_KILL_SCRIPTS_PATH().getValue();
+ String[] cmdArr = new String[appIds.size() + 2];
+ cmdArr[0] = "sh";
+ cmdArr[1] = yarnAppKillScriptPath;
+ for (int i = 0; i < appIds.size(); i++) {
+ cmdArr[i + 2] = appIds.get(i);
+ }
+ logger.info("Starting to kill yarn applications " + engineConnInstance + ". Kill Command: " + StringUtils.join(cmdArr, " "));
+ Shell.ShellCommandExecutor exec =
+ new Shell.ShellCommandExecutor(cmdArr, null, null, 600 * 1000L);
+ exec.execute();
+ logger.info("Kill Success! id:" + engineConnInstance + ". msg:" + exec.getOutput());
+ }
+ } catch (IOException ioEx) {
+ if (ioEx instanceof FileNotFoundException) {
+ logger.error("the engine log file {} not found.", errEngineLogPath);
+ } else {
+ logger.error("the engine log file parse failed. the reason is {}", ioEx.getMessage());
+ }
+ } finally {
+ IOUtils.closeQuietly(in);
+ }
+ }, "EngineConnKillYarnAppThread");
+ t.setDaemon(true);
+ t.start();
Review Comment:
should use thread poll. eg:Utils.newCachedThreadPool
##########
linkis-computation-governance/linkis-engineconn-manager/linkis-engineconn-manager-server/src/main/scala/org/apache/linkis/ecm/server/service/impl/DefaultEngineConnKillService.java:
##########
@@ -79,6 +92,94 @@ public EngineStopResponse dealEngineConnStop(EngineStopRequest engineStopRequest
return response;
}
+ private synchronized void killYarnAppIdOfOneEc(EngineConn engineConn) {
+ String engineConnInstance = engineConn.getServiceInstance().toString();
+ logger.info("try to kill yarn app ids in the engine of ({}).", engineConnInstance);
+ String engineLogDir = engineConn.getEngineConnManagerEnv().engineConnLogDirs();
+ final String errEngineLogPath = engineLogDir.concat(File.separator).concat("stderr");
+ logger.info("try to parse the yarn app id from the engine err log file path: {}", errEngineLogPath);
+ Thread t = new Thread(() -> {
+ BufferedReader in = null;
+ try {
+ in = new BufferedReader(new FileReader(errEngineLogPath));
+ String line;
+ String regex = getYarnAppRegexByEngineType(engineConn);
+ if (StringUtils.isBlank(regex)) {
+ return;
+ }
+ Pattern pattern = Pattern.compile(regex);
+ List<String> appIds = new ArrayList<>();
+ while ((line = in.readLine()) != null) {
+ if (StringUtils.isNotBlank(line)) {
+ Matcher mApp = pattern.matcher(line);
+ if (mApp.find()) {
+ String candidate1 = mApp.group(mApp.groupCount());
+ if (!appIds.contains(candidate1)) {
+ appIds.add(candidate1);
+ }
+ }
+ }
+ }
+ if (!appIds.isEmpty()) {
+ String yarnAppKillScriptPath = EngineConnConf.ENGINE_CONN_YARN_APP_KILL_SCRIPTS_PATH().getValue();
+ String[] cmdArr = new String[appIds.size() + 2];
+ cmdArr[0] = "sh";
+ cmdArr[1] = yarnAppKillScriptPath;
+ for (int i = 0; i < appIds.size(); i++) {
+ cmdArr[i + 2] = appIds.get(i);
+ }
+ logger.info("Starting to kill yarn applications " + engineConnInstance + ". Kill Command: " + StringUtils.join(cmdArr, " "));
+ Shell.ShellCommandExecutor exec =
+ new Shell.ShellCommandExecutor(cmdArr, null, null, 600 * 1000L);
Review Comment:
It is recommended to abstract public methods, which can be called by the shell engine. You can refer to:GovernanceUtils.killProcess
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] codecov[bot] commented on pull request #2479: [Feature-2228] The ECM kill engine needs to be able to complete the kill of the yarn appid
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #2479:
URL: https://github.com/apache/incubator-linkis/pull/2479#issuecomment-1186547660
# [Codecov](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2479](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (010102a) into [dev-1.2.0](https://codecov.io/gh/apache/incubator-linkis/commit/86664ec877ce32bf75e81802ee1be3a758b1936e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86664ec) will **decrease** coverage by `0.84%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## dev-1.2.0 #2479 +/- ##
===============================================
- Coverage 16.64% 15.79% -0.85%
Complexity 1105 1105
===============================================
Files 636 668 +32
Lines 19529 20570 +1041
Branches 2770 2929 +159
===============================================
Hits 3250 3250
- Misses 15842 16883 +1041
Partials 437 437
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ver/service/impl/DefaultEngineConnKillService.java](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL3NlcnZpY2UvaW1wbC9EZWZhdWx0RW5naW5lQ29ubktpbGxTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...r/service/impl/DefaultLocalDirsHandleService.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL3NlcnZpY2UvaW1wbC9EZWZhdWx0TG9jYWxEaXJzSGFuZGxlU2VydmljZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...che/linkis/ecm/server/listener/ECMReadyEvent.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL2xpc3RlbmVyL0VDTVJlYWR5RXZlbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [.../linkis/ecm/server/metrics/DefaultECMMetrics.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL21ldHJpY3MvRGVmYXVsdEVDTU1ldHJpY3Muc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [.../server/service/impl/DefaultECMHealthService.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL3NlcnZpY2UvaW1wbC9EZWZhdWx0RUNNSGVhbHRoU2VydmljZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [.../ecm/server/engineConn/DefaultYarnEngineConn.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL2VuZ2luZUNvbm4vRGVmYXVsdFlhcm5FbmdpbmVDb25uLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...ache/linkis/ecm/server/conf/ECMConfiguration.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL2NvbmYvRUNNQ29uZmlndXJhdGlvbi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...er/service/impl/DefaultEngineConnListService.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL3NlcnZpY2UvaW1wbC9EZWZhdWx0RW5naW5lQ29ubkxpc3RTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...service/impl/AbstractEngineConnLaunchService.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbXB1dGF0aW9uLWdvdmVybmFuY2UvbGlua2lzLWVuZ2luZWNvbm4tbWFuYWdlci9saW5raXMtZW5naW5lY29ubi1tYW5hZ2VyLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9lY20vc2VydmVyL3NlcnZpY2UvaW1wbC9BYnN0cmFjdEVuZ2luZUNvbm5MYXVuY2hTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| ... and [23 more](https://codecov.io/gh/apache/incubator-linkis/pull/2479/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86664ec...010102a](https://codecov.io/gh/apache/incubator-linkis/pull/2479?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org