You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/01/26 12:03:41 UTC
[GitHub] [incubator-dolphinscheduler] guodeliang opened a new issue #4578: [Bug][service] task success but node status failed
guodeliang opened a new issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578
node :
machine: **host1**
type: shell
content: ssh -t user@host2 -i ~/private_key " bash /home/vastdb/code/script/test.sh"
echo $?
### reason：
test.sh file contains hive query command, which will be executed on host2, host2 has an isolated hadoop system, **host1** can't direct visit the system.
>worker_log
`
org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s
[ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known
java.net.UnknownHostException: yarnIp1: Name or service not known
`
> console log
`
[INFO] 2021-01-26 19:39:49.332 - [taskAppId=TASK-5-50-70]:[127] - -> MapReduce Jobs Launched:
Stage-Stage-1: Map: 34 Cumulative CPU: 608.01 sec HDFS Read: 4108290122 HDFS Write: 93488199 SUCCESS
Stage-Stage-2: Map: 1 Cumulative CPU: 46.29 sec HDFS Read: 93527219 HDFS Write: 93091919 SUCCESS
Total MapReduce CPU Time Spent: 10 minutes 54 seconds 300 msec
OK
Time taken: 130.29 seconds
`
**Expected behavior**
supply a option to stop tracking hive sql status in shell type node
**Screenshots**
work log and console log above
**Which version of Dolphin Scheduler:**
-[1.3.4]
**Requirement or improvement**
- current tracking logic has some bug, exec script on ssh remote host is an example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-771715273
> org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s [ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known java.net.UnknownHostException: yarnIp1: Name or service not known
here is the root cause, could host1 visit the http://{host2}:8088/ ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong edited a comment on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
dailidong edited a comment on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-771715273
> org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s [ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known java.net.UnknownHostException: yarnIp1: Name or service not known
here is the root cause, could host1 visit the http://{host2}:8088/ ? if application id shows in the shell task, DS will to get the yarn run result through restful api provided by the yarn resource manager. obivously, DS worker server can't get the result because your config is wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] guodeliang commented on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
guodeliang commented on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-793647175
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-771715273
> org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s [ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known java.net.UnknownHostException: yarnIp1: Name or service not known
here is the root cause, could host1 visit the http://{host2}:8088/ ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] guodeliang commented on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
guodeliang commented on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-793598499
> > org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s [ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known java.net.UnknownHostException: yarnIp1: Name or service not known
>
> here is the root cause, could host1 visit the http://{host2}:8088/ ? if application id shows in the shell task, DS will to get the yarn run result through restful api provided by the yarn resource manager. obivously, DS worker server can't get the result because your common.properties configuration is wrong
host1 can't directly visit the http://{host2}:8088/ ?, host1 can login to host2 by ssh method,
so I can't modify the file common.properties.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong edited a comment on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
dailidong edited a comment on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-771715273
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong edited a comment on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
dailidong edited a comment on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-771715273
> org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : http://yarnIp1:8088/ws/v1/cluster/apps/%s [ERROR] 2021-01-26 19:39:49.371 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - yarnIp1: Name or service not known java.net.UnknownHostException: yarnIp1: Name or service not known
here is the root cause, could host1 visit the http://{host2}:8088/ ? if application id shows in the shell task, DS will to get the yarn run result through restful api provided by the yarn resource manager. obivously, DS worker server can't get the result because your common.properties configuration is wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] guodeliang commented on issue #4578: [Bug][service] task success but node status failed
Posted by GitBox <gi...@apache.org>.
guodeliang commented on issue #4578:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4578#issuecomment-793610768
the sh script file content : for example
sh "ssh -tt xxx@10.xx.xx.xx -p 10022 -i ./conf/private.key < vip_video_meta_info_base.sh"
I can login any host in script file, which is isolated by the ssh gateway, DS can get the child process,
but the restful api url can't visit on ds system
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org