You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/11/06 08:56:53 UTC
[GitHub] [incubator-dolphinscheduler] 15177592207 opened a new issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
15177592207 opened a new issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035
Basic environment:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] Eights-Li commented on issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
Posted by GitBox <gi...@apache.org>.
Eights-Li commented on issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-723518740
may be the wrong yarn conf setting of dolphinscheduler will cause this problem
Please check the worker-server log and check the yarn conf of dolphinscheduler
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 15177592207 commented on issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
Posted by GitBox <gi...@apache.org>.
15177592207 commented on issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-722962962
Basic environment:
spark 2.3.4
dolphinscheduler1.3.2
hdp3.1.0
Using dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails.
status of dolphinscheduler
An icon ×-interpreted as failure
Status of spark log:
[INFO] 2020-11-06 01:13:40.138 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:40 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:41.140 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:40 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: UNDEFINED
tracking URL: http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:41 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:42.143 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:42 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:43.145 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:43 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:44.147 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:44 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:45.149 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:45 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:46.151 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:46 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:47.153 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:47 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:48.155 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:48 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:49.159 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:49 INFO Client: Application report for application_1604404915071_0018 (state: FINISHED)
[INFO] 2020-11-06 01:13:49.226 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:49 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: SUCCEEDED
tracking URL: http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:49 INFO ShutdownHookManager: Shutdown hook called
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory /tmp/spark-761e77c5-b590-48a5-a8ad-df5061db6c8b
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory /tmp/spark-320a73bb-f26d-4861-b4b1-127e28850f91
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 15177592207 commented on issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
Posted by GitBox <gi...@apache.org>.
15177592207 commented on issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-723712353
Thank you very much，The problem has been solved。
------------------ 原始邮件 ------------------
发件人: "apache/incubator-dolphinscheduler" <notifications@github.com>;
发送时间: 2020年11月8日(星期天) 上午9:43
收件人: "apache/incubator-dolphinscheduler"<incubator-dolphinscheduler@noreply.github.com>;
抄送: "梁大大"<1052748788@qq.com>;"Author"<author@noreply.github.com>;
主题: Re: [apache/incubator-dolphinscheduler] Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails (#4035)
may be the wrong yarn conf setting of dolphinscheduler will cause this problem
Please check the worker-server log and check the yarn conf of dolphinscheduler
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-722980634
Check the worker log for errors
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen closed issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
Posted by GitBox <gi...@apache.org>.
xingchun-chen closed issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org