You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/11/06 09:01:44 UTC
[GitHub] [incubator-dolphinscheduler] 15177592207 commented on issue #4035: Use dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails
15177592207 commented on issue #4035:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-722962962
Basic environment:
spark 2.3.4
dolphinscheduler1.3.2
hdp3.1.0
Using dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the task is successfully executed normally, but dolphinscheduler real task execution fails.
status of dolphinscheduler
An icon ×-interpreted as failure
Status of spark log:
[INFO] 2020-11-06 01:13:40.138 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:40 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:41.140 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:40 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: UNDEFINED
tracking URL: http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:41 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:42.143 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:42 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:43.145 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:43 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:44.147 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:44 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:45.149 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:45 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:46.151 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:46 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:47.153 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:47 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:48.155 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:48 INFO Client: Application report for application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:49.159 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:49 INFO Client: Application report for application_1604404915071_0018 (state: FINISHED)
[INFO] 2020-11-06 01:13:49.226 - [taskAppId=TASK-4-4-10]:[121] - -> 20/11/06 01:13:49 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: SUCCEEDED
tracking URL: http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:49 INFO ShutdownHookManager: Shutdown hook called
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory /tmp/spark-761e77c5-b590-48a5-a8ad-df5061db6c8b
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory /tmp/spark-320a73bb-f26d-4861-b4b1-127e28850f91
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org