You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/05/06 10:27:10 UTC
[GitHub] [incubator-dolphinscheduler] dukeshiny opened a new issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
dukeshiny opened a new issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611
When I use sqoop import data into hive,the dolphinscheduler log shows success,but the task instance shows failed!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624579352
> Fellow Worker log.
> /dolphinscheduler/install/logs
It seems that has a error log in dolphinscheduler-worker.log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624589639
Modify the yarn.application.status.address of the /opt/dolphinscheduler/conf/common.properties file so that you can just restart the worker.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] jzmq commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
jzmq commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-632559868
DS version : 1.2.1
Hadoop version: hdp 2.7.3 installed with ambari
maybe yarn resources web analysis error
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-625041921
If you are sure that it is not a sqoop problem but a DS problem, you can try to change the log level of the DS worker to debug. If not, you can only look at the source code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624627629
> Modify the yarn.application.status.address of the /opt/dolphinscheduler/conf/common.properties file so that you can just restart the worker.
After I set yarn.resourcemanager.ha.rm.ids=cdh201,cdh202,another error happens,even if the sqoop task succeeded!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny edited a comment on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny edited a comment on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624638057
> The worker log ?
> It doesn't look like a yarn problem. At present, Sqoop's support does not seem to be very good. It is recommended that you try using a shell to see if the error is reported.
The dolphinscheduler-worker.log shows this log.The yarn problem disappears！
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624581764
# if it is a single resourcemanager, you only need to configure one host name. If it is resourcemanager HA, the default configuration is fine.
singleYarnIp="ark1"
I find this config in install.sh ,my yarn is HA,so is this right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624606326
> Modify the yarn.application.status.address of the /opt/dolphinscheduler/conf/common.properties file so that you can just restart the worker.
Now my common.properties has “yarn.application.status.address=http://ark1:8088/ws/v1/cluster/apps/%s
” ，my resourcemanager is HA,I think should change yarn.resourcemanager.ha.rm.ids=cdh201,cdh202,
am I right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624969645
This seems to be a problem with your sqoop command. It is recommended that you go to the full log of your sqoop command on MAPREDUCE. Generally there will be a complete sqoop execution log.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624633539
The worker log ?
It doesn't look like a yarn problem. At present, Sqoop's support does not seem to be very good. It is recommended that you try using a shell to see if the error is reported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624584158
> It is best to configure both
OK,by the way ,if I change this singleYarnIp="cdh201",should I execute the install.sh shell？Or I just need to restart dolphinscheduler?Thank you!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny edited a comment on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny edited a comment on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624581764
if it is a single resourcemanager, you only need to configure one host name. If it is resourcemanager HA, the default configuration is fine.
singleYarnIp="ark1"
I find this config in install.sh ,my yarn is HA,so is this right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] jzmq commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
jzmq commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-631965172
I met the same error , here the error message
` [taskAppId=TASK-14-45-114]:[189] - Conversion = ';'
java.util.UnknownFormatConversionException: Conversion = ';'
at java.util.Formatter.checkText(Formatter.java:2579)
at java.util.Formatter.parse(Formatter.java:2565)
at java.util.Formatter.format(Formatter.java:2501)
at java.util.Formatter.format(Formatter.java:2455)
at java.lang.String.format(String.java:2940)
at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.isSuccessOfYarnState(AbstractCommandExecutor.java:410)
at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.updateState(AbstractCommandExecutor.java:241)
at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:170)
at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105)
at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[ERROR] 2020-05-21 16:13:40.973 - [taskAppId=TASK-14-45-114]:[107] - shell task failure
java.lang.RuntimeException: process error . exitCode is : -1
at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:190)
at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105)
at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[ERROR] 2020-05-21 16:13:40.973 org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread:[153] - task scheduler failure
java.lang.RuntimeException: process error . exitCode is : -1
at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:190)
at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105)
at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO] 2020-05-21 16:13:40.974 - [taskAppId=TASK-14-45-114]:[262] - cancel process: 28845
[INFO] 2020-05-21 16:13:40.986 org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread:[161] - task instance id : 114,task final status : FAILURE`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624584958
Only restart worker-server.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] itbasketplayer commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
itbasketplayer commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-625356170
please check the config of your cloudera-manager
'yarn.resourcemanager.max-completed-applications'
if too small, the appIds's state will not return 'SUCCESS'
then shell task's state is FAILURE.
you can add a checking use historyserver,but you have to check the config:mapreduce.jobhistory.max-age-ms first
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624608187
I found this issue can solve this error
https://github.com/apache/incubator-dolphinscheduler/issues/2491
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624582751
It is best to configure both
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624638057
> The worker log ?
> It doesn't look like a yarn problem. At present, Sqoop's support does not seem to be very good. It is recommended that you try using a shell to see if the error is reported.
The dolphinscheduler-worker.log shows this log.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] Eights-Li commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
Eights-Li commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-722908013
please check worker logs for this sqoop job. when sqoop job finished, ds-worker will check the yarn app status through yarn apis. So you should check the yarn conf in dolphinscheduler. single yarn ip or yarn HA ips in dolphinscheduler/conf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-626100683
> please check the config of your cloudera-manager
> 'yarn.resourcemanager.max-completed-applications'
> if too small, the appIds's state will not return 'SUCCESS'
> then shell task's state is FAILURE.
>
> you can add a checking use historyserver,but you have to check the config:mapreduce.jobhistory.max-age-ms first
The 'yarn.resourcemanager.max-completed-applications' is 10000.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-625016163
> This seems to be a problem with your sqoop command. It is recommended that you go to the full log of your sqoop command on MAPREDUCE. Generally there will be a complete sqoop execution log.
I have checked the mr sqoop execution log,there is no error.The result of the sqoop task succeeded,but the dolphinscheduler web task instance shows failed!How can I solve this problem?
Thanks a lot!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] Eights-Li closed issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
Eights-Li closed issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-632419046
> I met the same error , here the error message
> ` [taskAppId=TASK-14-45-114]:[189] - Conversion = ';' java.util.UnknownFormatConversionException: Conversion = ';' at java.util.Formatter.checkText(Formatter.java:2579) at java.util.Formatter.parse(Formatter.java:2565) at java.util.Formatter.format(Formatter.java:2501) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.isSuccessOfYarnState(AbstractCommandExecutor.java:410) at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.updateState(AbstractCommandExecutor.java:241) at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:170) at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105) at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [ERROR] 2020-05-21 16:13:40.973 - [taskAppId=TASK-14-45-114]:[107] - shell task failure java.lang.RuntimeException: process error . exitCode is : -1 at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:190) at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105) at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [ERROR] 2020-05-21 16:13:40.973 org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread:[153] - task scheduler failure java.lang.RuntimeException: process error . exitCode is : -1 at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:190) at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:105) at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:147) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [INFO] 2020-05-21 16:13:40.974 - [taskAppId=TASK-14-45-114]:[262] - cancel process: 28845 [INFO] 2020-05-21 16:13:40.986 org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread:[161] - task instance id : 114,task final status : FAILURE`
Which version do you use? It's better to take a screenshot of some more logs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny edited a comment on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny edited a comment on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624638057
> The worker log ?
> It doesn't look like a yarn problem. At present, Sqoop's support does not seem to be very good. It is recommended that you try using a shell to see if the error is reported.
I just use the shell with sqoop command !The dolphinscheduler-worker.log shows this log.The yarn problem disappears！
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] 743294668 commented on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
743294668 commented on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-624576301
Fellow Worker log.
/dolphinscheduler/install/logs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dukeshiny edited a comment on issue #2611: Sqoop import data into hive succeeded,but task instance shows failed!
Posted by GitBox <gi...@apache.org>.
dukeshiny edited a comment on issue #2611:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2611#issuecomment-626100683
> please check the config of your cloudera-manager
> 'yarn.resourcemanager.max-completed-applications'
> if too small, the appIds's state will not return 'SUCCESS'
> then shell task's state is FAILURE.
>
> you can add a checking use historyserver,but you have to check the config:mapreduce.jobhistory.max-age-ms first
The 'yarn.resourcemanager.max-completed-applications' is 10000.
But my yarn tasks list has 67760 ,is that mean I should increate the yarn.resourcemanager.max-completed-applications??
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org