You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/04/01 13:49:05 UTC
[GitHub] [incubator-dolphinscheduler] oceanos opened a new issue #2356:
[BUG] task is running endlessly
oceanos opened a new issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356
**Describe**
When the worker goes down while the task is running, the task will run endlessly.
**Reproduce**
1. run processA on node1（specify the worker group）
2. shut down node1 (or node1 was crashed) （script or jar in the temporary directory was not generated）
3. restart node1
4. task will run endlessly
5. no button will be enabled on process instance list page after you click one of them
**Expected behavior**
1, Endless task should be able to fail (or other ops) manually.
2, Or endless task should be able to fail (or other ops) automatically.
**Screenshots**
Task Is Running Endlessly：
**Version of Dolphin Scheduler**
-[1.2.0]
**Additional context**
After restart the master，these endless tasks were all fault-tolerance handled.
**Requirement or improvement
- I can't always restart the master if I have these problems. So i think DS should provide some way to get people involved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #2356: [BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #2356:
URL: https://github.com/apache/dolphinscheduler/issues/2356#issuecomment-1004701579
Does this still exists in dev branch @caishunfeng @lenboo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo commented on issue #2356: [BUG]
task is running endlessly
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608269620
we have worker fault tolerance.
when a worker goes down, the master will change the status of the task to "need fault tolerance" , and wait for other workers to re-execute the task.
obviously this mechanism does not work, but i don't know what happened on this case, maybe require more tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] Technoboy- commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608255868
> > This is a default value, for we want to use in server module only. But as development goes, and api module uses the same config as server. so causes the issue.
> > We will refactor quartz in the future. but in current version, you can set it as below in api module。
> > org.quartz.threadPool.threadCount = 1
>
> Your reply has nothing to do with this issue. Do you mean by #2351?
sorry. mail list linked this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] oceanos commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
oceanos commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-610267962
> we have worker fault tolerance.
> when a worker goes down, the master will change the status of the task to "need fault tolerance" , and wait for other workers to re-execute the task.
> obviously this mechanism does not work, but i don't know what happened on this case, maybe require more tests.
Task that has been specified to crashed workers，worker fault tolerance will not work. I think DS should check the status of target worker before executing the task.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] oceanos commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
oceanos commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608201595
> This is a default value, for we want to use in server module only. But as development goes, and api module uses the same config as server. so causes the issue.
> We will refactor quartz in the future. but in current version, you can set it as below in api module。
> org.quartz.threadPool.threadCount = 1
Your reply has nothing to do with this issue. Do you mean by #2351?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [dolphinscheduler] caishunfeng closed issue #2356: [BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
caishunfeng closed issue #2356:
URL: https://github.com/apache/dolphinscheduler/issues/2356
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] Technoboy- commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608198938
This is a default value, for we want to use in server module only. But as development goes, and api module uses the same config as server. so causes the issue.
We will refactor quartz in the future. but in current version, you can set it as below in api module。
org.quartz.threadPool.threadCount = 1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] Technoboy- removed a comment on issue
#2356: [BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
Technoboy- removed a comment on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608198938
This is a default value, for we want to use in server module only. But as development goes, and api module uses the same config as server. so causes the issue.
We will refactor quartz in the future. but in current version, you can set it as below in api module。
org.quartz.threadPool.threadCount = 1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [dolphinscheduler] caishunfeng commented on issue #2356: [BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #2356:
URL: https://github.com/apache/dolphinscheduler/issues/2356#issuecomment-1004796071
Hi @oceanos the failover logic was updated and you can check it again by the latest version.
This issue will be closed, you can reopen it if need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-607273861
thanks for your detail description，we will disscuss it tomorrow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2356:
[BUG] task is running endlessly
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2356: [BUG] task is running endlessly
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2356#issuecomment-608469829
> > > This is a default value, for we want to use in server module only. But as development goes, and api module uses the same config as server. so causes the issue.
> > > We will refactor quartz in the future. but in current version, you can set it as below in api module。
> > > org.quartz.threadPool.threadCount = 1
> >
> >
> > Your reply has nothing to do with this issue. Do you mean by #2351?
>
> sorry. mail list linked this issue.
i am sorry, mislead you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services