You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/03/19 03:35:45 UTC
[GitHub] [incubator-dolphinscheduler] wangchangliang1008 opened a new issue #2231: [QUESTION] The same task start twice(任务重复启动)
wangchangliang1008 opened a new issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231
**Installation machine distribution(安装机器分布)**
master: master1、master2
worker: worker1、worker2
alertServer: dolphin1
apiServer: dolphin1
**Describe the question**
I have created 200+ tasks to start at 1:00 a.m., and about 60 tasks will run repeatedly every day
After observation, master1 and master2 will start a workflow instance respectively, and the interval between the two instances is 1-5 seconds
(我创建了200+个任务在凌晨1点定时启动，每天大约会有60个任务重复运行
经过观察，master1和master2会分别会启动一个工作流实例，两个实例启动的间隔时间在1-5秒内)
**Which version of DolphinScheduler:**
-[1.2.0]
**Additional context**
Related log description(相关日志描述)
in master1/master2 ：dophinscheduler-master-2020-03-XX_XX.0.log
no error message
find one command id:10021,type:SCHEDULER
...
...
submit task : mytask state SUBMITED_SUCCESS complete, instance id 7867 state: RUNNING_EXEUTION
**Requirement or improvement
How can I ensure that only one instance is generated for each task？THX！
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] wangchangliang1008 commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
wangchangliang1008 commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-602226616
Set two or more masters, open at least 400 instances at the same time, may be repeated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] gabrywu commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
gabrywu commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605556116
The PR https://github.com/apache/incubator-dolphinscheduler/pull/1866 may help you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605559006
the issue: #1863 describe this problem, but nobody met this problem, so I forgot it
thanks for @gabrywu 's reminder
update ： PR #1866 maybe can solve this problem， you can try set :
`org.quartz.jobStore.acquireTriggersWithinLock=true `
in quartz.properties, looking forward your deep message, if you solve the issue, please reply the comment
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605549823
We also meet this problem, but can’t find any clue of this, we still digging into it. I think it may be a critical issue, because it make our data duplicate or lost since two process instance are running same time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] gabrywu edited a comment on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
gabrywu edited a comment on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605556116
The PR may help you https://github.com/apache/incubator-dolphinscheduler/pull/1866
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605560148
Thanks for your quick reply, we will try this configuration, and keep watching if it works
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605550379
I noticed this problem, I will discssion it with developers in the community, and if you find any clue, please share, thx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-601712830
very valuable issue
do you think how to reproduce this problem？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-605553286
In our situation, the version is [1.1.0], and cluster has 2 master and 3 worker
We found that the duplicate process instance
1. Has two row in table `t_tds_process_instance` with different id
2. Those process are shedule by two different master, and in master log, we found this log each `org.apache.dolphinscheduler.server.master.runner.MasterSchedulerThread:[114] - find one command: id: xxxx, type: SCHEDULER`, the command id is different, we belive two command had generated
3. we also check the Quartz with table `QRTZ_JOB_DETAILS`, `QRTZ_CRON_TRIGGER`, `QRTZ_TRIGGER`, and only one record can found
So, we think the the problem may happen when the schedule command is generate, but master log can not provide any more information about the job quartz schedule.
If you need any more info, we are happy to share
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-601771961
I review the relate code ， but i can't find the problem
I will try to find the cause, if you have something new to report, please leave a message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #2231:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-682351187
if your problem was solved, please close the issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] gabrywu closed issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
gabrywu closed issue #2231:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] KEN-LJQ commented on issue #2231: [QUESTION] The same task start twice(任务重复启动)
Posted by GitBox <gi...@apache.org>.
KEN-LJQ commented on issue #2231:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2231#issuecomment-616916579
Wonderful, It works, and process instance did not duplicate any more
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org