You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "wangxj3 (via GitHub)" <gi...@apache.org> on 2023/10/31 06:51:27 UTC

[I] [Bug] [MASTER-SERVER] command has possibility of repeated consumption [dolphinscheduler]

wangxj3 opened a new issue, #15099:
URL: https://github.com/apache/dolphinscheduler/issues/15099

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   **This behavior will be extreme and will not normally happen, but there are still risks**
   IF we have 3 master ,master-A ,master-B ,master-C ,and the thisMasterSlot are 0.1.2,now the masterCount is 3.
   THe DB has more command ,and one of them id is 3.
   master-A is runing the code between MasterSchedulerBootstrap.176 and ProcessServiceImpl.344(code version 3.2.0),master-A read the command 3%3 = 0,master-A  execute the comand that id is 3;
   master-B is runing the code before MasterSchedulerBootstrap.173(code version 3.2.0);
   master-C is down suddenly;
   master-B's masterCount from 3 trun to 2,and reaad the command ,3%2 = 1,this thisMasterSlot =1,so master-B execute the comand that id is 3;
   The command is runing repeated.
   
   **这个情况会比较极端,一般情况不会发生,但依旧存在风险**
   假设有三台master,A、B、C,thisMasterSlot分别为0、1、2
   此时command有多条数据,其中第一条id为3
   master-A运行到MasterSchedulerBootstrap.176-ProcessServiceImpl.344行之间(3.2.0版本),master-A 读取comandid为15,id(3)%3=0,master-A消费id为3的command
   master-B运行到MasterSchedulerBootstrap.173行之前(3.2.0版本)
   master-C在运行过程中挂掉。masterCount从3变为2
   master-B继续向下运行两行,A此时没有到ProcessServiceImpl.344行之后(id为3的command还在数据库中)
   此时,master-A的masterCount为3,thisMasterSlot为0
   master-B的masterCount为2,thisMasterSlot为1,command的最新id还是3,3%2=1,master-B,消费id为3的command。
   command重复消费
   
   
   ### What you expected to happen
   
   I don't hava the logs
   
   ### How to reproduce
   
   read the code
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.2.x
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [MASTER-SERVER] command has possibility of repeated consumption [dolphinscheduler]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15099:
URL: https://github.com/apache/dolphinscheduler/issues/15099#issuecomment-1786550922

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   **This behavior will be extreme and will not normally happen, but there are still risks**
   IF we have 3 master ,master-A ,master-B ,master-C ,and the thisMasterSlot are 0.1.2,now the masterCount is 3.
   THe DB has more command ,and one of them id is 3.
   master-A is runing the code between MasterSchedulerBootstrap.176 and ProcessServiceImpl.344(code version 3.2.0),master-A read the command 3%3 = 0,master-A  execute the comand that id is 3;
   master-B is runing the code before MasterSchedulerBootstrap.173(code version 3.2.0);
   master-C is down suddenly;
   master-B's masterCount from 3 trun to 2,and reaad the command ,3%2 = 1,this thisMasterSlot =1,so master-B execute the comand that id is 3;
   The command is runing repeated.
   
   **这个情况会比较极端,一般情况不会发生,但依旧存在风险**
   假设有三台master,A、B、C,thisMasterSlot分别为0、1、2
   此时command有多条数据,其中第一条id为3
   master-A运行到MasterSchedulerBootstrap.176-ProcessServiceImpl.344行之间(3.2.0版本),master-A 读取comandid为15,id(3)%3=0,master-A消费id为3的command
   master-B运行到MasterSchedulerBootstrap.173行之前(3.2.0版本)
   master-C在运行过程中挂掉。masterCount从3变为2
   master-B继续向下运行两行,A此时没有到ProcessServiceImpl.344行之后(id为3的command还在数据库中)
   此时,master-A的masterCount为3,thisMasterSlot为0
   master-B的masterCount为2,thisMasterSlot为1,command的最新id还是3,3%2=1,master-B,消费id为3的command。
   command重复消费
   
   
   ### What you expected to happen
   
   I don't hava the logs
   
   ### How to reproduce
   
   read the code
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.2.x
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [MASTER-SERVER] command has possibility of repeated consumption [dolphinscheduler]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15099:
URL: https://github.com/apache/dolphinscheduler/issues/15099#issuecomment-2369846264

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [MASTER-SERVER] command has possibility of repeated consumption [dolphinscheduler]

Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on issue #15099:
URL: https://github.com/apache/dolphinscheduler/issues/15099#issuecomment-1790752908

   Hi @wangxj3 , you can check the comments in `WorkflowExecuteContextFactory.createWorkflowInstance`, we use transaction to make sure command will be consumed only once.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org