You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/30 03:43:31 UTC
[GitHub] [dolphinscheduler] cadl opened a new issue, #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
cadl opened a new issue, #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Now SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY only work on the same version process instances(query
via `queryByProcessDefineCodeAndProcessDefinitionVersionAndStatusAndNextId` at [ProcessServiceImpl.java](https://github.com/apache/dolphinscheduler/blob/13af2adfc150e6e16ce07c5d1abeeb25f9cd9340/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessServiceImpl.java#L321))
In some cases, the process definition updating and running are independent. Maybe want the process definition run serially, instead of a specific version.
### Use case
Maybe we can add one configuration `serial_scope` to control the scope of `SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY`?
If the value is `process_definition_version`, the process instances with the same version are executed serially.
If the value is `process_definition`, all process instance is executed serially.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] WangJPLeo commented on issue #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
Posted by GitBox <gi...@apache.org>.
WangJPLeo commented on issue #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288#issuecomment-1259106660
> After some research, I found the behavior of serial execution was changed at #9869.
>
> In the first version #6185, serial execution works on all process instances.
>
> After the #9869, SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY only works on the same version process instances.
>
> To fix this issue, I have 2 ideas:
>
> 1. Recover the serial execution logic back to [[Feature-#6268][server-master] Serial execte process #6185](https://github.com/apache/dolphinscheduler/pull/6185). But also need to fix [[Bug-BE] Task flow definition cannot execute serial task flow while parallel serial alternate running. #9868](https://github.com/apache/dolphinscheduler/issues/9868), such as applying the [commit](https://github.com/apache/dolphinscheduler/pull/9869/commits/8b6363d2d38b961105202e0c4b405fc511f2ad56)
> 2. Based on 1, add `VERSIONING_SERIAL_WAIT/VERSIONING_SERIAL_DISCARD/VERSIONING_SERIAL_PRIORITY`.
>
> I prefer the first one.
>
> Can you give me any advice?
>
> cc @SbloodyS @WangJPLeo @caishunfeng
I agree wit @caishunfeng, i think the real role of a workflow definition is to provide a portal to create workflow instances, each version may be slightly different or the core content may be different, and each version is a separate entity. So using a certain version of the strategy to affect other versions, it does not seem appropriate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] cadl commented on issue #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
Posted by GitBox <gi...@apache.org>.
cadl commented on issue #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288#issuecomment-1141640694
After some research, I found the behavior of serial execution was changed at #9869.
In the first version #6185, serial execution works on all process instances.
After the #9869, SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY only works on the same version process instances.
To fix this issue, I have 2 ideas:
1. Recover the serial execution logic back to #6185. But also need to fix #9868, such as applying the [commit](https://github.com/apache/dolphinscheduler/pull/9869/commits/8b6363d2d38b961105202e0c4b405fc511f2ad56)
2. Based on 1, add `VERSIONING_SERIAL_WAIT/VERSIONING_SERIAL_DISCARD/VERSIONING_SERIAL_PRIORITY`.
I prefer the first one.
Can you give me any advice?
cc @SbloodyS @WangJPLeo @caishunfeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288#issuecomment-1140659490
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://join.slack.com/t/asf-dolphinscheduler/shared_invite/zt-omtdhuio-_JISsxYhiVsltmC5h38yfw) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] cadl commented on issue #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
Posted by GitBox <gi...@apache.org>.
cadl commented on issue #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288#issuecomment-1334708421
> > After some research, I found the behavior of serial execution was changed at #9869.
> > In the first version #6185, serial execution works on all process instances.
> > After the #9869, SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY only works on the same version process instances.
> > To fix this issue, I have 2 ideas:
> >
> > 1. Recover the serial execution logic back to [[Feature-#6268][server-master] Serial execte process #6185](https://github.com/apache/dolphinscheduler/pull/6185). But also need to fix [[Bug-BE] Task flow definition cannot execute serial task flow while parallel serial alternate running. #9868](https://github.com/apache/dolphinscheduler/issues/9868), such as applying the [commit](https://github.com/apache/dolphinscheduler/pull/9869/commits/8b6363d2d38b961105202e0c4b405fc511f2ad56)
> > 2. Based on 1, add `VERSIONING_SERIAL_WAIT/VERSIONING_SERIAL_DISCARD/VERSIONING_SERIAL_PRIORITY`.
> >
> > I prefer the first one.
> > Can you give me any advice?
> > cc @SbloodyS @WangJPLeo @caishunfeng
>
> I agree wit @caishunfeng, i think the real role of a workflow definition is to provide a portal to create workflow instances, each version may be slightly different or the core content may be different, and each version is a separate entity. So using a certain version of the strategy to affect other versions, it does not seem appropriate.
Hi @WangJPLeo @caishunfeng , thanks for your replies.
Looks like in current codebase, once processDefine is updated, the `version` will change. If each version is a separate entity, then the decision of whether to update the `version` should be left to the user. Otherwise there is no way for the user to keep the processDefine as a "same entity" while being "harmless updated"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #10288: [Feature] Make SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY work across multiple process instance versions
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #10288:
URL: https://github.com/apache/dolphinscheduler/issues/10288#issuecomment-1258946981
> After some research, I found the behavior of serial execution was changed at #9869.
>
> In the first version #6185, serial execution works on all process instances.
>
> After the #9869, SERIAL_WAIT/SERIAL_DISCARD/SERIAL_PRIORITY only works on the same version process instances.
>
> To fix this issue, I have 2 ideas:
>
> 1. Recover the serial execution logic back to [[Feature-#6268][server-master] Serial execte process #6185](https://github.com/apache/dolphinscheduler/pull/6185). But also need to fix [[Bug-BE] Task flow definition cannot execute serial task flow while parallel serial alternate running. #9868](https://github.com/apache/dolphinscheduler/issues/9868), such as applying the [commit](https://github.com/apache/dolphinscheduler/pull/9869/commits/8b6363d2d38b961105202e0c4b405fc511f2ad56)
> 2. Based on 1, add `VERSIONING_SERIAL_WAIT/VERSIONING_SERIAL_DISCARD/VERSIONING_SERIAL_PRIORITY`.
>
> I prefer the first one.
>
> Can you give me any advice?
>
> cc @SbloodyS @WangJPLeo @caishunfeng
I don't think we should support cross version execute strategy, it will make users misunderstand earsily.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org