You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/10/18 15:52:11 UTC
[GitHub] [dolphinscheduler] frouhi opened a new issue, #12427: Enable or disable a task using a global param condition, without a default branch
frouhi opened a new issue, #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Currently, the only way we can enable/disable a specific task via a condition on global params is using a switch. However, a switch requires a default branch. My current workaround is having a dummy task as a default, but this makes the DAG unnecessary cluttered. It would be nice if either the default branch for switch was optional, or some other kind of task could support this use case.
### Use case
Let’s say I have a cleanup_task, and I want it to execute only if the global param cleanup is true. Currently, I do the following:
Switch(
name=“cleanup_task_switch”,
condition=SwitchCondition(
condition=“${cleanup} == ‘true’”,
Task=cleanup_task,
),
Default(
task=dummy_task
)
)
I’m looking for a way to eliminate the need for a dummy_task so that I can simply have something like this:
Switch(
name=“cleanup_task_switch”,
condition=SwitchCondition(
condition=“${cleanup} == ‘true’”,
Task=cleanup_task,
),
)
(as evident from my example, I'm generating my DAGs using python).
### Related issues
A related discussion: https://github.com/apache/dolphinscheduler/discussions/12426
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1369322929
Hi @frouhi are you still here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1541201338
> Sorry about the delay! This task completely slipped under my radar.
>
> my team ended up not adopting dolphin (although, personally, I think its amazing, and hope to use it at some extend in the future)! but that meant I was pulled in a different direction, and haven't had time to work on this .
>
> if someone has time to pick this one up, I'd appreciate it! otherwise, I'll try to wrap it up by mid March.
>
> Thanks for understanding.
Well, it is ok. could you share what tool do you choose in the end? and why do you thing it is better than dolphinscheduler and your team want to choose it instead of dolphinscheduler?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1297945287
Do you need any help @frouhi? Are you still WIP in this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] DarkAssassinator commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
DarkAssassinator commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1282629459
+1 i think this is a good feature
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1314609595
>
Hi @frouhi, FYI in https://github.com/apache/dolphinscheduler/pull/12779 we just separate our Python API code in to an independent repo, named https://github.com/apache/dolphinscheduler-sdk-python, which mean we should both add a PR to https://github.com/apache/dolphinscheduler for java code, and to https://github.com/apache/dolphinscheduler-sdk-python for python one
And you can see more detail from https://github.com/apache/dolphinscheduler-sdk-python/pull/9 and https://github.com/apache/dolphinscheduler/pull/12871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1284248161
Good feature, we can make the `Default` as optional.
```
if(conditionA) {
go taskA
return
}
if(conditionB) {
go taskB
return
}
if(Default != null) {
go Default
return
}
```
cc @caishunfeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] frouhi commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
frouhi commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1298573355
Thanks for following up @zhongjiajie,
I'm still waiting for approval from my organization to commit to this repo (this is an IP related requirement in the company I work for).
Can you maybe give me another week, if it doesn't come through I'll reach out to you. When it comes through, I think I have a general idea for the Java code causing this issue, but I'll confirm the details with you.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] frouhi commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by "frouhi (via GitHub)" <gi...@apache.org>.
frouhi commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1427136057
Sorry about the delay! This task completely slipped under my radar.
my team ended up not adopting dolphin (although, personally, I think its amazing, and hope to use it at some extend in the future)! but that meant I was pulled in a different direction, and haven't had time to work on this .
if someone has time to pick this one up, I'd appreciate it! otherwise, I'll try to wrap it up by mid March.
Thanks for understanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1312379715
> Thanks for following up @zhongjiajie,
>
> I'm still waiting for approval from my organization to commit to this repo (this is an IP related requirement in the company I work for).
>
> Can you maybe give me another week, if it doesn't come through I'll reach out to you. When it comes through, I think I have a general idea for the Java code causing this issue, but I'll confirm the details with you.
>
> Thanks
Hi @frouhi How's it going?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1282751706
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] frouhi commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
frouhi commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1314707229
> >
>
>
>
> Hi @frouhi, FYI in https://github.com/apache/dolphinscheduler/pull/12779 we just separate our Python API code in to an independent repo, named https://github.com/apache/dolphinscheduler-sdk-python, which mean we should both add a PR to https://github.com/apache/dolphinscheduler for java code, and to https://github.com/apache/dolphinscheduler-sdk-python for python one
>
>
>
> And you can see more detail from https://github.com/apache/dolphinscheduler-sdk-python/pull/9 and https://github.com/apache/dolphinscheduler/pull/12871
Got it. Thanks for letting me know. Will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1284236591
@frouhi looking good, I see you want to do by yourself. As far as I know, the limited of switch task should have default branch is in switch task parameter valid function, and it is in Java codebase, so it is mean this is not only the python code change for this PR. If you need any help, please ping me in GitHub. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1284776561
> Good feature, we can make the `Default` as optional.
>
> ```
> if(conditionA) {
> go taskA
> return
> }
>
> if(conditionB) {
> go taskB
> return
> }
>
> if(Default != null) {
> go Default
> return
> }
> ```
>
> cc @caishunfeng
Looks good to me, if the default branch is empty and not other branchs match, the switch task will be an end task of this edge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1322875109
> > Hi @frouhi, FYI in #12779 we just separate our Python API code in to an independent repo, named https://github.com/apache/dolphinscheduler-sdk-python, which mean we should both add a PR to https://github.com/apache/dolphinscheduler for java code, and to https://github.com/apache/dolphinscheduler-sdk-python for python one
> > And you can see more detail from [apache/dolphinscheduler-sdk-python#9](https://github.com/apache/dolphinscheduler-sdk-python/pull/9) and #12871
>
> Got it. Thanks for letting me know. Will do.
Thanks for your pay attention to DolphinScheduler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] frouhi commented on issue #12427: Enable or disable a task using a global param condition, without a default branch
Posted by GitBox <gi...@apache.org>.
frouhi commented on issue #12427:
URL: https://github.com/apache/dolphinscheduler/issues/12427#issuecomment-1314706248
> Hi @frouhi How's it going?
Still pending :( I'll try to give an update here soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org